I've never taken much of an interest to optics, so I can't reccomend anythnig there, I can reccomend a book for thermodynamics though;
Introduction to Thermodynamics and Statistical Mechanics - Keith Stowe
You'll need to know about partial derivatives and the likes for that though but most of it has pretty simple maths.
If you're just starting calculus based physics, general relativity, nuclear physics and modern physics are really too advanced for you atm. You'll need to know about some pretty tricky calculus, differential geometry, tensors, group theory, linear algebra and all kinds of mathematical goodies.
Mathematical Methods in the Physical Sciences - Mary Boas
To get you stared on the maths, but you'll still need to know more advanced material after that, but you should know where to go once you've finished that book.
Most of the sections on GR, nuclear physics and modern physics in introduction books do more harm than good imo.
If you want to start on the mystical journey of relativity and quantum mechanics on any meaningful level, you really need to learn your maths first. I'm a strong supporter of the idea of learning a lot of maths before you even start thinking about applying it to physics.