I don't know, on what level mathematics and physics education at UK high school is, since I am native German. However, in my opinion every physicist should have read at least some chapters of the famous Feynman lectures, and certainly non-physicists will benefit from doing so, as well.
Depending on how deep you want to go, Landau Lifshitz would be a good choice, although the volumes are really hard. But they give you something that no (to my mind) other textbook gives you: Giving you insight into Landau's brilliant mind, they show you how to create a physical theory. His arguments are always very subtle, for example when the form of the Lagrangian in classical mechanics or the Minkowski metric or the action in relativistic mechanics are derived. It is an intellectual challenge to reflect about these.
As said, these volumes are very hard. A comparably easy piece is "The theoretical minimum" by Leonard Susskind. I have not read it, though, despite some pages that lead me to the conclusion that I should not read it. However, it might be exactly the right book for someone who is interested in (theoretical) physics as a hobby. There are also lectures by Leonard Susskind on YouTube. However, you could find him too slow (because he is), so I suggest to read the book(s).
Here in Germany, the volumes on theoretical physics by Florian Scheck have an excellent reputation. They are hard, but not too hard (such as Landau Lifshitz) and really go into great detail (actually they go further than most other mechanics books such as Landau Lifshitz or Goldstein). At least the mechanics volume has been translated into English (Springer) and could possibly exactly the right book to read for a mathematics student.
By the way: I noticed that the lectures by Arnold Sommerfeld exist in translation, too. These get a true recommendation by me: They could be regarded as obsolete, nowadays, but like the Feynman lectures, they provide you not just with knowledge, but understanding (but on a higher level as the latter ones).
EDIT: Somehow I associated "University physics" with Alonso/Finn. I don't know it, but my prof adores it. And by the way: Don't waste your time with books Halliday or Tipler. They give you nothing but coloured pictures. I don't understand why so many people like them...
