but still I think that an average pure mathematician has a lot better understanding of mathematics than a fairly professional theoretical physicist in general.
Sounds like you might do fairly well at mathematical physics? I mean
of course mathematicians understand mathematics better than theoretical physicists - because that's what they work in and spend all day on.
Can a pure mathematician study the TP courses on his/her own and be a professional theoretical physicist? If yes, what courses should he/she study on his/her own?
I would answer a resounding
NO. Why? Just look at the question - a pure mathematician is a pure mathematician, not a theoretical physicist - now sure, they could in addition also be physicists, but certainly not just by taking some additional courses.
In my world, there is no such thing as a pure mathematician without some equivalent of a PhD in mathematics, and the same goes for theoretical physics. Usually, "equivalent of" could be dropped, but there are some geniuses who can conduct PhD level research in mathematics as youngsters.
So what you are really asking, as per my interpretation of the words, is whether you can get a PhD + significant research depth and experience in pure mathematics and physics by taking some physics courses, and the answer is simply no.
Can a pure mathematician learn some physics? Yes. There are many good books in physics which are geared towards mathematicians, although these usually assume very good command over mathematics.
Another piece of advice - if you want to become a
professional theoretical physicist, you'll need a great deal of depth of thought in certain areas of physics. Consider that some of your cohorts will have spent time both in class, outside, etc reading and working on physics topics. Reading a book helps, but you should spend hours and hours through a long period of time understanding these things, especially if you want to work in them as a professional. Being a professional in something takes like a thousand times the work it does to get a basic class out of the way.
Now if your question is just - can someone doing a math major self-study enough physics to go on to physics grad school and then concentrate in physics to become a professional in physics, that's a different story. Sure. I believe you can learn stuff without classes if you really make sure you get the intuition and think hard about it. But you asked about a
pure mathematician and
theoretical physicist, which are different from math and physics majors respectively.