I'm a junior undergraduate working under a mathematical physicist, so I have some idea of what they do, but certainly a graduate student or practicing mathematical physicist would have better knowledge.
Mathematical physicists tend to reside in math departments rather than physics, and tend to be much more concerned with rigorous argument than theoretical physicists. A lot of them publish papers in pure mathematics as well (though usually it is at least tangentially related to physics).
Take the quantum hall effect for example. In order for it to happen you need a very disordered sample to create Anderson localization and prevent Bragg scattering, and this is mathematically represented by a random potential. The quantum hall effect has been proven to work for certain weak conditions on the potential, but it is an open problem to show that it can happen for any realistic random potential. Mathematical physicists are the type that would try to prove it rigorously. The style in theoretical physics is to present evidence for things happening one way vs. another, and then if the predictions are correct then the theory is usually thought to be true. That doesn't satisfy mathematical physicists though - they will try to prove it is true in a mathematically rigorous way.
Mathematical physicists are also interested in formalism even if there is already more simple machinery that gets the job done. For example, a theorist might not choose to use the differential forms formulation of electrodynamics to do some calculations if he does not have to, but a mathematical physicist might do it just to show that it can be done rigorously (I don't think anyone actually studies electrodynamics anymore, but the differential forms formulation is an example of something that is completely unnecessary for the most part but still of interest to some people).
If you want more information, I highly recommend John Baez's column, This Week in Mathematical Physics. There are hundreds of articles that focus on the mathematical aspects of physics aimed at someone with an advanced undergraduate/beginning graduate student level of knowledge and it is what originally got me interested in mathematical physics. Just a few weeks ago, he was writing about how circuits can be seen as simplicial complexes (also as labelled digraphs) and that analyzing circuits in terms of (co)homology is not so absurd an idea as it sounds! Is it useless, unnecessary formalism? Almost certainly, but that does not mean that it is not interesting!
String theory - THE largest and deepest area combining both physics and mathematics. Algebraic geometry, algebraic topology, differential geometry... all got used in string theory.
There is also this field called "physical mathematics", such as Topological Field Theory, in which you can use physics methods to calculate some topological invariants.