As far as I know, in order to fully understand GR, you might want to take a Tensor Analysis Class, along with a differential geometry class. For QM, i think you basically the mathematics required when you are learning it.
Tensor geometry, diff topology, and functional analysis are very broad subjects. A lot of what you study in a strict maths class you won't really use in physics. Generally the mathematics you need will be taught in the physics class.
Of course, if you have an interest in doing further work/research in mathematical physics then doing the extra maths is very helpful because you may find interesting applications.