This is good to know. Previously my aim was to go to graduate school for theoretical physics, yet the more physics I study the more I feel it lacks rigor, and sometimes even the beauty you find in pure mathematical fields such as Number Theory. To me, mathematics feels closer to the "truth" whatever that may be.
I always regret not being in close enough touch with reality, myself. I am really a physicist at heart, albeit a very mathematical one. In the end, reality ought to matter more than a mathematical fantasy world. The mathematical fantasy world can be fun and applicable to reality, though. Math is overwhelmingly huge; physics is overwhelmingly huge. But math just leaves me wondering what the point of doing something so endless and hard is if it doesn't connect back up to reality at some point. Even if it's interesting. It's easy to argue that mathematicians don't need to worry about applications, but an abstract argument, even if you agree with it completely, is not always completely psychologically convincing. You may not care about these things now, but after doing math, math, math, math, for several years, the issue can come up. At least with physics, you know that you are dealing with reality (probably, you have similar issues in physics).
So while I am interested in learning more mathematical physics, I am also considering research in a pure field such as Algebraic Geometry or Number theory. Would taking some Physically inclined subjects detract from study in fields such as these? I am aware that most grad schools have a breadth component, and I may consider writing a minor thesis in a more physically relevant field (Mathematics of string theory or something similar).
Writing something meaningful about string theory is pretty hard, much more so if you're not a specialist in it. This is what you are up against, and this is just the math side:
http://superstringtheory.com/math/index.html
I know all the math there, except maybe 3 subjects towards the very end, and it would still take me considerable effort to learn any string theory, let alone write something about it.
Then, on the physics side, you have to know a little GR and QFT. QFT is extremely daunting. Picking up string theory on the side while you do a PhD is nearly impossible in my judgement, if you are doing something demanding like algebraic geometry. You have to specialize pretty drastically--that's something I've learned the hard way.
That's what I'm talking about when I say it's harder than you'd think. I have a number theory grad student friend who is interested in interactions between number theory and QFT, though. Seems to work for him. I'm not sure if that's his main research or just a diversion.
Graduate classes are not all uniformly difficult. Depends what university you are at, who the prof is, and so on. It's a good idea to take a couple (and more than a couple if you want to get into one of the top schools) to prepare yourself for grad school.