I was asking myself a similar question and still don't have the exact details of what formal degrees I will be getting. But what I am making sure though is that I take advanced classes in both math and physics. So if you're interested in the "cutting-edge" fields of theoretical physics these days regardless of what you major in, you need to try your best to take as much advanced math and physics classes as possible. Here's what you should take beyond the intro-level:
Physics: A year of undergraduate Quantum Mechanics followed by a year of more advanced/graduate-level QM and then a year of Quantum Field Theory (you should start undergrad QM your second year if you want to do this), a year of E&M, at least a semester of advanced Classical Mechanics, Statistical Mechanics and General Relativity each.
Mathematics: ODEs/PDEs of course, Complex Analysis, Advanced Linear Algebra, Abstract Algebra, Real Analysis, General Topology, Algebraic and Differential Topology. Classes on Differential Geometry and Lie Algebras/Representation Theory would also be nice.
So with this rather ambitious list one will no doubt be able to get a Mathematics degree, however for a Physics major most places would require at least 2-3 semesters of lab. So with these if you have time to fit labs in as well, get a Physics degree too. I personally haven't had time to fit in those labs yet and I only have 3 more semesters left and a whole bunch of other classes I would rather take, so I may not get a BS in Physics. Regardless class-wise this should be excellent preparation for graduate school in theoretical physics.
As far as what to apply for in grad school, I'll just say that I've seen quite some people with physics backgrounds who work in mathematics departments but none the other way around.