A 'theoretical physicist' is a physicist who flogs theories of the physical universe about. Just like in many science and engineering disciplines, mathematics forms an integral part of the preparation to become a theoretical physicist and is useful when one starts to practice the physics.
You ask what is 'theoretical' about the physics. Well, at one time, quantum mechanics and relativity were just theories which a couple of very bright fellows happen to stumble on almost by accident. In time, after the implications of these theories were understood, then experiments could be designed to test their validity.
At one time, much further back in history, the theory of gravitation was toyed with by Isaac Newton, who was able to show that his theory could be used to explain how planets orbited the sun. Many observations of planetary motions had been made by the astronomer Tycho Brahe, and his records were analyzed by Johannes Kepler who developed three laws of planetary motion from this data. It only took Kepler 20 years' work to develop his three laws. By using his theory of gravitation, Newton was able to show mathematically that Kepler's three laws were a consequence of assuming that the force of attraction between two bodies was proportional to the product of their masses and inversely proportional to the square of the distance between their centers. Needless to say, Newton's work took him considerably less time to develop Kepler's laws.
Now, Newton's theories about planetary motion worked quite well and agreed with the observed motion of the planets until astronomical instruments had become better refined and more accurate. In the middle of the nineteenth century, the orbit of mercury was observed to have the location of its perihelion (the point where Mercury is closest to the sun) precess by 43 arc seconds per century, which is an incredibly small amount. This tiny discrepancy in the orbit of mercury could not be explained by Newton's theory.
When Einstein developed his theory of general relativity, one of the first tests the theory was subjected to was analyzing the orbit of Mercury. GR was found to account for the observed discrepancy, and this was powerful proof that Einstein's new theory had some validity. More experiments were performed in later years to continue testing general and special relativity.
http://en.wikipedia.org/wiki/Perihelion_precession_of_Mercury#Perihelion_precession_of_Mercury