I thought briefly about becoming a math major. But I decided to major in physics because I liked science fiction when I was growing up, and I was fascinated by space travel, radioactivity, nuclear power, lasers, SETI, gravity, relativity, particle accelerators, time travel, warp drives, transporters, antimatter, electron microscopes, holography, and so on.
I saw that physicists got to build and work with amazing devices. I love the way physics allows us to uncover nature's secrets so that the world is no longer so mysterious, and we can control it to some extent. By the time I was at university I also hoped that physics would answer the whole "meaning of life" question or at least enable us to contact aliens who know more than we do.
As for the whole beauty thing, that is not my motivation, but I do admit I am attracted to equations that can explain nature and in fact even looking at equations is a bit of an emotional experience. Consider Maxwell's equations, for example. Even the symbols are beautiful. Writing them is such a pleasure. I particularly like calculus and differential equations. This is one reason I don't like working with computers even though I must at times. I miss that hands-on experience of writing equations or drawing diagrams with a pen or pencil.
I enjoy pure math to some extent, but I really only care about math because it is the language of physics. I don't like solving puzzles just as an intellectual exercise. In other words in physics I get the thrill that comes from knowledge, but not just any knowledge. It's the knowledge that leads to nuclear power, for example.
To sum up, I want to understand how nature works and at least to some extent get control over it and bend it to my will.