I'm a firm believer that your IQ isn't an indicator of your success in college or school. Your work ethic and personality are far more predictive of success and happiness. What I mean is that it's important to have a good work ethic and enjoy thinking mathematically and outside of the box, but knowing how to score high on an IQ test won't bring food on the table or help you build relationships with great people in life.
Unless one is in the rare and elite .001% of the population for whom math and physics seems to be relatively effortless, most of us have to constantly acknowledge our shortcomings and grapple with our personal shortcomings. It's how we improve and develop as human beings. I saw a video of Richard Feynman saying he feels 'like a big dumb ape, trying to figure out how to rub two sticks to make a fire' when he's doing physics. So if you're not too smart for Feynman, you're not too smart to never admit you're dumb.
It's true that some people are called prodigies for a reason, but no one is born with all the secrets of the universe in their minds. And furthermore, even a 'mathematical genius' like Terrance Tao is woefully ignorant of subjects outside of his specialty.
My message is that if you want to be happy, try not to compare yourself to other people and run your own race. Chances are that your life circumstances and other inherent differences set your apart from others, and you should embrace them while working hard to make yourself better, smarter, faster, kinder, and etc.