Can I take a different approach?
I think I know enough about how academia works to know that I probably will not have a great shot at being a professor. However, it would kill me to not have at least tried. So right now, I'm trying my best, giving it my all. I am by no means a genius, never have, never will be.
But what is wrong with trying and having a backup plan if it doesn't work out?
Let me also reiterate, I am not trying to convince anyone to keep up with their physics or math studies. Frankly, I don't care either way. But I would like to offer the other perspective.
My plan is to give a PhD a real try, study my butt off and a few things can happen:
1) Get my PhD, no good postdoc offers. Having a PhD was one of my goals, so I accomplished my goal, could not quite accomplish the goal of getting a nice postdoc position, but it's nice to move on and try something else. I gave it an honest try. Plus having a math or physics PhD is pretty damn good. I'm sure you can get employed somewhere or enter a new field (business, finance).
2) Get my PhD, get a nice postdoc offer and the road to tenure track now depends on how I do during my postdoc
3) Not get my PhD, probably get a terminal masters and pursue some other field.
Either one of those I am fine with. I understand how the game works. I try to take on a much more basic view than the I want to pursue math to further mankind's understanding or anything like that. I like doing math. If I can get paid to do math, then I will try like hell for that, if I can't, i'll move on, hopefully to something related to math.
I can understand why there is this "If I'm not the best, then why bother" since all of us have admired the great mathematicians. But in reality, most mathematicians will not be remembered. Most of them will not have a great theorem named after them, most of them will not produce any kind of ground breaking results. This includes PhD's from Harvard, MIT, Princeton, Berkeley, etc. I would say well over 50% of mathematicians and physicists will go by the wayside, never to be mentioned again.
So it's not the worst thing in the world. Plenty of good mathematicians go "unnoticed" over time. Yes as a graduate student or as a professor you will know more mathematicians than an undergrad would, but I think my point is still valid. Not every PhD from Harvard or MIT go onto to become studs, in fact, most do not.
With all that being said, it makes sense to me, to just pursue what I want to pursue and try to take an original view on the subject matter. That to me is the best way to produce a meaningful result.