I don't know if hard work is the only requirement. I work as a tutor at my university and mainly handle pre-algebra to calculus I topics. I see many people who are in there every single day working through their College Algebra homework. They are in there, face in the book, asking good questions, having problems, figuring them out, for sometimes 8 hours a day, and they seem thankful when they get a test back that is in the 70's. I absolutely never put that much time into College Algebra of all things, and I would die if I ever got a 70. These are perfectly smart people in many other aspects of life/academics, mathematics is just not one of them.
That's not to say that anyone couldn't accomplish any degree of mathematical proficiency they desired, but it would certainly take some people much more time and involvement to accomplish the same level of understanding, and I think that eventually a point would come when time no longer permits that much "grit".