Did your teacher actually say you are stupid? A teacher should never say such a thing, and not just because it's not nice. It's likely just wrong - there are many ways to be intelligent.
Also, even in just one area, people often don't really know what skills you may have, because they have limited information. People can also be very biased. When I was in high school, my teachers thought that I had a math learning disability. I was shunted into a vocational track where I cleaned ink off printing presses and other practical graphic arts duties. It wasn't until college that I realized what math really is and how good I really could be at it.
To answer your question, it takes both a certain amount of talent and hard work to become good at anything, physics included. I tend to think that, with a certain minimal level of natural ability, hard work is most important. There have been lots of studies done on this for different skills (like chess or playing the violin, for example) and, to summarize and simplify (hopefully not over-simplify!), they say that while hard work may not make you the best in the world, it can get you pretty far.