We all have fears...if there were no chance of failure would there be any challenge?
One thing you have to think about is; how do you define failure? So what if you fail a class, or a test, or an exam - you can try again. You can take the class over, study harder, seek tutoring, figure out why you failed and choose a different method of preparing, etc. The only person who fails absolutely is the person who gives up. Delay is not failure; it is only disappointment. We all disappoint ourselves from time to time, that is how we learn and grow.
If your definition of failure is becoming a Physics Engineering major instead of an Applied Physics major, then maybe you should reassess your definition. What are the little things that make you happy? What are you actually DOING when you are happiest? Those are your internal drivers. In other words, you may view becoming a theoretical physicist with "success," but becoming an experimental physicist with "failure." Physics is a big world. I spent years working in the area of device physics, but now I am back in school to learn other areas of physics. For me nothing is more fun than mucking about in the lab; for others that would be their idea of hell. Find out what YOU actually enjoy doing, and do that! The label "failure" is a label you apply to yourself based on external criteria that may or may not have anything to do with what you are truly all about.