I know I'm not a major Physicist - just someone who deeply wants to learn it. Granted, my two cents here might be null and void in the Academic Department as I myself am desperately seeking Physicist friends to help guide me forward, but I do know a little something about giving up.
Don't back down just because something is challenging. Who cares if you got a D in those classes? If you want to succeed, you will. Just try again. And again, after that, if you need to. It might seem childlike to say something like that, but if you truly are a scientist and are passionate about what it is you're doing, you won't let something like grades stop you. Just do the concept until you get it right.
One thing I notice is that you read the chapter prior to taking the lecture. I don't think this is a good idea at all. For one, it doesn't give your brain ample time to rest and absorb/comprehend the concepts you've just learned. You should either read it first thing when you wake up or the night before (or BOTH), take notes and hilight what you don't understand, and then go to the lectures.
If there are words or rationales that just don't make sense to you - Hilight them. It means that the building blocks are lost. You may have to go back to the basics to understand them, but there's nothing wrong with that. Do what you got to do.
Also, get tutors. If you're struggling, you NEED that one-on-one time, otherwise, you'll never be able to untangle the frustrating knot of learning concepts at your own pace. It might be expensive and it might be difficult as hell to get one, but do it.
I know I'm not qualified to make any presuppositions or assumptions on Physics. But I do know that you don't just give up because something is difficult, hard, or challenging. That's total bullshit. Sure, someone else might be better at it than you - but they had to learn it. They had to learn it and so do you. Maybe the concept was easier for them, but I'm sure you know things that come easier to you than others. Everyone learns differently and everyone learns at different paces. Be patient with yourself and do not give up. If you're not passing a class, take notes and write down why, what, and how you think you'll get better. If you fail a class. Oh well. Take it again.
JUST DO IT!