Personally, I did reasonably well in E&M (and E&M is important to my research, so that's not surprising.) My nightmare class was Stat Mech! And yet, I did reasonably well in QFT later on. It really depends on the individual and your situation. I did poorly in stat mech because I had several other courses at the same time and struggled to keep up in all of them. In contrast QFT was my only course that term. Thus, I got much more out of it.
The common denominator is that your workload will increase. a lot! Any good grad program will make sure to immerse you in physics during your first year. You will be taking 3-4 physics courses concurrently, unlike most undergrad programs where you take 1-2 physics courses a term.