Well, in my university we start with the theory course in the 1st semester, and electrodynamics is taught in the 3rd semester (Theo 1 is about mathematical methods and some mechanics, including SRT, Theo 2 is analytical mechanics). In the experimental course you have Mechanics and Thermodynamics (Exp. 1) and electrodynamics (Exp. 2) before. I'm not sure about the math course, but I guess that there also some calculus of many independent variables has been taught in the 2nd semester. So theoretical E&M in the 3rd semester seems to work out ok for most students though it's known to be one of the toughest lectures in the entire undergrad curriculum (only followed by Theo 5, statistical mechanics).
My own experience about 25 years ago is similar: We started with theoretical physics in the 3rd semester and E&M was Theo 2 (i.e., in the 4th semester), and it was also the most challenging of all theory lectures in the undergrad. curriculum.
I've always thought since this experience that E&M is taught to early, because you need pretty much all of the vector calculus (even with some glance at tensor calculus, when it comes to the Maxwell stress tensor) machinery, which is a lot of mathematical material to deal with, often for the first time, in parallel with the physics of E&M itself. I think, it would be better to flip the order of subjects in the theory course and teach QM 1 before E&M, because for most of the time in QM 1 you deal with a scalar field (the "wave function") and you basically only need gradients and Laplacians, and you learn the most important techniques for the solution of a partial differential equation (Schrödinger equation) with one scalar field only but you also collect in a natural way also the special functions (another topic, which I found pretty challenging when heard about them at the first time) like spherical harmonics, Bessel functions, spherical Bessel function, Hermite and Laguerre polynoms and "generalized" Fourier series and integrals. This is all stuff you can well use in the E&M lecture.
Of course you can argue also to the opposite: In E&M you deal with a classical theory and the physics concepts are thus much closer to everyday experience than in QM. So you can argue that the traditional order of subjects is better, because you learn all the math needed in QM already in the E&M lecture...