It depends on what you want to do - even within physical chemistry, there are sub-specialties for which linear algebra, optics, and E&M will have varying levels of utility.
I'm currently a PhD student in physical chemistry. I work in two research groups - one focusing on ultrafast/non-linear...
I'll start out by describing my situation at the present. I'm in my fourth year of undergraduate studies, pursuing a B.S. in Chemistry with a minor in Computer Science. I'd consider myself to have a fairly strong academic profile. My math/science GPA is great, my overall GPA is still good...