Don't do it for the job prospects, you'll either drop out or hate yourself in a few years if you do.
That's about right. Aside from electromagnetics and semi-conductors, all the physics is so applied that it's hard to see the physics part. What I mean is that it's all devices and signals and very much an extension of E&M physics, but on such a level that it's more of a conceptual leap than the ChemE/ME physics (which is a closer extension of mechanics.) If you like mechanics, go the ChemE/ME route, but if you just like physics, well EE's got lots of it too.