I assume this is a bad thing

(haven't heard the expression before myself)? You're definitely right and I'm not disagreeing with you on anything. I'm just grappling with my personal priorities: do I pay the ridiculous sum just to go to a top university and get a BS in physics that might or might not get me a PhD and might or might not get me a good paying job with no PhD, do I change majors within same university to something more realistic like EE and at least know that I have a much better chance of getting a good paying job, or do I transfer to Stony Brook and have to pay much less (~20K for the two years ill be there + ~60K for the two years I'll have been at Cornell by the time I transfer, if I do choose that) and get a physics degree from there but give up Cornell?
Honestly speaking, after your and others' advice, it seems like transferring might not be the worst of ideas because let's be honest it's just Cornell it isn't Harvard, MIT, or Caltech so it's not like I'm getting an
amazing education. Unfortunately it seems in today's world, prestige is always an issue when applying to top grad schools or when getting an industry job. Staying at Cornell and doing a BS in physics seems the worst of the three choices. Maybe I can dwell on switching to something engineering related within Cornell or go to Stony brook before its too late (even if I transfer I will have already taken a ~60k blow to my future funds because of the past year and the year to come).