I understand where your coming from saying the teacher in high school was bad, because most of my high school peers felt the same way regarding our physics teacher.
One thing to note is that odds are, most students weren't interested in the material at all, this makes it very hard to teach...
I'm am undergrad and I originally had no preference between math and physics. This past semester I found out how very different the subjects are. I found math to be much harder because I had no intuition, or physical reality to relate it to. It is very abstract. Physics is a study of nature...