I'm in Year 12 in Australia, and doing both a Pure Mathematics and Physics course (alongside Chem and Comp Sci). I find mathematics in its very raw, pure, abstract form much easier and more beautiful than physics. I love proofs. I also have a deep love of philosophy, in topics such as philosophy of consciousness, religion, art.
Interestingly enough, for a long while I wanted to do theoretical physics as a career, however, I have not been doing very well at all with my physics course this year (physics is full of constants, one of which being my mark, at an average 60% very consistently). Compare this to my marks in mathematics, typically hovering just below 90%. I now want to be a Pure Mathematician.
Physics this year has involved only basic mathematics, but admittedly the concepts are difficult. Much of the difficulty in physics, as I see it, is that rigorous proofs of the mathematics used are not presented (at least in my course). There is typically no axiomatic approach, and even very few actual derivations or philosophical discussion on the meaning of various abstractions used (in formulae, etc). All intuition and formality is lost, such that the only measure of skill in physics at this level is dictated by the person's memorisation of techniques and various formulae, to be 'applied' in certain contexts.
I wonder what Calculus-based physics is like?
I am not great at remembering lists of facts: I have difficulty with history because of this very issue. I think abstract reasoning and generality is not necessarily in synch with a more specific problem-solving approach; I am far better at the former, although I constantly have to employ the latter in computer science. This makes computer science harder for me than mathematics, but there are still many parallels, so I am able to do quite well in it.
I feel at this level that much of Physics is disjoint - perhaps only the impression given to me by my course and teacher. I often find real-world application more difficult than a less applied and far more abstract approach, as is typical in pure mathematics.
I will go off on a tangent, and make a possibly contentious statement: Mathematics in pure form is qualitative, Physics in its raw form is quantitative. Can a duality ever be reached? Is this perhaps the underlying cause for many people being good at one but not the other? There are, of course, many exceptions - look at Ed Witten for example.
Just a few thoughts of mine. I'm not sure whether I wish to do first-year university physics anymore, but I'd be glad to hear of others' experiences. Especially since it would be introductory calculus-based physics. I would guess more seasoned mathematicians would find this easier than non-calculus-based physics?