If you do that maths with physics option you'd do 75% the maths courses and 25% the physics ones (with 3 of the 6 modules you do in the maths course being 'applied' too). At the end of the 1st year you get the choice of doing the maths course or the physics course.
I used to want to do physics when I was in 6th form but I'm glad I picked maths because there's plenty of applied things to do in maths, and if you plan to do high energy physics, you need to do a lot of maths (even the pure stuff!).
I cannot speak for Oxford, but at Cambridge you'd get no real advantage or disadvantage going for one college over another, because you all attend the same lectures, and the average qualities of supervisors (which supervisors you get depends on your college since most use their own PhD students and fellows) is the same across the colleges. Even in maths, colleges like Trinity doesn't have a vastly superior bunch of supervisors, despite there being some image Trinity is head and shoulders above all the other colleges, it isn't (and I'm not saying that because I go to a different college, I'm in Trinity!).
You'd be much better off picking the college you like the look of, whose location is nice, seems to have a nice social scene (all do really though). Going on an open day or two is a very good idea. If you cannot make open days, just go when you can and have a wander around a few colleges which you like the look of from their prospectuses. Some charge you to walk around the grounds, but if you explain you're looking at colleges with plans to apply you might be let in free. That or ring the college a few days before and ask if you can look round, most are happy to let you do so free.