Actually, at the undergraduate level, it doesn't!
There have been similar threads such as this. What you want out of an undergraduate institution are (i) good instructions (ii) number of available courses each semester/quarter (iii) wide selection of classes that you can optionally take by the time you're a senior and (iv) possible research/individual study opportunities.
All of the above are not restricted to only "brand-name" schools. In fact, some of the lesser well-known schools can excel or even surpass those popular schools. Smaller schools, because they don't have that large of a range of research projects, concentrate more on instruction qualities. At the undergraduate level, you want a place that can give you the best instruction on the fundamentals of physics. These fundamentals are the same no matter where you study physics, and chances are, you would be using the same text as those kids in Harvard, Princeton, etc., but with smaller classes and more one-on-one interactions with your instructors.
Don't be fooled by schools touting all the big research projects, bit research grants, etc. Chances are, as an undergraduate, these things would not have any effect on you. You have plenty of opportunity to get involved with those when you start considering for graduate schools.
I hate to continue being tacky and advertise my journal, but I'm in the middle of writing a series of articles titled "So You Want To Be A Physicist". We are at Part 6 now. You may wan to give those a read...
Zz.