You come to the right place. I earned my bachelors in physics at Rochester, and my doctorate at Colorado.
My Bachelors was 1978 (some time ago), my doctorate was more recent within the last 5 years.
I can personally say you are fortunate. Both schools are excellent in physics, especially quantum optics. I know Colorado is ranked number one in AMO which includes quantum optics, but do not let that fool you, Rochester is excellent in quantum optics. It has Dr. Wolf (one of my teachers), who worked with Max Born. Dr Eberly (another of my teachers) is also extraordinary. I am sorry to say Dr Mandel (a third teacher) has passed away. I expect Rochester's excellence in quantum optics persists, although these teachers are emeritus.
High energy physics at Rochester also has a stellar faculty.
Of course, Colorado has excellent faculty as well. I am more familiar with AMO, than high energy there.
I honestly would have a hard time choosing between them on academic grounds. For a doctoral degree, slight edge to Colorado, for a Bachelor's degree, slight edge to Rochester.
I found that Junior and Senior level undergraduate physics classes at Colorado and Rochester were roughly the same size. Colorado has more engineering majors so the Freshman and Sophomore classes were (just a) bit larger.
Now for the real difference.
Weather: Believe it or not the weather is worse in Rochester. (Almost always overcast). At least some snow covers the ground from early December to April. In 1975, in early April, a blizzard buried cars and people were unable to find them for a few days (true). In 1977-8, Rochester had 161 inches of snow. I have seen some flurries on graduation day in early May.
Rochester is a river campus, although there is little if any recreational activity on the Genesee River. It is nothing like if you imagine Harvard and MIT's relationship with the Charles River. Rochester was once a beautiful city with several waterfalls, but it has fallen on some hard times. Rochester is a few miles from downtown. Parts of the city within 1-2 miles from the campus are unsafe (I know some of the student population used to dispute this with me, but Rochester is my hometown, so I know it).
It gets cold and snowy in Colorado, but not as cold or snowy as Rochester. The suns shines much more in Colorado. If you have indoor spirit in your blood, Rochester is the place, but outdoorsmen and women like Colorado. The air is thinner, it is over a mile above sea level. Students in general are in good physical condition, but they are in excellent physical condition here. Colorado has the reputation as the state with the lowest obesity rate, however I have to admit the food tastes better in Rochester. This may be because it is my hometown and I grew up eating the local food, like pizza for instance.
Rochester is smaller, the buildings are attractive. During my years there, they opened up Wilson Commons a student union designed by I.M.Pei replacing Todd Union, but this is less of a draw these days. The buildings are more numerous and also attractive at Colorado.
Summarizing, either school will prepare you well for a doctoral degree anywhere in the country. If you might rather try EE or Aero Engineering, it would be best to choose Colorado, though. If you are an indoor type Rochester, has the edge. Outdoor type Colorado has the edge. Colorado area (not school) beats Rochester area(not school) in scenic beauty.
I think I prefer the combination Rush Rhee's and Bausch and Lomb library at Rochester to the library system at Colorado, but I have not been in those libraries for > 30 years.
Fellow Students: Rochester has many students from the New York City area, some of whom look down their noses at the city of Rochester. Jokes about the (local) Genesee beer are common and undeserved. Boulder students are more diverse and laid back.
Boulder is more like a college town than Rochester.
Hope this helps.