No, you normally don't have to commit to a specific field when you enter (in US graduate schools at least). At my graduate school (Michigan), most students spent the first two years taking the required courses (QM, E&M, etc.), teaching introductory labs or recitations, and preparing for the qualifying exam. You're encouraged to participate in research, of course; Michigan had a program specifically for the summer following the first year, in which students could work for a research project without the project having to pay you out of their own budget (it came out of the departmental budget). You can talk to people, try working for a couple of different groups, and commit to a specific field only when you're ready to form a dissertation committee and become a formal Ph.D. candidate.
I spent my first summer working with one of the low-temperature people, trying to get a helium dilution refrigerator to work. I was also interested in computer programming (this was back in the 1970s before PCs existed), and spent my spare time fiddling with that. The LT guy at that time didn't need much in the way of computing expertise, but he noticed my interest. He mentioned me to one of the profs in the bubble-chamber group, which was doing experimental high-energy neutrino physics, and of course needed a lot of programming done. So during my second year I got an invitation to try that group, and I ended up doing my dissertation with them.