I originally started doing Chemistry at university (for a bureaucratic reason my options were limited to chemistry and engineering degrees) and although I had a long-standing interest in it, I quickly realized that I was paying more attention to the physics aspects in the courses than the chemistry. I liked lab work but I felt like I'd rather be studying physics most of the time. After my 2nd year I had the opportunity of switching to physics and did so, and don't regret my decision.
My thoughts are that if you're certain you want to do chemistry & lots of lab work, stick with chemistry. If you have the slightest inkling towards physics, then try physics instead of trying to convince yourself you'll get to do as much physics as you'd like in chemistry (as I tried to and wasn't satisfied).
I definitely see a lot of theoretical physicists doing "theoretical chemistry", making computer models of molecular interactions, working out bonding energies, etc. at my university. So if that's the bit of chemistry you like the most (like myself), you'll probably be making a good decision (and have a lot of other options available too).