I don't know how helpful my advice can be, but I think you are good for a PhD in Material Science. I think for a degree like Material Science, you can never really be very separated from physics. A PhD in (Applied) Physics focusing on solid state materials/ condensed matter may also be an option for you.
Many people with physics degrees go into materials work. The nature of the research can be different depending on the labs though. Most engineering labs have a purpose in mind when they do research. Something that they can use their research to improve. Physics labs tend to be interested in fundamental properties and explaining things. Of course these categories are not mutually exclusive. And getting a degree in material science does not prevent you from working in a physics lab and vice-versa.
Given that you said that you like the idea of studying new materials and improving them, you could be good with both degrees. I think material science may be better, but I find I am more partial to engineering research. The major does not really prevent you from doing the research you want I think. It's just that doing a materials degree would have you do more material related coursework. Possibly may make finding a nonacademic job easier later too, but don't quote me on that.
I hope this was helpful.