If you are interested in doing physics, then as sandplasma says, go into physics.
If you are interested in applying modern physics to design new materials, devices, etc., then the primary options I can think of are:
materials science - much of materials science is effectively applied solid state physics. Some modern materials science academic research involves designing materials starting with quantum mechanics.
electrical engineering - semiconductor physics, materials design, device design. Lasers and quantum optics are sometimes in EE departments.
applied / engineering physics - can do similar work as all of the above, depending upon department.
Of course, here I am assuming by "modern physics" you mean that in the traditional sense where quantum mechanics is important. Cutting edge research in "modern physics" will only be found in physics departments.
If you decide to do engineering and are really interested in these areas, I recommend taking extra courses from the physics department (unless your engineering department is unusually strong with quantum mechanics courses).
best of luck,
jason