Nuclear Engineering involves some knowledge of physics, particularly nuclear physics, but the use of 'physics' will depend on one's specialty. Working with cross-section libraries and neutron interaction, or radiation effects would indeed involve quite a lot of physics. On the other hand, doing core design or working in operations will not necessarily involve a lot of physics, and certainly not QM, electrodynamics/magnetohydrodynamics, or Stat Mech.
Similar, there are specialties in Mechanical Engineering where one would use a lot of physics or chemical physics, and others that don't.
Besides, Nuclear Engineering typcially includes a fair amount of topics from Mechanical Engineering, e.g., thermodynamics and heat transfer, fluid mechanics, mechanics of materials, . . . . In addition, for either NE or ME, one should probably consider a few courses in materials science/engineering. In addition, one would probably be exposed to introductory topics in EE, e.g., circuit analysis and electromechanics, and control theory.
Whether one pursues ME or NE, try to be as diverse as possible. One hot area now is 'multiphysics simulation'.