A lot of things are defined in respect to very specific situations. For example, the second is defined as "9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the caesium-133 atom". The meter is defined as "the length of the path traveled by light in absolute vacuum during a time interval of 1/299,792,458 of a second".
That doesn't mean that we are restricted to using seconds only when talking about cesium atoms, or using meters only when talking about how far light travels.
The amu is just a convenient unit to use when talking about things with mass on the scale of atoms. (And those things are typically atoms)