If the gain margin is 5 dB, this means the open loop gain of the system can only increase by 5 dB before going unstable.
Your controller gain may not change, but a change in the parameters of the plant will throw everything off.
Basically, the higher the gain margin, the less likely a change...