I think the word "gauge" can be loosely translated as "phase". It is phase for the U(1) group and mathematical extensions of the concept for other groups. Electromagnetism is a (the?) U(1) group. Thus phase (i.e. complex numbers) is important to electrical engineers. Strong force engineers (if any existed) would be more concerned with octonions which are like phase on steroids.
My understanding is that the rules (Maxwell's equations) fall out of the math for a U(1) group. But there are other solutions which also work like mfb points out. Adding one volt to everything doesn't really change how the equations work. We choose to use 0V as the base because it is convenient. But we could define everything with ground at 10,000V. Then a 9V battery would give 10,009 volts output. Of course that's silly. Instead we choose ground to be 0V.
As I understand it, any choice of reference that doesn't affect the underlying algebra is a degree of freedom. This might include a scalar base value as mfb illustrated, or a scaling variable such as unit choice.
Other choices do affect the algebra. The U(2) group uses quaternions. These are non-commutative, or more specifically anti-commutative. This means a⋅b = - b⋅a rather than the more common a ⋅ b = b ⋅ a seen in real and complex numbers. (The U(3) group uses octonions which do something similar with the associative principle.)
I'm not sure why SU(2) and SU(3) are special.
You might get a better, but perhaps less understandable answer on the Quatum Physics forum. This really isn't my field.