I like Serena said:
How about the distinction between a ring and a field?
For instance, when we write $x \equiv 11 \pmod{341}$ or $x \equiv 11 \pmod{347}$, it is not clear whether we will have an inverse or not. If we know that $n$ is prime, this is immediately clear.
So it would be nice if we could somehow indicate this in the notation.
I'm not aware of any such notation though, other than writing for instance the cumbersome
$$x \equiv 11 \pmod{347} \quad \text{ where 347 is prime}$$
which contains the implication that we're talking about a field.
We don't have to know the modulus is prime, we have to know the element we're trying to invert is co-prime to the modulus.
Sometimes, it's helpful to "tag" the modulus (Wikipedia does this in its article on the Chinese Remainder Theorem, where we deal with simultaneous congruences of different moduli). So you might want to write:
$[a]_{13}$ for example, or if the prime is not known:
$[a]_p$, where $p$ is prime.
My point isn't about properties of the modulus itself (primes are obviously special; we have, for example Fermat's Little Theorem, instead of the less-comprehensive Euler's Theorem which requires restrictions on the relationship between $[a]_n$ and $n$ itself), it's about trying to take everything back to things like:
$a \equiv m \text{ (mod }n) \implies a = m + kn$ <---already we have some spurious integer $k$ which we don't need.
All of this is already coded in $[a]_n = [m]_n$, and we can work with the equivalence classes DIRECTLY. This is similar to what is done with field extensions:
formally, $\Bbb C = \Bbb R[x]/\langle x^2 + 1\rangle$, but nobody works with complex numbers "carrying around the principal ideal generated by $x^2 + 1$". We give the coset of $x$ a name ($i$ or $j$), and just deal with $a + bi$.
When I actually do calculations mod 5, for example, I will do stuff like:
3x + 4 = 2
3x = 3 (add 1 to both sides)
x = 1 (multiply both side by 2)
If I was working mod 12, I would start:
3x + 4 = 2
3x = 10 (add 8 to both sides)
0 = 4 (multiply by 4: contradiction, no solution).
I don't even have to know ahead of time if an element is invertible, I can find out by looking at gcd's.
Yes, not all moduli are equally "nice" (not all integers are equally "nice"), but the arithmetic still works (sure, not all finite cyclic rings are fields: the integers aren't a field either. Sure, some cyclic rings have zero-divisors, some integers are highly divisible).
We lose some information when we consider the integers modulo (something). We also gain something: we have less information to wade through. If the information we lost wasn't relevant, we can make progress on our original problem.
Sure, the $n$ in $\Bbb Z \to \Bbb Z_n$ MATTERS, it's the characteristic in our new ring. Rings of different characterstic have (dare I say this?) different characteristics. But they're still rings, and everything we do is still "kosher" (except division-this is sort of a "special case", diving by zero isn't allowed in ANY structure, and zero-divisors act much like zero, they're "bad").
My vaguely-defined strategy for problem-solving: simplify the problem to something "smaller", play in the small sandbox until what I want to show is trivial: blow it back up.
When working with problems involving integers: often one uses prime factorization to consider prime or co-prime cases, and then use these "special cases" to (hopefully) recover the "general case". Sometimes this works, sometimes it doesn't. Some problems (like Goldbach's conjecture) are really HARD.
I believe if Euler or Gauss were alive today, they would happily adapt to a more modern point of view: algebra was in its infancy at those times, and they did the best they could with the tools available.