Why should mathematics be exempt from reality?
Because mathematics does not deal with reality; it deals with the formal consequences of axioms.
An axiom is simply a logical statement; it gets the name "axiom" because of the way we use the statement.
Here are a few of the axioms of the real numbers:
a + (b + c) = (a + b) + c
a * b = b * a
a * (b + c) = a * b + a * c
There are about 10 in all, of varying complexity.
Now, it is entirely possible to define "infinity" to mean "7". If you did so, then infinity really would be a real number. However, no standard definition of infinity yields a real number. In fact, no definition of infinity yields an object that is
any sort of familiar number! In particular,
Infinite numbers (such as those in the Surreals, or in the Cardinals) are
not called "infinity".
Any sort of connection between mathematics and "reality" falls under the purview of
science. It is
science that says real numbers have some sort of connection to the real world. If
science determined that there was a fundamental length, then that would mean that science would no longer attempt to say that real numbers are lengths; it does not mean that the mathematical meaning of a real number should change.