Infinity is what it's defined to be.
In the arithmetic of real numbers, there is no infinity. So statements like
infinity - X = infinity
are gibberish.
It is possible to consider
different systems of arithmetic. Two practical examples are the projective real numbers and the extended real numbers. In the extended real numbers, for example, it is true that
+\infty = +\infty - x
whenever x is not +\infty. The arithmetic of the extended real numbers is a
different system; in particular, the statement
if x > 0, then x + y > y
is simply not true.
Another useful number system is the ordinal numbers. There is no ordinal number called "infinity", but there are lots of infinite ordinals. The smallest infinite ordinal is \omega. In the ordinal numbers, the following statements are true:
1 + \omega = \omega
\omega + 1 > \omega
(note that addition is not commutative! If you add in different orders, you usually get different ordinal numbers)
The ordinals are related to orderings, and sequences. \omega is the ordinal that describes the sequence of natural numbers. 1 + \ometa is what you get by prepending an extra thing to the natural numbers... clearly the sequences
0 < 1 < 2 < ...
* < 0 < 1 < ...
have the same
order type. The ordinal number \omega + 1 is what you get if you append an extra thing to the natural numbers:
0 < 1 < 2 < ... | *
Every natural number appears in this sequence before the *, and there is no number that comes immediately before *. This sequence is clearly different than the other two above.
(The | is my own notation for this sort of thing. It's like a parenthesis: it's meant to indicate that the stuff on the left is to be taken as one "group", and the stuff on the right is to be taken as another "group")
I suppose the simple answer is, if you intend to learn about the infinite, you should forget everything you "learned" about it from nonmathematical sources.