There already exists notion of "infinite integers", called p-adic numbers, for prime numbers p. Written in base p, p-adic numbers are on the form ...a_3a_2a_1 for infinite sequences of integers a_i such that 0 \leq a_i < p. For example, in base 2, the number ...111111, which you seem to be interested, is a 2-adic number. It exists in any base p.
The set of p-adic numbers are written as \mathbb{Z}_p, and contains the formal series \pm \sum_{n=0}^{\infty}a_ip^n, where 0 \leq a_i < p. The usual integers \mathbb{Z} are contained as a subset in \mathbb{Z}_p (we just say that the terms are eventually 0). These formal infinite series may be added, subtracted, multiplied and even divided by (except for by 0). Unlike \mathbb{Z}, it is a field (you can't in general divide integers in \mathbb{Z} and end up with integers), just like \mathbb{Q} and \mathbb{R}, and is of particular interest in number theory.
Note that these infinite series are purely formal, and not convergent as infinite series in \mathbb{R}. So don't confuse the two notions. There is a notion of convergence in the picture here too, but it is not analytic convergence. For the algebraically inclined, \mathbb{Z}_p may be defined as an inverse limit of rings \mathbb{Z} / p \gets \mathbb{Z} / p^2 \gets \mathbb{Z} / p^3 \gets ..., and the p-adic numbers are the "convergent" elements of this sequence.
As a concrete example of what goes on here, let p = 5. Then we have a convergent element
(1,1+5,1+5 + 2 \cdot 5^2,1+5+2 \cdot 5^2+2 \cdot 5^3,1+5+2 \cdot 5^2+2\cdot 5^3+2\cdot 5^4,...) in the sequence \mathbb{Z} / 5 \gets \mathbb{Z} / 5^2 \gets \mathbb{Z} / 5^3 \gets ....
In \mathbb{Z} / 5^{n+1} our element is 1+ 5 + 2 \cdot 5^2 + ... + 2 \cdot 5^n, written as 22...2211 in base 5. It is clear that when we move from \mathbb{Z} / 5^{n+1} \to \mathbb{Z} / 5^{n} (modulo 5^n), we end up with 1+5 + 2 \cdot 5^2 + ... + 2 \cdot 5^{n-1}, so this sequence is, in techincal terms, stable under these (modulo) maps. In this way we make sense of the "infinite integer" ...2222222211 in \mathbb{Z}_5, equivalently written as 1 +5+ \sum^{\infty}_{n=2}2 \cdot 5^n.
You can just as well speak of 10-adic numbers, infinite formal sums written in base 10. But the big difference when the base is not prime is not only that you do not end up with a field (so you may not divide in general), but not even an integral domain, which means that two non-zero integers in \mathbb{Z}_{10} multiplied might yield 0. So it behaves very differently from \mathbb{Z}. Your sequence 1, 11, 111, 1111, ... is indeed a "convergent" element of the sequence \mathbb{Z} / 10 \gets \mathbb{Z} / 10^2 \gets \mathbb{Z} / 10^3 \gets ..., and "converges" to ...11111111. It isn't quite the last element in the sequence (it has no last element), but just like for analytical limits, it is the "limiting" value.