There already exists notion of "infinite integers", called p-adic numbers, for prime numbers p. Written in base p, p-adic numbers are on the form [itex]...a_3a_2a_1[/itex] for infinite sequences of integers [itex]a_i[/itex] such that [itex]0 \leq a_i < p[/itex]. For example, in base 2, the number ...111111, which you seem to be interested, is a 2-adic number. It exists in any base p.
The set of p-adic numbers are written as [itex]\mathbb{Z}_p[/itex], and contains the formal series [itex]\pm \sum_{n=0}^{\infty}a_ip^n[/itex], where [itex]0 \leq a_i < p[/itex]. The usual integers [itex]\mathbb{Z}[/itex] are contained as a subset in [itex]\mathbb{Z}_p[/itex] (we just say that the terms are eventually 0). These formal infinite series may be added, subtracted, multiplied and even divided by (except for by 0). Unlike [itex]\mathbb{Z}[/itex], it is a field (you can't in general divide integers in [itex]\mathbb{Z}[/itex] and end up with integers), just like [itex]\mathbb{Q}[/itex] and [itex]\mathbb{R}[/itex], and is of particular interest in number theory.
Note that these infinite series are purely formal, and not convergent as infinite series in [itex]\mathbb{R}[/itex]. So don't confuse the two notions. There is a notion of convergence in the picture here too, but it is not analytic convergence. For the algebraically inclined, [itex]\mathbb{Z}_p[/itex] may be defined as an inverse limit of rings [itex]\mathbb{Z} / p \gets \mathbb{Z} / p^2 \gets \mathbb{Z} / p^3 \gets ...[/itex], and the p-adic numbers are the "convergent" elements of this sequence.
As a concrete example of what goes on here, let [itex]p = 5[/itex]. Then we have a convergent element
[itex](1,1+5,1+5 + 2 \cdot 5^2,1+5+2 \cdot 5^2+2 \cdot 5^3,1+5+2 \cdot 5^2+2\cdot 5^3+2\cdot 5^4,...)[/itex] in the sequence [itex]\mathbb{Z} / 5 \gets \mathbb{Z} / 5^2 \gets \mathbb{Z} / 5^3 \gets ...[/itex].
In [itex]\mathbb{Z} / 5^{n+1}[/itex] our element is [itex]1+ 5 + 2 \cdot 5^2 + ... + 2 \cdot 5^n[/itex], written as 22...2211 in base 5. It is clear that when we move from [itex]\mathbb{Z} / 5^{n+1} \to \mathbb{Z} / 5^{n}[/itex] (modulo [itex]5^n[/itex]), we end up with [itex]1+5 + 2 \cdot 5^2 + ... + 2 \cdot 5^{n-1}[/itex], so this sequence is, in techincal terms, stable under these (modulo) maps. In this way we make sense of the "infinite integer" ...2222222211 in [itex]\mathbb{Z}_5[/itex], equivalently written as [itex]1 +5+ \sum^{\infty}_{n=2}2 \cdot 5^n[/itex].
You can just as well speak of 10-adic numbers, infinite formal sums written in base 10. But the big difference when the base is not prime is not only that you do not end up with a field (so you may not divide in general), but not even an integral domain, which means that two non-zero integers in [itex]\mathbb{Z}_{10}[/itex] multiplied might yield 0. So it behaves very differently from [itex]\mathbb{Z}[/itex]. Your sequence 1, 11, 111, 1111, ... is indeed a "convergent" element of the sequence [itex]\mathbb{Z} / 10 \gets \mathbb{Z} / 10^2 \gets \mathbb{Z} / 10^3 \gets ...[/itex], and "converges" to ...11111111. It isn't quite the last element in the sequence (it has no last element), but just like for analytical limits, it is the "limiting" value.