as i see it, the problem ultimately lies with $\Bbb N$, the set of natural numbers.
to speak of the natural numbers as "one object" is what used to be referred to as "a completed infinity". it is, in essence, a LIMITING process (think of it as $n \to \infty$). the issues regarding whether such a thing is "proper mathematics" isn't really a mathematical question in and of itself, but a meta-mathematical one, with perhaps philosophical overtones and implications.
most modern mathematicians are readily willing to accept the "existence" of the natural numbers as a valid mathematical object, and are also comfortable with the concept of a function, or a mapping.
in this case, an "infinite decimal" is just a mapping:
$$f:\Bbb N \to \{0,1,2,3,4,5,6,7,8,9\}$$
(f(n) specifies the n-th digit, which is usually written $a_n$ or something similar).
the (philosophical) question tied up in this is:
can "any (abstract) function" be allowed, or do we need a RULE (algorithm) to DETERMINE f(n)? function spaces (sets), generally speaking, are quite large, and often unpredictable and counter-intuitive. in practice, arbitrary functions are not used much, some sort of restriction is usually placed on such a set (continuity, differentiability, polynomial, algebraic, rational, periodic, and so on). because "general functions" (even on such a nice domain as simply the natural numbers) don't behave very well (for example, most sequences aren't convergent).
in looking for a "general case" to include "the math we know and love", we are forced to include some pathological instances as well (such as "totally undefinable real numbers"). that bothers some people. it's good to keep in mind that the real numbers are an abstract system we invented to make life easier for us (some have gone so far as to call them "a convenient fiction"). if we accept a continuum, then continuity of certain functions becomes almost "automatic" (yes, i mean epsilons and deltas). it may be the case we actually live in a finitary universe, and will be in the peculiar position of making "continuous approximations" to "discrete phenomena" (instead of the other way around), in much the same way as we use a "bell-curve" to predict discrete test scores on a standard examination.
the proof that the real numbers form a field is actually the hard part (there are various subtleties in defining -x, and x+y and xy, and proving these satisfy the field axioms). proving they form a complete, ordered archimedean field is the easy part (or equivalently, showing the real numbers have the "least upper bound" property).
mathematicians are still not of a single mind regarding the axiom of choice: some believe that positing the existence of a choice function is sufficient; others want a function to be explicitly described.
if one looks closely enough, one sees (in logic) this side-stepped by replacing second-order axioms with axiom schema, which is a logician's way of saying: "yes i KNOW i can't list an infinite set, but if you give me an example of an element, i can tell you if the axiom holds for it or not".
i suppose, in the end, people ought to be free to choose the level of abstraction they "agree" with. what they regard as "true" will be a consequence of such a choice.
