The planet math main article on l'Hospitals rule doesn't clearly state the hypotheses of the rule. (It even misspells the word "existence".) You are correct that the proof you linked ought to justify the assumption that f(c) and g(c) are finite.
I think the key to doing that is to interpret the hypothesis that [itex]lim_{x \rightarrow a} \frac{g'(x)}{f'(x)}[/itex] exists. (The Wikipedia version of the rule says "exists or is infinite".) As I interpret this hypothesis, it is not assumption that f'(a) and g'(a) exists as individual functions. I think we must try to argue that if the limit of the quotient exists at x = a then there must be an open interval containing a where both f'(x) ang g'(x) exist everywhere in the interval except possibly at x = a. The functions f and g are differentiable at those points, hence continuous and hence finite. We can pick c to be in this interval.