That's not true. As a counter example, consider f(x) = (x2 - 9)/(x - 3). This function is discontinuous at x = 3, yet its limit as x approaches 3 is 6. The graph of f is identical to the graph of y = x + 3 except that the graph of f has a discontinuity (a hole) at the point (3, 6). This kind of discontinuity is called a removable discontinuity.
That's the basic idea. A limit fails to exist if the limit from the left and the limit from the right aren't equal.
A function can be unbounded and we say its limit is infinity if the left- and right-side limits are the same. For example,
\lim_{x \to 0} \frac{1}{x^2} = \infty
In one sense, a limit doesn't exist, since infinity is not a number. The closer x gets to 0 on either side, the larger 1/x2 gets. However, the left- and right-side limits are both doing the same thing.