Check out this wikipedia page.
http://en.wikipedia.org/wiki/LTI_system_theory#Important_system_properties
It depends on your application. If you are doing Fourier transforms, then what you said is true and to be stable the impulse responds h(t) must be finite.
Its different for Z-transforms.
Ill assume you are working with Fourier transforms.
Lets say you have a circuit, and you input an impulse. The output you would read is the impulse response, h(t).
A common mathematical thing you might do, is use a Fourier transform on h(t) to get the frequency response, H(jw), because we like having frequency response of systems.
If you have a h(t) that doesn't 'approach zero for sufficiently large time', when you compute the Fourier transform, you could possible get an answer that is infinite, or something not find able.
If h(t) was stable (finite), then finding the H(jw) would be possible mathematically, using a Fourier transform.
In simpler words, if your h(t) goes to infinite, then you will have problems integrating it. And we like integrating things, to get H(s), H(jw), and other things.