I can't see the page of the book, but your understanding is correct.
Another way to look at this is to consider the energy scale of things.
[itex] E = \hbar \omega = \frac{h}{T}[/itex]
If the time scale of the atom is much faster than the external perturbation, then the perturbation will be on a much smaller energy scale than what's going on in the atom. Hence the atom will not be affected significantly.
This is of course not an exact relation, it just gives you a rule of thumb of what is relevant and what can be neglected.
