Trig substitutions are very helpful when the integrand contains a sum or difference of squares: x2 + a2, x2 - a2, or a2 - x2, and especially when the integrand contains the square root of one of these three. I'll focus on integrands with the square root of one of these three expressions, but the idea is more general than that.
Rather than memorize which trig substitution goes with which form, I draw a right triangle and label the sides and hypotenuse in accordance with the expression I'm dealing with. I label the acute angle θ.
\sqrt{x^2 + a^2}
This expression represents the hypotenuse of the right triangle. You can label the two other sides as x and a in either combination, but most texts label the altitude as x and the base as a. This gives tan(θ) = x/a, or a tan(θ) = x. From this substitution you can get the relationships between the differentials: dx = a sec2(θ)dθ and another relationship that involves the radical; namely, sec(θ) = sqrt(x2 + a2)/a, or a*sec(θ) = sqrt(x2 + a2).
\sqrt{x^2 - a^2}
Here the radical suggests that it represents one of the sides of the triangle, with x being the length of the hypotenuse and sqrt(x2 - a2) being one of the sides. Many texts pick the altitude for this value and label the base as a.
This gives cos(θ) = a/x, or equivalently, sec(θ) = x/a, or a sec(θ) = x. From this you get dx = a sec(θ) tan(θ) dθ. An expression involving the radical is a*tan(θ) = sqrt(x2 - a2).
\sqrt{a^2 - x^2}
This is similar the one above, but the hypotenuse is labelled a. The two sides can be labelled as x and sqrt(a2 - x2) in either way, but most often I've seen it with the altitude labelled as x and the base labelled with the radical.
This gives sin(θ) = x/a, or a sin(θ) = x, so dx = a cos(θ) dθ. An expression for the radical is cos(θ) = sqrt(a2 - x2)/a, so a*cos(θ) = sqrt(a2 - x2)