You can find a short discussion of Hermite polynomials in a book on mathematical methods. I have the one written by Arfken, but I'm sure others (those by Boas or Riley) will cover it, too. An in-depth treatment is in Lebedev, Special Functions and Their Applications, which also has excellent coverage of the other important functions (polynomials, Bessel functions, spherical harmonics, etc.) with many physics applications. It's a Dover book so it's inexpensive.
Finally, Hermite polynomials are famous as the solution to the one dimensional quantum-mechanical harmonic oscillator. You can find this physics application in all quantum mechanics books. For an undergrad QM text, see any of the standards like Griffith, Shankar, Liboff, or an inexpensive used copy of E. Anderson.
As to the general question of how to solve physics problems, I think you need to start with a course or a basic physics text. It is traditional to start with mechanics.