I know some of their applications, but I wanted to know how they first appeared. Why were eigenvalues and eigenvectors needed?
Wiki knows all: http://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors Check under section "history"
I tried to understand that before asking here, but I didn't... How exactly did they arise from the study of quadratic forms and differential equations?
As a simple example of why principal axes are important, consider bending of a cantilever beam with a rectangular cross section. If you apply a force in the direction of one of the principal axes, the beam bends in the same direction as the force. The stiffness (Force / displacement) will be different for the two principal axes, depending on the relative width and depth of the beam (I = bd^{3}/12 in one direction and b^{3}d/12 in the other.) If you apply a force at an angle to the principal directions, the beam does NOT bend in the same direction as the force. You can find the direction by resolving the force into components in the principal directions, finding the corresponding components of displacement, and combining them. How all that relates to the eigenvalues and vectors of the 2x2 inertia matrix for the cross section of the beam should be fairly obvious.
The Wikipedia article is worth looking at. In the theory of linear ODEs, eigen vectors define a basis from which all other solutions are linear combinations.