Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

I don't get Eigenvalues or Eigenvectors

  1. Jun 8, 2008 #1
    I just finished Differential Equations, and I know how to find eigenvalues/eigenvectors, and I understand how to use them to solve a differential equation.

    But I don't really understand "what they are". How is a matrix with complex eigenvalues any different than a matrix with real eigenvalues? What does the eigenvalue tell us about the form of a matrix? What does it tell us about its character - its form?

    I've always understood things like derivatives, because they make sense. The derivative of a function is just how steep its slope is. But linear algebra just doesn't make sense to me. What is an eigenvalue? What is a determinant? What is the matrix?
  2. jcsd
  3. Jun 8, 2008 #2
    The most general linear relation between quantities [tex]x_{i}[/tex] and [tex]y_{i}[/tex] is:


    right? :smile:
  4. Jun 8, 2008 #3
    I guess you could find matrices that fit that rule...
  5. Jun 8, 2008 #4
    A map f from a vector space U (over some field F, usually the field of real numbers or the field of complex numbers) to a vector space V is linear iff f(a + sb) = f(a) + sf(b) for all a, b in U and s in the field F.
    From linear algebra, we know that all vectors u in U and v in V can be written as linear combination of a basis (a list of vectors in the space that span the space) of that space. That is to say that u = u1e1 + u2e2 + ... + unen, where each ui is an element of the field and the ei's are elements of U and U is n-dimensional. Suppose f is a linear transformation from U into V where V is an m-dimensional space. Use the basis property to show that f can be specified by m*n numbers (in the field F). These are the numbers Aij in Count Iblis's reply.
    Last edited: Jun 8, 2008
  6. Jun 8, 2008 #5
    To get a better understanding of the eigen-stuff consider this equation:

    [tex] A \, \vec{x} = \lambda \, \vec{x} [/tex]

    On the left side of the equation, A operates on an unknown vector x. That means A transforms a vector x into a different vector. Now the right side of the equation forces the transformed vector to equal to itself.

    Said differently, x is a vector that remains unchanged under a transformation. If you try to alter it (matrix multiplication), the net result will spit out the same vector you tried to alter.

    We call that vector an eigenvector. However, the magnitude of the eigenvector can change under a transformation, and we call that magnitude an eigenvalue (lambda in the equation). But the vector's direction is unchanged.
  7. Jun 8, 2008 #6
    So if f(a + b) = f(a) + f(b) then f is linear, and by that same reasoning f(constant * x) = constant * f(x).

    I understand that much.

    Is that relying on some Linear Algebra stuff, or should a Differential Equations student know what that means?
  8. Jun 8, 2008 #7
    Eigenvalues and eigenvectors are linear algebra terms. Multivariable calculus and diffEq's make a lot more sense if you study linear algebra first. There are quite a few resources online as well.
  9. Jun 8, 2008 #8


    User Avatar
    Science Advisor

    It's a shame that you didn't take Linear Algebra before taking differential equations. At my university I was instrumental in having Linear Algebra added as a pre-requisite to the introductory differential equations course.

    I would recommend that you take it now. Not only will it answer your questions but it is as useful a subject in all applications as differential equations.
  10. Jun 8, 2008 #9
    Well, I took differential equations at the community college before heading off to college (Worcester Polytechnic Institute, Electrical Engineering), and I'm going to take Linear Algebra first semester there. I couldn't take linear algebra at my high school or local community college.

    The thing is, I understand eigenvalues in a formal sense. I know how to use them to solve differential equations. I know the definitions, I just don't... get them, if you know what I mean. Like, I can't just look at a matrix and think "Wow, that matrix must have some big eigenvalues", the way I'd look at a parabola and immediately get a sense of what it would look like.
  11. Jun 8, 2008 #10


    User Avatar
    Science Advisor
    Homework Helper

    what is the simplest functioin with regard to differentiation? many would say its the exponential because D(e^at) = ae^at.

    i.e. the derivative is just a multiple of the function. thats what characterizes an eigenvector, for a given operation, namely when you operate you just multiply by a number.

    it doesnt get any better than that.
  12. Jun 8, 2008 #11
    And given the exponential function we can write any analytical function as:

    f(x) = Exp(x D) f(0)

  13. Jun 8, 2008 #12

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    Suppose a and x are scalars, and the time derivative of x is

    [tex]\frac{d}{dt}x = ax[/tex]

    The solution to this differential equation is of the form

    [tex]x(t) = e^{at}x(0)[/tex]

    Now, instead suppose x is a vector. The corresponding differential equation is

    [tex]\frac{d}{dt}\vec x = \mathbf A \vec x[/tex]

    where [itex]\mathbf A[/itex] is a square matrix. The solutions to this multidimensional differential equation are of the form

    [tex]\vec x(t) = e^{\mathbf A t}\vec x(0)[/tex]

    where the matrix exponential is defined as

    [tex]e^{\mathbf A t} = \boldsymbol 1 + \mathbf A t + \frac 1 2 \mathbf A^2 t^2 + \cdots[/tex]

    (the first term on the right-hand side is the identity matrix).

    The eigenvalues and eigenvectors of the matrix describe the solution. Suppose [itex]\vec x_i[/itex] is an eigenvector of [itex]\mathbf A[/itex] with eigenvalue [itex]k_i[/itex]. With a little work,

    [tex]e^{\mathbf A t}\vec x_i = e^{k_i t} \vec x_i[/tex]

    Now decompose the initial state [itex]\vec x(0)[/itex] in terms of the eigenvectors:

    [tex]\vec x(0) = \sum_i a_i x_i[/tex]

    With this, the solutions of the vector differential equation become

    [tex]\vec x(t) = \sum_i a_i e^{\mathbf k_i t}\vec x_i[/tex]
  14. Jun 8, 2008 #13
    So, basically an eigenvector X of some transformation is the vector X for which multiplying by the matrix A is equivalent to that transform?

    In Diff Eq. that transform would be differentiation, but it could be anything.
  15. Jun 9, 2008 #14
    An eigenvector is basically a vector with certain properties such that when operated on by a certain operator (or matrix) it gives you the same vector back. Its coupled in a way to the operator itself. A certain operator will only give you the same vector back if its special.
    But most of the time its the direction of the vector that matters. If you operate on (2,2,2) and you get (6,6,6) back, well its the same direction once normalized, but with a different magnitude. So O.(2,2,2) == (6,6,6) == 3 (2,2,2). So its "eigenvalue" is 3.

    In physics it comes into play in quantum mechanics mainly because everything is a wave function (A sum of eigenvectors) and operators that work on a wavefunction without changing it are usually pretty special. Because if your operator destroys the wavefunction then its probably not very useful. So operators that can work on what you have and leave it intact(having the same basis vectors) are quite useful, and they represent many known observables.
  16. Jun 9, 2008 #15
    If you're able to visualise things like parabola's ellipses and hyperbolas fairly easily, then it may be helpful to look at the connections between eigenvalues and these objects. Specifically, look into the Principle axis theorem and how eigenvalues and vectors relate to the major and minor axes of these figures in 2D. The eigenvalues tell you which is the major and which the minor axis, and the eigenvectors tell you the directions of the axes. True, this analogy only holds for symmetric matrices, but it's a good starting point.

    There are geometrical relations present in much if not all linear algebra. Unfortunately, most presentations of the subject tend to ignore these entirely.
  17. Jun 9, 2008 #16
    Very stupid example but sometimes works on weird individuals like me... (and it is not general enough to capture all the properties) but life is too short to explain everything at one shot, here it goes...

    Imagine you are pushing a box with a very weird shape and suppose you don't see the whole box so it is difficult to estimate how to push it.

    You want to push it so that the direction of the force that you are applying is exactly the direction the box moves, that means force vector and the displacement vector pointing the same direction. Possibly with different quantities! What usually happens is that you get both translation and rotation if you choose a bad direction. But there might be a direction where you get pure translation. That is your eigenvector. How much translation you get is proportional to your force and that is your eigenvalue, with respect to the that direction (or eigenvector) Now let A be your matrix explaining the relations between individual directions and translations you getin each direction, then after some concrete argument, you can show that there are some directions that your biiiiig matrix really acts like scalar, (you get only translation on that particular direction)

    Now, if it makes sense, read the rigorous arguments above again. If not, forget about these immediately :)
  18. Jun 9, 2008 #17
    So if I have a transform A, the eigenvector is the matrix which I can perform A on and get back that eigenvector, or the eigenvector with a bigger magnitude (the multiple of the magnitude is the eigenvalue).
  19. Jun 9, 2008 #18
    or smaller!

    That's exactly what happens when we write
    [tex]Ax = \lambda x[/tex]

    Actually, it is better to write like this (with a horrible math style!)

    [tex]Something = Ax = \lambda x[/tex]

    So for some x vectors, (not all!), the matrix is only shrinking/stretching. The amount is defined by the eigenvalue.
  20. Jun 14, 2008 #19
    Alex, what community college did you go to? Was it NSCC by chance? If so, I may have been in that class with you.
  21. Jun 14, 2008 #20
    Ugh, Carroll. :uhh:
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook