# I don't get Eigenvalues or Eigenvectors

1. ### Alex6200

75
I just finished Differential Equations, and I know how to find eigenvalues/eigenvectors, and I understand how to use them to solve a differential equation.

But I don't really understand "what they are". How is a matrix with complex eigenvalues any different than a matrix with real eigenvalues? What does the eigenvalue tell us about the form of a matrix? What does it tell us about its character - its form?

I've always understood things like derivatives, because they make sense. The derivative of a function is just how steep its slope is. But linear algebra just doesn't make sense to me. What is an eigenvalue? What is a determinant? What is the matrix?

2. ### Count Iblis

The most general linear relation between quantities $$x_{i}$$ and $$y_{i}$$ is:

$$y_{i}=\sum_{j}A_{i,j}x_{j}$$

right?

3. ### Alex6200

75
I guess you could find matrices that fit that rule...

4. ### slider142

956
A map f from a vector space U (over some field F, usually the field of real numbers or the field of complex numbers) to a vector space V is linear iff f(a + sb) = f(a) + sf(b) for all a, b in U and s in the field F.
From linear algebra, we know that all vectors u in U and v in V can be written as linear combination of a basis (a list of vectors in the space that span the space) of that space. That is to say that u = u1e1 + u2e2 + ... + unen, where each ui is an element of the field and the ei's are elements of U and U is n-dimensional. Suppose f is a linear transformation from U into V where V is an m-dimensional space. Use the basis property to show that f can be specified by m*n numbers (in the field F). These are the numbers Aij in Count Iblis's reply.

Last edited: Jun 8, 2008
5. ### waht

To get a better understanding of the eigen-stuff consider this equation:

$$A \, \vec{x} = \lambda \, \vec{x}$$

On the left side of the equation, A operates on an unknown vector x. That means A transforms a vector x into a different vector. Now the right side of the equation forces the transformed vector to equal to itself.

Said differently, x is a vector that remains unchanged under a transformation. If you try to alter it (matrix multiplication), the net result will spit out the same vector you tried to alter.

We call that vector an eigenvector. However, the magnitude of the eigenvector can change under a transformation, and we call that magnitude an eigenvalue (lambda in the equation). But the vector's direction is unchanged.

6. ### Alex6200

75
So if f(a + b) = f(a) + f(b) then f is linear, and by that same reasoning f(constant * x) = constant * f(x).

I understand that much.

Is that relying on some Linear Algebra stuff, or should a Differential Equations student know what that means?

7. ### slider142

956
Eigenvalues and eigenvectors are linear algebra terms. Multivariable calculus and diffEq's make a lot more sense if you study linear algebra first. There are quite a few resources online as well.

8. ### HallsofIvy

40,226
Staff Emeritus
It's a shame that you didn't take Linear Algebra before taking differential equations. At my university I was instrumental in having Linear Algebra added as a pre-requisite to the introductory differential equations course.

I would recommend that you take it now. Not only will it answer your questions but it is as useful a subject in all applications as differential equations.

9. ### Alex6200

75
Well, I took differential equations at the community college before heading off to college (Worcester Polytechnic Institute, Electrical Engineering), and I'm going to take Linear Algebra first semester there. I couldn't take linear algebra at my high school or local community college.

The thing is, I understand eigenvalues in a formal sense. I know how to use them to solve differential equations. I know the definitions, I just don't... get them, if you know what I mean. Like, I can't just look at a matrix and think "Wow, that matrix must have some big eigenvalues", the way I'd look at a parabola and immediately get a sense of what it would look like.

10. ### mathwonk

9,673
what is the simplest functioin with regard to differentiation? many would say its the exponential because D(e^at) = ae^at.

i.e. the derivative is just a multiple of the function. thats what characterizes an eigenvector, for a given operation, namely when you operate you just multiply by a number.

it doesnt get any better than that.

11. ### Count Iblis

And given the exponential function we can write any analytical function as:

f(x) = Exp(x D) f(0)

### Staff: Mentor

Suppose a and x are scalars, and the time derivative of x is

$$\frac{d}{dt}x = ax$$

The solution to this differential equation is of the form

$$x(t) = e^{at}x(0)$$

Now, instead suppose x is a vector. The corresponding differential equation is

$$\frac{d}{dt}\vec x = \mathbf A \vec x$$

where $\mathbf A$ is a square matrix. The solutions to this multidimensional differential equation are of the form

$$\vec x(t) = e^{\mathbf A t}\vec x(0)$$

where the matrix exponential is defined as

$$e^{\mathbf A t} = \boldsymbol 1 + \mathbf A t + \frac 1 2 \mathbf A^2 t^2 + \cdots$$

(the first term on the right-hand side is the identity matrix).

The eigenvalues and eigenvectors of the matrix describe the solution. Suppose $\vec x_i$ is an eigenvector of $\mathbf A$ with eigenvalue $k_i$. With a little work,

$$e^{\mathbf A t}\vec x_i = e^{k_i t} \vec x_i$$

Now decompose the initial state $\vec x(0)$ in terms of the eigenvectors:

$$\vec x(0) = \sum_i a_i x_i$$

With this, the solutions of the vector differential equation become

$$\vec x(t) = \sum_i a_i e^{\mathbf k_i t}\vec x_i$$

13. ### Alex6200

75
So, basically an eigenvector X of some transformation is the vector X for which multiplying by the matrix A is equivalent to that transform?

In Diff Eq. that transform would be differentiation, but it could be anything.

14. ### K.J.Healey

641
An eigenvector is basically a vector with certain properties such that when operated on by a certain operator (or matrix) it gives you the same vector back. Its coupled in a way to the operator itself. A certain operator will only give you the same vector back if its special.
But most of the time its the direction of the vector that matters. If you operate on (2,2,2) and you get (6,6,6) back, well its the same direction once normalized, but with a different magnitude. So O.(2,2,2) == (6,6,6) == 3 (2,2,2). So its "eigenvalue" is 3.

In physics it comes into play in quantum mechanics mainly because everything is a wave function (A sum of eigenvectors) and operators that work on a wavefunction without changing it are usually pretty special. Because if your operator destroys the wavefunction then its probably not very useful. So operators that can work on what you have and leave it intact(having the same basis vectors) are quite useful, and they represent many known observables.

15. ### ObsessiveMathsFreak

406
If you're able to visualise things like parabola's ellipses and hyperbolas fairly easily, then it may be helpful to look at the connections between eigenvalues and these objects. Specifically, look into the Principle axis theorem and how eigenvalues and vectors relate to the major and minor axes of these figures in 2D. The eigenvalues tell you which is the major and which the minor axis, and the eigenvectors tell you the directions of the axes. True, this analogy only holds for symmetric matrices, but it's a good starting point.

There are geometrical relations present in much if not all linear algebra. Unfortunately, most presentations of the subject tend to ignore these entirely.

16. ### trambolin

341
Very stupid example but sometimes works on weird individuals like me... (and it is not general enough to capture all the properties) but life is too short to explain everything at one shot, here it goes...

Imagine you are pushing a box with a very weird shape and suppose you don't see the whole box so it is difficult to estimate how to push it.

You want to push it so that the direction of the force that you are applying is exactly the direction the box moves, that means force vector and the displacement vector pointing the same direction. Possibly with different quantities! What usually happens is that you get both translation and rotation if you choose a bad direction. But there might be a direction where you get pure translation. That is your eigenvector. How much translation you get is proportional to your force and that is your eigenvalue, with respect to the that direction (or eigenvector) Now let A be your matrix explaining the relations between individual directions and translations you getin each direction, then after some concrete argument, you can show that there are some directions that your biiiiig matrix really acts like scalar, (you get only translation on that particular direction)

Now, if it makes sense, read the rigorous arguments above again. If not, forget about these immediately :)

17. ### Alex6200

75
So if I have a transform A, the eigenvector is the matrix which I can perform A on and get back that eigenvector, or the eigenvector with a bigger magnitude (the multiple of the magnitude is the eigenvalue).

18. ### trambolin

341
or smaller!

That's exactly what happens when we write
$$Ax = \lambda x$$

Actually, it is better to write like this (with a horrible math style!)

$$Something = Ax = \lambda x$$

So for some x vectors, (not all!), the matrix is only shrinking/stretching. The amount is defined by the eigenvalue.