I don't get Eigenvalues or Eigenvectors

[Alex6200]

I just finished Differential Equations, and I know how to find eigenvalues/eigenvectors, and I understand how to use them to solve a differential equation.

But I don't really understand "what they are". How is a matrix with complex eigenvalues any different than a matrix with real eigenvalues? What does the eigenvalue tell us about the form of a matrix? What does it tell us about its character - its form?

I've always understood things like derivatives, because they make sense. The derivative of a function is just how steep its slope is. But linear algebra just doesn't make sense to me. What is an eigenvalue? What is a determinant? What is the matrix?

 

[Count Iblis]

The most general linear relation between quantities x_{i} and y_{i} is:

y_{i}=\sum_{j}A_{i,j}x_{j}

right?

 

[Alex6200]

The most general linear relation between quantities x_{i} and y_{i} is:

y_{i}=\sum_{j}A_{i,j}x_{j}

right?

I guess you could find matrices that fit that rule...

 

[slider142]

A map f from a vector space U (over some field F, usually the field of real numbers or the field of complex numbers) to a vector space V is linear iff f(a + sb) = f(a) + sf(b) for all a, b in U and s in the field F.
From linear algebra, we know that all vectors u in U and v in V can be written as linear combination of a basis (a list of vectors in the space that span the space) of that space. That is to say that u = u¹e₁ + u²e₂ + ... + uⁿe_n, where each uⁱ is an element of the field and the e_i's are elements of U and U is n-dimensional. Suppose f is a linear transformation from U into V where V is an m-dimensional space. Use the basis property to show that f can be specified by m*n numbers (in the field F). These are the numbers A_ij in Count Iblis's reply.

 

[waht]

To get a better understanding of the eigen-stuff consider this equation:

A \, \vec{x} = \lambda \, \vec{x}


On the left side of the equation, A operates on an unknown vector x. That means A transforms a vector x into a different vector. Now the right side of the equation forces the transformed vector to equal to itself.

Said differently, x is a vector that remains unchanged under a transformation. If you try to alter it (matrix multiplication), the net result will spit out the same vector you tried to alter.

We call that vector an eigenvector. However, the magnitude of the eigenvector can change under a transformation, and we call that magnitude an eigenvalue (lambda in the equation). But the vector's direction is unchanged.

 

[Alex6200]

A map f from a vector space U (over some field F, usually the field of real numbers or the field of complex numbers) to a vector space V is linear iff f(a + sb) = f(a) + sf(b) for all a, b in U and s in the field F.

So if f(a + b) = f(a) + f(b) then f is linear, and by that same reasoning f(constant * x) = constant * f(x).

I understand that much.

From linear algebra, we know that all vectors u in U and v in V can be written as linear combination of a basis (a list of vectors in the space that span the space) of that space. That is to say that u = u¹e₁ + u²e₂ + ... + uⁿe_n, where each uⁱ is an element of the field and the e_i's are elements of U and U is n-dimensional. Suppose f is a linear transformation from U into V where V is an m-dimensional space. Use the basis property to show that f can be specified by m*n numbers (in the field F). These are the numbers A_ij in Count Iblis's reply.

Is that relying on some Linear Algebra stuff, or should a Differential Equations student know what that means?

 

[slider142]

Is that relying on some Linear Algebra stuff, or should a Differential Equations student know what that means?

Eigenvalues and eigenvectors are linear algebra terms. Multivariable calculus and diffEq's make a lot more sense if you study linear algebra first. There are quite a few resources online as well.

 

[HallsofIvy]

It's a shame that you didn't take Linear Algebra before taking differential equations. At my university I was instrumental in having Linear Algebra added as a pre-requisite to the introductory differential equations course.

I would recommend that you take it now. Not only will it answer your questions but it is as useful a subject in all applications as differential equations.

 

[Alex6200]

Well, I took differential equations at the community college before heading off to college (Worcester Polytechnic Institute, Electrical Engineering), and I'm going to take Linear Algebra first semester there. I couldn't take linear algebra at my high school or local community college.

The thing is, I understand eigenvalues in a formal sense. I know how to use them to solve differential equations. I know the definitions, I just don't... get them, if you know what I mean. Like, I can't just look at a matrix and think "Wow, that matrix must have some big eigenvalues", the way I'd look at a parabola and immediately get a sense of what it would look like.

 

[mathwonk]

what is the simplest functioin with regard to differentiation? many would say its the exponential because D(e^at) = ae^at.

i.e. the derivative is just a multiple of the function. that's what characterizes an eigenvector, for a given operation, namely when you operate you just multiply by a number.

it doesn't get any better than that.

 

[Count Iblis]

And given the exponential function we can write any analytical function as:

f(x) = Exp(x D) f(0)

 

[D H]

Suppose a and x are scalars, and the time derivative of x is

\frac{d}{dt}x = ax

The solution to this differential equation is of the form

x(t) = e^{at}x(0)

Now, instead suppose x is a vector. The corresponding differential equation is

\frac{d}{dt}\vec x = \mathbf A \vec x

where \mathbf A is a square matrix. The solutions to this multidimensional differential equation are of the form

\vec x(t) = e^{\mathbf A t}\vec x(0)

where the matrix exponential is defined as

e^{\mathbf A t} = \boldsymbol 1 + \mathbf A t + \frac 1 2 \mathbf A^2 t^2 + \cdots

(the first term on the right-hand side is the identity matrix).

The eigenvalues and eigenvectors of the matrix describe the solution. Suppose \vec x_i is an eigenvector of \mathbf A with eigenvalue k_i. With a little work,

e^{\mathbf A t}\vec x_i = e^{k_i t} \vec x_i

Now decompose the initial state \vec x(0) in terms of the eigenvectors:

\vec x(0) = \sum_i a_i x_i

With this, the solutions of the vector differential equation become

\vec x(t) = \sum_i a_i e^{\mathbf k_i t}\vec x_i

 

[Alex6200]

So, basically an eigenvector X of some transformation is the vector X for which multiplying by the matrix A is equivalent to that transform?

In Diff Eq. that transform would be differentiation, but it could be anything.

 

[K.J.Healey]

An eigenvector is basically a vector with certain properties such that when operated on by a certain operator (or matrix) it gives you the same vector back. Its coupled in a way to the operator itself. A certain operator will only give you the same vector back if its special.
But most of the time its the direction of the vector that matters. If you operate on (2,2,2) and you get (6,6,6) back, well its the same direction once normalized, but with a different magnitude. So O.(2,2,2) == (6,6,6) == 3 (2,2,2). So its "eigenvalue" is 3.

In physics it comes into play in quantum mechanics mainly because everything is a wave function (A sum of eigenvectors) and operators that work on a wavefunction without changing it are usually pretty special. Because if your operator destroys the wavefunction then its probably not very useful. So operators that can work on what you have and leave it intact(having the same basis vectors) are quite useful, and they represent many known observables.

 

[ObsessiveMathsFreak]

The thing is, I understand eigenvalues in a formal sense. I know how to use them to solve differential equations. I know the definitions, I just don't... get them, if you know what I mean. Like, I can't just look at a matrix and think "Wow, that matrix must have some big eigenvalues", the way I'd look at a parabola and immediately get a sense of what it would look like.

If you're able to visualise things like parabola's ellipses and hyperbolas fairly easily, then it may be helpful to look at the connections between eigenvalues and these objects. Specifically, look into the Principle axis theorem and how eigenvalues and vectors relate to the major and minor axes of these figures in 2D. The eigenvalues tell you which is the major and which the minor axis, and the eigenvectors tell you the directions of the axes. True, this analogy only holds for symmetric matrices, but it's a good starting point.

There are geometrical relations present in much if not all linear algebra. Unfortunately, most presentations of the subject tend to ignore these entirely.

 

[trambolin]

Very stupid example but sometimes works on weird individuals like me... (and it is not general enough to capture all the properties) but life is too short to explain everything at one shot, here it goes...


Imagine you are pushing a box with a very weird shape and suppose you don't see the whole box so it is difficult to estimate how to push it.


You want to push it so that the direction of the force that you are applying is exactly the direction the box moves, that means force vector and the displacement vector pointing the same direction. Possibly with different quantities! What usually happens is that you get both translation and rotation if you choose a bad direction. But there might be a direction where you get pure translation. That is your eigenvector. How much translation you get is proportional to your force and that is your eigenvalue, with respect to the that direction (or eigenvector) Now let A be your matrix explaining the relations between individual directions and translations you getin each direction, then after some concrete argument, you can show that there are some directions that your biiiiig matrix really acts like scalar, (you get only translation on that particular direction)


Now, if it makes sense, read the rigorous arguments above again. If not, forget about these immediately :)

 

[Alex6200]

So if I have a transform A, the eigenvector is the matrix which I can perform A on and get back that eigenvector, or the eigenvector with a bigger magnitude (the multiple of the magnitude is the eigenvalue).

 

[trambolin]

or smaller!

That's exactly what happens when we write
Ax = \lambda x

Actually, it is better to write like this (with a horrible math style!)

Something = Ax = \lambda x

So for some x vectors, (not all!), the matrix is only shrinking/stretching. The amount is defined by the eigenvalue.

 

[Saladsamurai]

Well, I took differential equations at the community college before heading off to college (Worcester Polytechnic Institute, Electrical Engineering), and I'm going to take Linear Algebra first semester there. I couldn't take linear algebra at my high school or local community college.

The thing is, I understand eigenvalues in a formal sense. I know how to use them to solve differential equations. I know the definitions, I just don't... get them, if you know what I mean. Like, I can't just look at a matrix and think "Wow, that matrix must have some big eigenvalues", the way I'd look at a parabola and immediately get a sense of what it would look like.

Alex, what community college did you go to? Was it NSCC by chance? If so, I may have been in that class with you.

 

[Alex6200]

Alex, what community college did you go to? Was it NSCC by chance? If so, I may have been in that class with you.

Ugh, Carroll.

 

[Saladsamurai]

Ugh, Carroll.

Is Carroll the name of the college? Or are you asking me if my name is Carroll? Because it's not, it's Casey.

 

[Alex6200]

No, Carroll's the name of the college.

 

[Howers]

edit

 

[Howers]

If this guy hasn't done any linear algebra how do you expect him to know about linear maps and even analytical geometry? I will offer a layman explanation, and hopefully you can build up on it when you take linear algebra.

First of all, you need to know that vectors of Rn can be represented as column matrices. For example, the vector (1,-4,3) would look like this in linear algebra: \begin{pmatrix}<br /> 1 \\<br /> -4 \\ <br /> 3 \\<br /> \end{pmatrix}<br />

Eigen is german for "the same". So when you are working with a matrix A, you are trying to find an X and \lambda such that AX=\lambdaX. Here, X is the vector (represented as a colum matrix) and \lambda is some number. It should be clear that when multiplying matrices A (nxn) and X (nx1) you get an nx1 matrix in return. As you know, multiplying a scalar by X will also give you an nx1 matrix. So you are trying to find an X, called the eigenvector, so that AX is equivalent to X multiplied by some scalar, \lambda, called the eingenvalue. Sometimes more than one X will work for a certain \lambda.

To find eigenvectors, you work with AX = \lambdaX, which is the same as AX = \lambdaIX, which is the same as (A - \lambdaI)X = 0. Obviously X=0 works for this, but by definition no eigen can ever be zero. The only way to guarantee that there exist some X other than zero is to make (A - \lambdaI) not invertible.

Theorem: Suppose BX=0. If B is invertible, then X=0 is the only solution.
Proof: If B is invertible, then there is some B^- such that B^-B=I.
So X = IX = B^-BX = B^-(BX) = 0, because BX=0.

See? The theorem tells us (A - \lambdaI) must not be invertible otherwise we will only have X=0 as an eingenvector, which is not allowed! The way you do this is make the determinant zero. There is a theorem is linear algebra that says A is not invertible iff its determinant is zero. So that's what you are doing when finding eigenvalues, expanding the determinant in terms of \lambda and making it equal zero. Because only
(A - \lambdaI) with those \lambda are non-invertible, so they are the only ones with nonzero X.

Once you have your eigenvalues, you find out which Xs work which you already know how to do. Ultimately, there are infinitely many Xs that work so we overcome this problem by excluding all other scalar multiples of each eigenvector. So you've found an eigenvector so that AX is the same as \lambdaX, pretty weird huh? A matrix acting like a scalar towards some X! Each \lambda has its own eigenvectors, and sometimes more than 1 are possible.

Eigenvalues and eigenvectors are used in linear algebra to diagonalize matrices, that is to make them into a nice and cute form. Certain matrices have predictable eigenvalues. Others not so predictable. You will learn other ways to use them in linear algebra. For now, I hope this clears things up.

 

[insanenoodle]

Very stupid example but sometimes works on weird individuals like me... (and it is not general enough to capture all the properties) but life is too short to explain everything at one shot, here it goes...


Imagine you are pushing a box with a very weird shape and suppose you don't see the whole box so it is difficult to estimate how to push it.


You want to push it so that the direction of the force that you are applying is exactly the direction the box moves, that means force vector and the displacement vector pointing the same direction. Possibly with different quantities! What usually happens is that you get both translation and rotation if you choose a bad direction. But there might be a direction where you get pure translation. That is your eigenvector. How much translation you get is proportional to your force and that is your eigenvalue, with respect to the that direction (or eigenvector) Now let A be your matrix explaining the relations between individual directions and translations you getin each direction, then after some concrete argument, you can show that there are some directions that your biiiiig matrix really acts like scalar, (you get only translation on that particular direction)


Now, if it makes sense, read the rigorous arguments above again. If not, forget about these immediately :)

My God. This explanation is quite possibly the best I've ever heard. Looking at it in the way you mentioned makes explaining eigenvalue/vectors much more intuitive than throwing numbers and vague terms. Better than the ways any of my professors have tried explaining it. This coming from a junior at carnegie mellon university...

 

[Tac-Tics]

But linear algebra just doesn't make sense to me.

I have the same problem some times (though I've studied linear algebra for a few years now).

Sometimes, I think what we really want isn't so much to know what something is, but to figure out an intuitive understanding of it. You know what an eigenvector of an operator is. It's any vector whose direction is not changed under that operator (only it's length). And then eigenvalues are the scalars that give the change in length.

But what does it physically signify? I'm not sure myself (I was actually browsing through Wikipedia earlier today for just this reason). But eigenvectors and eigenvalues have lots of super weird properties. The sum of the eigenvalues is the trace of an operator. The product is the determinant. The determinant is an expression that "determines" the number of solutions (values for x) to the equation Tx = y (where T is a transformation and y is a vector). The determinant is also used in finding the inverse transformation.

But I don't understand this crap either, so I'll leave you with this instead!

What is the matrix?

Neo: What is the Matrix?
Trinity: The answer is out there, Neo, and it's looking for you, and it will find you if you want it to.

 

[crasshopper]

There are several more good answers on Quora.

http://www.quora.com/What-is-the-be...explain-what-eigenvectors-and-eigenvalues-are

(admin - sorry if this is against link policy)

 

[MathAmateur]

The classic example is the Earth rotating. If one can create a 3-D vector for each point on the surface of the Earth. As the Earth rotates, all but 2 of these vectors will change direction. The two that don't change directions (on the North and South Poles), are the Eigenvectors. And since they also don't move, their Eigenvalues are one.

 

[HallsofIvy]

How did you unearth this thread (and why)? It is about 2 and a half years old.

And you example isn't very clear. You say "create a 3-D vector for each point on the surface of the Earth" but you appear to be assuming, without saying it, that the vector at each point is pointing directly away from the center of the earth.

 

[mathwonk]

Thats true Halls, but, at least for someone that understands it, it is a pretty nice example, and by now the OP is probably long gone...! So we can enjoy it and think about how to use it next time this is asked. best wishes.