Normalisation of eigenvectors convention for exponentiating matrices

[Master1022]

Homework Statement

Given a matrix $A$, compute $e^{A}$

Relevant Equations

$A = V \Lambda V^{-1}$
$e^{A} = V \times diag(e^{\lambda_1}, ... , e^{\lambda_n}) \times V^{-1}$

Hi,

I just have a quick question when I was working through a linear algebra homework problem. We are given a matrix
$$A = \begin{pmatrix} 2 & -2 \\ 1 & -1 \end{pmatrix}$$ and are asked to compute $e^{A}$. In earlier parts of the question, we prove the identities
$$A = V \Lambda V^{-1}$$ and $$e^{A} = V \times diag(e^{\lambda_1}, ... , e^{\lambda_n}) \times V^{-1}$$ (apologies, I put the $\times$ as I was writing some text in).

My main question is: should we normalise the eigenvectors in the matrix $V$?
I thought we should, but the answer doesn't seem to. I have looked on the internet and most sources tend to agree with me, but I just wanted to confirm whether I was right or wrong in this scenario.

My attempt:
I understand the method, so I can just skip to what I got. I found the following forms for the matrices:
$$\Lambda = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$$ and

$$V = \begin{pmatrix} \frac{2}{\sqrt{5}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{5}} & \frac{1}{\sqrt{2}} \end{pmatrix}$$

From there, I can work out $exp(A)$ using the expression above. However, the answer does not normalize the eigenvectors and therefore gets a different answer.

Any help would be greatly appreciated.

 

[PeroK]

My main question is: should we normalise the eigenvectors in the matrix $V$?
I thought we should, but the answer doesn't seem to. I have looked on the internet and most sources tend to agree with me, but I just wanted to confirm whether I was right or wrong in this scenario.

The matrix $aV$ for any non-zero constant $a$ would do just as well.

 

[Master1022]

The matrix $aV$ for any non-zero constant $a$ would do just as well.

Thanks for your reply. However, given that the eigenvectors have different magnitudes ($\sqrt{5}$ and $\sqrt{2}$), then surely that would be a problem we need to let $a = \sqrt{10}$? The answer in the back of the book simply leaves
$$V = \begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix}$$

 

[PeroK]

Thanks for your reply. However, given that the eigenvectors have different magnitudes ($\sqrt{5}$ and $\sqrt{2}$), then surely that would be a problem we need to let $a = \sqrt{10}$? The answer in the back of the book simply leaves
$$V = \begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix}$$

What do you think the problem is? That matrix looks different from the one you found.

 

[Master1022]

What do you think the problem is? That matrix looks different from the one you found.

Apologies for the delay in my response, this reply only just came through now on my computer. How is the matrix different from mine (other than the normalization)? Note: I did mistype the upper right element in the matrix when I first posted it, but had edited it to 1 as soon as I posted.

 

[Master1022]

What do you think the problem is?

I think the answers may have made a mistake when choosing not to normalize the eigenvectors. When carrying out the calculations, I get:
$$e^A = \begin{pmatrix} \frac{2}{\sqrt{5}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{5}} & \frac{1}{\sqrt{2}} \end{pmatrix} \times \begin{pmatrix} e & 0 \\ 0 & 1 \end{pmatrix} \times \begin{pmatrix} \frac{2}{\sqrt{5}} & \frac{1}{\sqrt{5}} \\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{pmatrix} = \begin{pmatrix} \frac{4}{5}e + \frac{1}{2} & \frac{2}{5}e + \frac{1}{2} \\ \frac{2}{5}e + \frac{1}{2} & \frac{1}{5}e + \frac{1}{2} \end{pmatrix}$$EDIT: I just looked up an online widget (here) which gets a different answer to me (same as the back of the book)

 

[PeroK]

I think the answers may have made a mistake when choosing not to normalize the eigenvectors. When carrying out the calculations, I get:
$$e^A = \begin{pmatrix} \frac{2}{\sqrt{5}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{5}} & \frac{1}{\sqrt{2}} \end{pmatrix} \times \begin{pmatrix} e & 0 \\ 0 & 1 \end{pmatrix} \times \begin{pmatrix} \frac{2}{\sqrt{5}} & \frac{1}{\sqrt{5}} \\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{pmatrix} = \begin{pmatrix} \frac{4}{5}e + \frac{1}{2} & \frac{2}{5}e + \frac{1}{2} \\ \frac{2}{5}e + \frac{1}{2} & \frac{1}{5}e + \frac{1}{2} \end{pmatrix}$$

Yes, you definitely have to normalise the eigenvectors to begin with. Once you have the matrix $V$ with normalised eigenvectors you can multiply that through by a constant if that makes things easier.

The matrix at the back of the book looks wrong.

 

[Master1022]

Yes, you definitely have to normalise the eigenvectors to begin with. Once you have the matrix $V$ with normalised eigenvectors you can multiply that through by a constant if that makes things easier.

The matrix at the back of the book looks wrong.

Thank you very much, that is reassuring to hear. Just have two quick follow up questions:
1) If we did multiply through by some constant $a$ to aid our multiplication, then I presume we would need to divide by $a$ at the end?

2) Apologies for editing my previous post, but would you therefore agree that the online widget (it was just the first result on Google) also yielded the incorrect answer?

Thank you very much for the help.

 

[PeroK]

Thank you very much, that is reassuring to hear. Just have two quick follow up questions:
1) If we did multiply through by some constant $a$ to aid our multiplication, then I presume we would need to divide by $a$ at the end?

2) Apologies for editing my previous post, but would you therefore agree that the online widget (it was just the first result on Google) also yielded the incorrect answer?

Thank you very much for the help.

It looks like for this process you don't have to normalise the eigenvectors. I'm used to doing it to produce orthogonal or unitary matrices - but for simple diagonalisation it doesn't matter. Something I've learned.

In this case, you haven't calculated the inverse matrix correctly. These eigenvectors are not orthogonal, so you don't have an orthogonal matrix. You can't just transpose to get the inverse. The inverse is easier to calculate if you don't normalise the eigenvectors, of course.

PS I agree with the widget!

 

[DaveE]

The matrix $aV$ for any non-zero constant $a$ would do just as well.

Yes, exactly. (aV)^-1 = a^-1V^-1. The scalar multiple will cancel in the end.

edit: of course this isn't the same as normalizing the e-vectors, since the columns might not have the same magnitude.

 

[PeroK]

My main question is: should we normalise the eigenvectors in the matrix $V$?

To see that the answer is no, consider the matrix $A$ with eigenvectors $v_1, v_2$. And let $V = (v_1 \ \ v_2)$. Now:
$$AV = A(v_1 \ \ v_2) = (Av_1 \ Av_2) = (\lambda_1 v_1 \ \ \lambda_2 v_2)$$
And:
$$I = V^{-1}V = V^{-1}(v_1 \ \ v_2) = (V^{-1}v_1 \ \ V^{-1}v_2)$$
So that:
$$V^{-1}v_1 =
\begin{pmatrix}
1 \\
0
\end{pmatrix}
\ \ \text{and} \ \
V^{-1}v_2 =
\begin{pmatrix}
0 \\
1
\end{pmatrix}
$$
Therefore:
$$V^{-1}AV = V^{-1}(\lambda_1 v_1 \ \ \lambda_2 v_2) = (\lambda_1 V^{-1}v_1 \ \ \lambda_2 V^{-1}v_2) =
\begin{pmatrix}
\lambda_1 & 0 \\
0 & \lambda_2
\end{pmatrix}
$$
Without any normalisation.

 

[Master1022]

It looks like for this process you don't have to normalise the eigenvectors. I'm used to doing it to produce orthogonal or unitary matrices - but for simple diagonalisation it doesn't matter. Something I've learned.

I have just seen your latest post (perfect timing as I was just about to ask about this part) - I will take some time to digest it and make sure I understand it. I suppose fewer square roots lying around will only make the arithmetic simpler!

In this case, you haven't calculated the inverse matrix correctly. These eigenvectors are not orthogonal, so you don't have an orthogonal matrix. You can't just transpose to get the inverse. The inverse is easier to calculate if you don't normalise the eigenvectors, of course.

PS I agree with the widget!

Yes, that is right - I completely missed the fact that my eigenvectors weren't orthogonal...

Thank you very much once again.

 

[Master1022]

Yes, exactly. (aV)^-1 = a^-1V^-1. The scalar multiple will cancel in the end.

Thanks for clearing this up! This means that dividing the end result by $a$ isn't required as the (aV)^-1 = a^-1V^-1 deals with the initial factor of $a$ in the $aV$

 

[etotheipi]

eigenvectors $v_1, v_2$. And let $V = (v_1 \ \ v_2)$.

Apologies if I am unfamiliar with the notation, does this just mean $V$ is a matrix formed by 'putting the two vectors next to each other'?

 

[Master1022]

Apologies if I am unfamiliar with the notation, does this just mean $V$ is a matrix formed by 'putting the two vectors next to each other'?

I believe so. $v_1$ and $v_2$ are column vectors

 

[DaveE]

Apologies if I am unfamiliar with the notation, does this just mean $V$ is a matrix formed by 'putting the two vectors next to each other'?

Yes.

 

[Master1022]

To see that the answer is no, consider the matrix A with eigenvectors v1,v2. And let V=(v1 v2). Now:
AV=A(v1 v2)=(Av1 Av2)=(λ1v1 λ2v2)
And:
I=V−1V=V−1(v1 v2)=(V−1v1 V−1v2)
So that:
$$V^{-1}v_1 =
\begin{pmatrix}
1 \
0
\end{pmatrix}
\ \ \text{and} \ \
V^{-1}v_2 =
\begin{pmatrix}
0 \
1
\end{pmatrix}
$$
Therefore:
$$V^{-1}AV = V^{-1}(\lambda_1 v_1 \ \ \lambda_2 v_2) = (\lambda_1 V^{-1}v_1 \ \ \lambda_2 V^{-1}v_2) =
\begin{pmatrix}
\lambda_1 & 0 \
0 & \lambda_2
\end{pmatrix}
$$
Without any normalisation.

Thanks for posting this, it does help. Along with @DaveE's post to remind me how constants propagate through inverses, this concept is becoming more clear. One question I have is: do two orthogonal non-zero (eigen)-vectors $v_1$ and $v_2$ always have the same magnitude when expressed in their simplest form (i.e. the pair (1, -1) and (2, 2) would not count as (2, 2) can be reduced to (1, 1))?

[EDIT]: So if the magnitudes were the same, then we can use the non-normalized version without changing any of the elements as letting $a = ||v||$ would be the same for both and therefore would cancel out in the overall matrix multiplication (by $a$ and $\frac{1}{a}$)

The reason I ask is because I am wondering how likely I am to run-into a situation where orthogonal eigenvectors do not have the same magnitude. Therefore, if I weren't to normalize them individually then wouldn't that lead to a problem because $aV$ ($a$ is a constant and $V$ is the eigenvector matrix) would not lead to the non-normalized form of $V$ that we originally found - there would be extra square root terms in the numerator? I am not sure if I am being clear, but if the matrix $A$ in my problem was to yield orthogonal eigenvectors with different magnitudes, then we would need to multiply by $\sqrt{10}$ (or a multiple of that) to use the non-normalized form, which would put an extra factor into the numerators of the elements.

I hope this question makes sense. If not, please let me know and I will try to elaborate.

 

[DaveE]

When you solve for the eigenvectors the magnitude is undetermined. If (1,1)^T is an eigenvector, then so is (2,2)^T, (-4,-4)^T, etc. Any vector "in the same direction" as an eigenvector is also an eigenvector. The magnitude doesn't really matter. The length of one eigenvector has no relationship to other eigenvectors. In some cases it is convenient to normalize them to unit length, otherwise it is quite arbitrary.

 

[DaveE]

The V matrix is not unique in this calculation, it just has to be made of eigenvectors. The length of the vectors you use to construct it doesn't matter, any change will be corrected when you multiply by the inverse matrix.

 

[Master1022]

When you solve for the eigenvectors the magnitude is undetermined. If (1,1)^T is an eigenvector, then so is (2,2)^T, (-4,-4)^T, etc. Any vector "in the same direction" as an eigenvector is also an eigenvector. The magnitude doesn't really matter. The length of one eigenvector has no relationship to other eigenvectors. In some cases it is convenient to normalize them to unit length, otherwise it is quite arbitrary.

Thank you for your reply. I understand this point. Perhaps I am getting confused on what the main convention is, but you make a good point that the lengths of eigenvectors do not depend on one another. I was, incorrectly, trying to renconcile the process of normalisation with effect $aV$ and thought they were the same. However, these are not the same because, as you have pointed out, the magnitudes do not need to be the same.

I suppose the lesson is two-fold:
1) Normlisation can be done if convenient, etc. even if the magnitudes of the eigenvectors aren't the same. Whether or not I choose to normalise (in this scenario) will not affect the result
2) What I did above was incorrect (i.e. assuming the inverse is the same as the transpose) for a non-orthogonal matrix and I shall look to avoid that error in the future

 

[PeroK]

Thank you for your reply. I understand this point. Perhaps I am getting confused on what the main convention is, but you make a good point that the lengths of eigenvectors do not depend on one another. I was, incorrectly, trying to renconcile the process of normalisation with effect $aV$ and thought they were the same. However, these are not the same because, as you have pointed out, the magnitudes do not need to be the same.

I suppose the lesson is two-fold:
1) Normlisation can be done if convenient, etc. even if the magnitudes of the eigenvectors aren't the same. Whether or not I choose to normalise (in this scenario) will not affect the result
2) What I did above was incorrect (i.e. assuming the inverse is the same as the transpose) for a non-orthogonal matrix and I shall look to avoid that error in the future

There are times when you want to construct an orthogonal matrix for diagonalisation. If $A$ is a real symmetric matrix, then it can be diagonalised by an orthogonal matrix $O$:
$$A = ODO^T = ODO^{-1}$$
And the orthogonality of $O$ may be useful. In general, however, the eigenvectors may not be orthogonal and the diagonalising matrix will not be orthogonal (even if you normalise the eigenvectors). In general, you can leave the eigenvectors with any magnitude and the diagonalising process works - as shown in post #11.