Eigenproblem for non-normal matrix

[swampwiz]

I understand that a normal matrix has a complete, canonical eigendecomposition in which the normalized modal eigenvector matrix is unitary, and this its inverse is simply the transpose, and the modal eigenvalue matrix is diagonal (let's presume distinct eigenvalues). But I wonder if there is also an eigendecomposition for a non-normal matrix as well so long as the modal eigenvector matrix is invertible, although it would be clunky to have to take the inverse of [ A ], and would unclear as to whether there is a diagonalization (as opposed to being automatic for the case of a normal matrix). Perhaps what is termed an eigensolution is something that only diagonalizes, and therefore only normal matrices are considered eigensolvable?

[ A ] { x } = λ { x }

get → { λ } & [ Φ ]

{ x } = [ Φ ] { q }

[ A ] [ Φ ] { q } = λ [ Φ ] { q } = [ Φ ] [ Λ ] { q }

[ Φ ]^-1 [ A ] [ Φ ] { q } = [ Λ ] { q }

 

[StoneTemplePython]

(this post assumes we're dealing with n x n matrices with scalars in $\mathbb C$.)

Yes, normal matrices are the good stuff. And yes the conjugate transpose of a unitary matrix (collecting the eigenvectors in this case) is the inverse.

the modal eigenvalue matrix is diagonal (let's presume distinct eigenvalues). But I wonder if there is also an eigendecomposition for a non-normal matrix as well so long as the modal eigenvector matrix is invertible, although it would be clunky to have to take the inverse of [ A ], and would unclear as to whether there is a diagonalization (as opposed to being automatic for the case of a normal matrix)

no: you already stated that the eigenvalues are distinct. As you've pointed out, normal matrices are always diagonalizable. But any matrix with $n$ distinct eigenvalues must be diagonalizable irrespective of normality. That is, unique eigenvalues have linearly independent eigenvectors. (Why?) There are a few proofs of this, a lot of which rely on a contradiction. By far my favorite proof just shows this directly, using a Vandermonde matrix.

 

[swampwiz]

(this post assumes we're dealing with n x n matrices with scalars in $\mathbb C$.)

Yes, normal matrices are the good stuff. And yes the conjugate transpose of a unitary matrix (collecting the eigenvectors in this case) is the inverse.
no: you already stated that the eigenvalues are distinct. As you've pointed out, normal matrices are always diagonalizable. But any matrix with $n$ distinct eigenvalues must be diagonalizable irrespective of normality. That is, unique eigenvalues have linearly independent eigenvectors. (Why?) There are a few proofs of this, a lot of which rely on a contradiction. By far my favorite proof just shows this directly, using a Vandermonde matrix.

OK, so the results of null-space solutions to the eigenproblem is an "eigendecomposition", no matter what the eigenvector matrix looks like. It is only for a normal matrix that this solution is easily found, and also complete. And a complete eigendecomposition only requires that there be a diagonalizing matrix. Should I presume that a non-normal matrix never has a complete eigendecomposition?

 

[StoneTemplePython]

OK, so the results of null-space solutions to the eigenproblem is an "eigendecomposition", no matter what the eigenvector matrix looks like. It is only for a normal matrix that this solution is easily found, and also complete. And a complete eigendecomposition only requires that there be a diagonalizing matrix. Should I presume that a non-normal matrix never has a complete eigendecomposition?

wait what? Did you see what I wrote in my last posting about linear independence amongst unique eigenvalues?

it really is as simple as going through each eigenvalue.

you iterate through the definition for each one:

$\mathbf A \mathbf x_k = \lambda_k \mathbf x_k$

collect the relationships in a matrix

where

$\mathbf X := \bigg[\begin{array}{c|c|c|c|c} \mathbf x_1 & \mathbf x_2 &\cdots & \mathbf x_{n-1} & \mathbf x_n\end{array}\bigg]$

and diagonal matrix $\mathbf D$ is given by

$\mathbf D := \begin{bmatrix} \lambda_1 & 0 & \dots & 0 &0 \\ 0 & \lambda_2 & \dots & 0 &0 \\ \mathbf 0 & \mathbf 0 & \ddots & \mathbf 0 & \mathbf 0\\ 0& 0 & \dots & \lambda_{n-1} & 0\\ 0& 0 & \dots & 0 & \lambda_n \end{bmatrix}$

those eigenvalue relationships can be shown all at once as, in matrix form as:

$\mathbf A \mathbf X = \mathbf X \mathbf D$

(check the multiplication)

- - - -
now $\mathbf X$ is $\text{n x n}$, and you said all eigenvalues are unique, so $\mathbf X$ has $\text{n}$ linearly independent columns, and hence is invertible. multiply on the right by $\mathbf X^{-1}$ and get

$\mathbf A = \mathbf A \mathbf X \mathbf X^{-1} = \mathbf X \mathbf D \mathbf X^{-1}$

which is the diagonalization decomposition. If you prefer, you can show that $\mathbf A$ is similar to a diagonal matrix, i.e.

$\mathbf X^{-1} \mathbf A \mathbf X = \mathbf D$

- - - -

I also don't know what you mean about the decomposition being "easily found"? For large matrices, finding eigenvalues is almost always not easy (though there are a some great numerical approaches).

 

[swampwiz]

I also don't know what you mean about the decomposition being "easily found"? For large matrices, finding eigenvalues is almost always not easy (though there are a some great numerical approaches).

What I meant is that getting X^-1 is easily found since it is simply X^†; of course, this doesn't say anything about how easily X is found.

I guess that my overall question is whether the case where X^-1 is not X^† is still a valid "eigenproblem solution". (The only way that that case holds true is if X is unitary, which can only happen if the original matrix is normal.)

So if I have some Φ that is computed from the null-space solution, I define Λ (which is not necessarily diagonal, if A is not normal, or there are shared eigenvalues) as

Λ = Φ^-1 A Φ

and from which follows

Λ^† = Φ^-1 A^† Φ

so the normal products of the left & right hand sides of the eigenvalue equation are

LHS = [ A Φ ]^† [ A Φ ] = Φ^† A^† A Φ

RHS = [ Φ Λ ]^† [ Φ Λ ] = Λ^† Φ^† Φ Λ

so LHS = RHS yields

Φ^† Φ Λ Φ^-1 Φ^-† Λ^† Φ^† Φ = Λ^† Φ^† Φ Λ

which is known to work if Φ is orthogonal (i.e., unitary with possibly differently scaled columns) and Λ is normal; if A is normal and the eigenvalues are all distinct, then Φ is orthogonal and Λ is diagonal, and thus normal.

But even if these conditions are not met, it seems that it could turn out to work even if the matrices just happen to concatenate with the same net matrix product.

And it seems that even if it doesn't work out completely like this, there is still a "solution", just one that is not complete.

 

[StoneTemplePython]

I'm sorry your post doesn't make much sense to me. You seem to be creating new terms and ignoring standard well defined ones.

What I meant is that getting X^-1 is easily found since it is simply X^†; of course, this doesn't say anything about how easily X is found.

I guess that my overall question is whether the case where X^-1 is not X^† is still a valid "eigenproblem solution". (The only way that that case holds true is if X is unitary, which can only happen if the original matrix is normal.)

I don't know what constitutes a valid "eigenproblem solution".

So if I have some Φ that is computed from the null-space solution, I define Λ (which is not necessarily diagonal, if A is not normal, or there are shared eigenvalues) as

I have no idea what it means to come up with $\mathbf \Phi$ which is "computed from the null-space solution". If you are using $\mathbf \Lambda$ it better be diagonal. If you want it to be 'merely' upper triangular in general, use $\mathbf J$ or $\mathbf T$ or something like that.

If you want to write a matrix as $\mathbf A = \mathbf{PJP}^{-1}$ in terms of a Jordan Canonical Form, that's canonical and fine, but it involves a lot of heavy duty machinery. You may be interested in Schur Triangularization, $\mathbf A = \mathbf{UTU}^{*}$ with unitary $\mathbf U$ and upper triangular $\mathbf T$. Schur Triangularization unlocks spectral theory for these normal matrices you keep bringing up.

Note: "which is not necessarily diagonal, if A is not normal, or there are shared eigenvalues" is wrong.

The correct statement:
$\mathbf A$ isn't necessarily diagonalizable (if $\mathbf A$ is not normal, and there are shared / repeated eigenvalues).
- - - - -

which is known to work if Φ is orthogonal (i.e., unitary with possibly differently scaled columns) and Λ is normal; if A is normal and the eigenvalues are all distinct, then Φ is orthogonal and Λ is diagonal, and thus normal.

Again, huh?

What I think you mean is that $\mathbf \Phi$ is unitary -- the way you toggle unitary and orthogonal for a matrix is wrong... you seem to be confusing orthogonal (column) vectors with the well understood definition of an orthogonal matrix-- and again you don't need eigenvalues to be distinct. If you have repeated eigenvalues, you may still wisely select eigenvectors for any normal $\mathbf A$ such that the resulting $\mathbf \Phi$ is unitary.

- - - -
If you want to understand this stuff, you'll need to do some reading and exercises. In particular consider the freely available Linear Algebra Done Wrong, chapters 4, 5 and 6. If you want Jordan decomposition, then chapter 9 as well, though that may be out of reach (and not that great a payoff per amount of time invested).

https://www.math.brown.edu/~treil/papers/LADW/LADW_2017-09-04.pdf