Is there a formula for calculating the derivative of the largest eigenvalue?

[Leo321]

Assume A=f(t) is an nxn matrix with all elements being non-negative. The eigenvalue with the largest absolute value is real and positive. We will call it r.
Is there an analytical way to calculate dr/dt, or at least find the values of t for which dr/dt=0?

 

[mattrb]

Leo,

The largest eigenvalue of a matrix is a continuous function of A. (Demmel Applied Numerical Linear Algebra). If the largest eigenvalue has multiplicity 1 then it should be possible to calculate the derivative of the largest eigenvalue should be defined. I would consider your matrix as being $$A[t]=A[t_0]+A'[t_0](t-t_0)+...$$ (i.e. ignore second order time derivative terms). Then consider $$A=X\Sigma X^{-1}$$ (the diagonalization). Differentiating you get.
$$d\lambda_1/dt=y_1^*A'[t_0]x_1$$. Where $$y_1$$ and $$x_1$$ are the left and right eigenvectors of $$A[t_0]$$ respectively corresponding to the largest eigenvalue. They are the first row of $$X^{-1}$$ and the first column of $$X$$ respectively if the eigenvalues are sorted in descending order in $$\Sigma$$.
"www.ics.forth.gr/cvrl/publications/.../2000_eccv_SVD_jacobian.pdf"[/URL] gives the procedure for doing this with the SVD. I just generalized the result. Please check it over and do the detailed computations before using this formula.

I hope I was able to help!

Matthew

 

[AlephZero]

Leo,

The largest eigenvalue of a matrix is a continuous function of A. (Demmel Applied Numerical Linear Algebra).
Matthew

I'm not convinced about that for a general matrix (see below) - though it may be true if the largest eigenvalue is always real.

But the largest EV is not a differentiable function of A (at least, not everywhere) even if it is continuous.

Counterexample: a 2x2 diagonal matrix with A11 = 1, A12 = t

The max eigenvalue = 1 if t <= 1, t if t >= 1. This is not differentiable when t = 1.

The reason is connected to the fact that the corresponding eigenvector is not even a continuous function. It "flips" from (1, 0) to (0, 1) when t passes through 1.

It is easy to construct an 4x4 matrix with two pairs of complex conjugate eigenvalues, where the largest eigenvalue (maximum absolute value) is not even continuous when it "jumps" from being a member of one pair to a member of the other pair. The OP said the largest eigenvalue is real, but it's hard to characterize what that means for a completely general matrix.

 

[mattrb]

Aleph, you're perfectly right about the complex example and the general non-differentiability of eigenvalues. Eigenvectors are ill-posed when there are duplicate eigenvalues, since any linear combination of the two (or more) eigenvectors is also an eigenvector. However, Demmel states that eigenvalues are always continuous functions of the matrix entries, even if they are not differentiable. The max function applied to real functions does not affect continuity though it might affect differentiability (if two eigenvalues are equal). 'largest' is subjective when using the complex number system.

Matthew

 

[AlephZero]

If you know an eigenvalue $$\lambda$$ and the corresponding normalized eigenvector $$x$$ you can find derivative like this:

You know $$Ax = \lambda x$$ and $$x^T x = 1$$

$$(A + dA)(x + dx) = (\lambda + d\lambda)(x + dx)$$

Ignoring second order terms, that expands to

$$A.dx + dA.x = \lambda.dx + d\lambda.x$$

You also want $$(x + dx)$$ to be normalized. That gives

$$(x + dx)^T(x+dx) = 1$$

Ignoring second order terms, that expands to

$$x^T dx = 0$$ (which is an interesting fact in its own right)

So you have $$n+1$$ equations that you can solve for the $$n$$ components of $$dx$$, and the value of $$d\lambda$$.

If $$A = A(t)$$ you can obviously recast the idea in terms of derivatives w.r.t. $$t$$.

If all the eigenvalues are distinct, this works for any eigenpair, not just the largest. And if all the eigenvalues are distinct there is no ambiguity about which is the largest, so my earlier quibbles don't apply.