Second derivative test

[sandy.bridge]

Homework Statement

$$\begin{pmatrix} -2 & 0 & 0\\ 0 & -2 & 0\\ 0 & 0 & -2 \end{pmatrix}$$

If I evaluate with eigenvalues, I get:
$$det\begin{pmatrix} -2-\lambda & 0 & 0\\ 0 & -2-\lambda & 0\\ 0 & 0 & -2-\lambda \end{pmatrix}=0$$

$$(-2-\lambda{)}((-2-\lambda{)}(-2-\lambda{)})=0$$
and thus
$$\lambda{=}-2$$
So there exists a local maximum according to this. However, when I evaluate with a different method offered in my textbook, I get a different result, which is confusing me as this method has always worked before.

$$det(-2)=-2<0, det\begin{pmatrix} -2 & 0\\ 0 & -2 \end{pmatrix}=4>0, det\begin{pmatrix} -2 & 0 & 0\\ 0 & -2 & 0\\ 0 & 0 & -2 \end{pmatrix}=-8<0$$
By this method it is a saddle point. Not entirely sure what is going on here.

 

[Ray Vickson]

Homework Statement

$$\begin{pmatrix} -2 & 0 & 0\\ 0 & -2 & 0\\ 0 & 0 & -2 \end{pmatrix}$$

If I evaluate with eigenvalues, I get:
$$det\begin{pmatrix} -2-\lambda & 0 & 0\\ 0 & -2-\lambda & 0\\ 0 & 0 & -2-\lambda \end{pmatrix}=0$$

$$(-2-\lambda{)}((-2-\lambda{)}(-2-\lambda{)})=0$$
and thus
$$\lambda{=}-2$$
So there exists a local maximum according to this. However, when I evaluate with a different method offered in my textbook, I get a different result, which is confusing me as this method has always worked before.

$$det(-2)=-2<0, det\begin{pmatrix} -2 & 0\\ 0 & -2 \end{pmatrix}=4>0, det\begin{pmatrix} -2 & 0 & 0\\ 0 & -2 & 0\\ 0 & 0 & -2 \end{pmatrix}=-8<0$$
By this method it is a saddle point. Not entirely sure what is going on here.

If the matrix above is the Hessian H, then: a stationary point is a strict local max if H is negative-definite, is strict local max if H is positive definite, and is a saddle point otherwise. Furthermore, *a matrix with a negative diagonal element cannot be positive-definite*, so look at K = -H and check whether it is positive-definite (because if -H is positive definite, then H is negative-definite). Obviously, K is positive-definite (by inspection, no tests needed), so your point is a strict local max.

I find some textbooks treatments to be both confusing and unnecessarily complex. The simplest way is just to always check for positive-definiteness, by changing the sign if necessary (as I did above). The positive-definiteness tests are just a little bit less confusing and are less prone to making errors.

RGV

 

[I like Serena]

Hi sandy.bridge!

Your method with sub determinants works for eigenvalues greater than zero.
Note that is the test for the matrix to be positive-definite.

However, for negative-definite, which is what you are testing, the sub determinants have to alternate between negative an positive (as they do in your case).

 

[sandy.bridge]

Thanks to both of you. Does it matter in which order they alternate? (For example, can it go +, -, +, and -, +, -?) Not sure why the textbook failed to clarify this. I really like using the sub determinant method as occasionally solving the cubic eigenvalues can be quite time consuming by hand, and time is everything on an exam.

 

[Ray Vickson]

Thanks to both of you. Does it matter in which order they alternate? (For example, can it go +, -, +, and -, +, -?) Not sure why the textbook failed to clarify this. I really like using the sub determinant method as occasionally solving the cubic eigenvalues can be quite time consuming by hand, and time is everything on an exam.

Actually, the very best way is to use Cholesky factorization, as it involves simple calculations that can be done manually (using no more than a simple hand calculator), and does not need eigenvalues, determinants, or anything else like that. It takes a few minutes to learn, however. Testing matrices up to about 10x10 is quite doable by hand with this method! In actual optimization codes for computer applications, that (or a variant of it) is used for such tests; eigenvalue or determinant methods are rarely used, as they involve many, many times as much work.

RGV

 

[sandy.bridge]

I will look into that. However, I don't think I will have time to learn a new concept regarding this before my midterm this weekend. Thus far we have only encountered 3x3 matrices.

 

[I like Serena]

I must be - + -.
Odd sizes must be negative, even sizes must be positive.

 

[sandy.bridge]

And if for example it is -, +, -, it would be considered indefinite (saddle point)?

 

[Ray Vickson]

And if for example it is -, +, -, it would be considered indefinite (saddle point)?

If the Hessian H is _diagonal_, with diagonal elements d1,d2,...,dn, then the associated quadratic form Q(h) = h^T H h has the simple form Q = d1*h1^2 + d2*h2^2 + ... + dn*hn^2. When we test for positive-definiteness, negative-definiteness or indefiniteness, we are asking whether Q(h) > 0 for all nonzero vectors h (that is the *definition* of a positive-definite matrix H) or whether Q(h) < 0 for all nonzero vectors h (= definition of negative-definite) or whether we can have Q(h1) > 0 for some vector h1 and Q(h2) < 0 for some other vector h2 (which is what _indefinite_ means). So, for a diagonal matrix there is no need to test anything: if all di > 0 it is pos.-def., if all di < 0 it is neg.-def. and if the di have mixed signs it is indef.

What is the meaning of all this (i.e., what is the intuition underlying it)? The
quadratic form Q(h) is just the second-order term in the Taylor expansion of f(x01+h1, x02+h2,...,x0n+hn) about the stationary point (x01,x02,...,x0n). Q(h) > 0 for all nonzero h means that for any nearby point y, we have f(y) > f(x0), so x0 is a strict local min; Q < 0 means the opposite, so x0 is a local max; and Q indefinite means that x0 is neither a max nor a min, because we can have f(y1) > f(x0) > f(y2) for two nearby points y1 and y2.

RGV

 

[I like Serena]

And if for example it is -, +, -, it would be considered indefinite (saddle point)?

- + - is negative-definite (maximum)
+ + + is positive-definite (minimum)
anything else that is non-zero is a saddle point
if anything is zero it is inconclusive