Q about 2nd derivative test for multivariable functions

[kostoglotov]

Homework Statement

So the test is to take the determinant (D) of the Hessian matrix of your multivar function.

Then if D>0 & fxx>0 it's a min point, if D>0 & fxx<0 it's a max point.

For D<0 it's a saddle point, and D=0 gives no information.

My question is, what happens if fxx=0? Is that equivalent to D=0?

 

[wabbit]

Your first statement isn't true. D>0 doesn't tell much of anything about the type of extremum in the general multivariate case. What is the complete statement of your problem ?

 

[kostoglotov]

Your first statement isn't true. D>0 doesn't tell much of anything about the type of extremum in the general multivariate case. What is the complete statement of your problem ?

It's not my statement, it's the textbooks...there's no specific problem to which I can refer. I was just wondering what it tells us if fxx=0.

In the textbook (Stewart's Calculus 6 edt. p960 ) it says

Suppose the second partial derivatives of f are continuous on a disk with center (a,b) and suppose that fx(a,b) = 0 and fy(a,b) = 0 [that is, (a,b) is a critical point of f], Let

D = D(a,b) = fxx(a,b)fyy(a,b) - [fxy(a,b)]^2

(a) If D > 0 and fxx > 0, then f(a,b) is a local minimum

(b) If D > 0 and fxx < 0, then f(a,b) is a local maximum

(c) If D < 0 then f(a,b) is not a local min or max, it is a saddle point

 

[Ray Vickson]

It's not my statement, it's the textbooks...there's no specific problem to which I can refer. I was just wondering what it tells us if fxx=0.

In the textbook (Stewart's Calculus 6 edt. p960 ) it says

Suppose the second partial derivatives of f are continuous on a disk with center (a,b) and suppose that fx(a,b) = 0 and fy(a,b) = 0 [that is, (a,b) is a critical point of f], Let

D = D(a,b) = fxx(a,b)fyy(a,b) - [fxy(a,b)]^2

(a) If D > 0 and fxx > 0, then f(a,b) is a local minimum

(b) If D > 0 and fxx < 0, then f(a,b) is a local maximum

(c) If D < 0 then f(a,b) is not a local min or max, it is a saddle point

You and Wabbit are both right. For the special case of 2 variables, $f_{xx} > 0$ and $D > 0$ guarantee a strict local min. However, for 3 or more variables, having $D > 0$ really does not give much useful information at all. For example, for $f(x,y,z) = x^2-y^2-z^2$, the point (0,0,0) is stationary, and the Hessian is
$$H = \left[ \begin{array}{ccc}2 &0 & 0 \\ 0 & -2 & 0 \\ 0 & 0 & -2\end{array} \right]$$
which has $f_{xx} > 0$ and $D > 0$. But (0,0,0) is a saddle point.

In general, it is necessary (not sufficient) that all $f_{x_i x_i} > 0$ in order to have a strict local min. For the 2-variable case, having $f_{xx} > 0$ and $D > 0$ automatically guarantees $f_{yy} > 0$ as well.

 

[wabbit]

@kostoglotov. Yes as explained by Ray your statement is correct in the bivariate case and then fxx=0 leads to D<0 or D=0, it only excludes the case D>0.

And no, it's not the textbook that is to blame.

Your statement is false because it says "multivar", The textbook is correct because it does no such thing and talks about functions of two variables.

 

[kostoglotov]

@kostoglotov. Yes as explained by Ray your statement is correct in the bivariate case and then fxx=0 leads to D<0 or D=0, it only excludes the case D>0.

And no, it's not the textbook that is to blame.

Your statement is false because it says "multivar", The textbook is correct because it does no such thing and talks about functions of two variables.

Ah, ok, I'm being loose with my terminology. I'll take note of that! Thanks.

 

[HallsofIvy]

A little bit more detail: strictly speaking, "the derivative" of a multi-variable function is the gradient vector- the vector whose components, in a given coordinate system, are the partial derivatives of the function and the second derivative is the Hessian- the matrix having all second partial derivatives as components. Since $f_{xy}= f_{yx}$, this is a symmetric matrix and so is diagonalizable. That is, there exist a coordinate system in which the "mixed" second partials are 0.

Now, if the second derivatives at a critical point, in this coordinate system, are all positive the point is "relative minimum". If they are all negative a "relative maximum". If there are some positive and some negative second derivatives we have a "saddle point".

For two variables, it is fairly easy to see which we have without having to change coordinate system. If the matrix of second derivatives is
$$\begin{bmatrix} f_{xx} & f_{xy} \\ f_{xy} & f_{yy} \end{bmatrix}$$
then its determinant is $f_{xx}f_{yy}- f_{xy}^2$. Since the determinant is independent of coordinate changes, that would be the same as $f_{x'x'}f_{y'y'}$ in a coordinate system in which the mixed partials were 0. So if that determinant were positive, $f_{x'x'}$ and $f_{y'y'}$ must have the same sign so the critical point is either a maximum or a minimum. We can determine which by looking at either $f_{x'x'}$ or $f_{y'y'}$ separately. If that determinant were negative then we would have a saddle point.

But with more than two coordinates, it is no longer that simple. Now, the determinant might be positive if there are an even number of negative second derivatives in that diagonalized matrix. For example, in three dimensions, if the determinant is positive, we might have all three second derivatives positive, a minimum, or we might have two negative and one positive, a saddle point.

The only way to be sure is to find the specific eigenvalues of the Hessian matrix since they will be the numbers on the diagonal matrix. If all eigenvalues are positive, we have a minimum, if all are negative, a maximum, and, if some are positive and some negative, a saddle point.

 

[Ray Vickson]

Ah, ok, I'm being loose with my terminology. I'll take note of that! Thanks.

In the n-variable case, the stationary point $\vec{x}_0 = (x_{10}, x_{20}, \ldots, x_{n0})$ is a strict local minimizing point of $f(\vec{x})$ if the Hessian matrix $H_0 = H(\vec{x}_0)$ is positive-definite. One test for this is that all the principle minors of $H_0$ be positive; that is, the $1\times 1$ upper-left determinant should be > 0---and of course, that is just the statement $f_{x_1 x_1} > 0$; as well, the upper left $2 \times 2$ determinant should be >0, the upper-left $3 \times 3$ determinant should be >0, etc. In other words, the upper left submatrices having rows $1\ldots i$ and columns $1\ldots i$ should all have positive determinants for $i = 1, 2, \ldots, n$. Note that for the case $n = 2$ this is exactly what your textbook says.

If the Hessian is "indefinite", the point $\vec{x}_0$ is a saddle point; if the Hessian is not indefinite but is only positive semi-definite, we cannot say: the point could be a minimum or a saddle point (but for sure it is not a maximum).

 

[kostoglotov]

That's fascinating. I'm doing the MIT linear algebra course in the second half of this year, I'll have to bookmark your post and come back to it when I understand changing of coordinate systems and eigen-stuff better! Thanks.