Why Is a Quadratic Approximation Used for Critical Points of Functions?

[Benny]

Hi, I posted a related question (about the 2nd derivative test) a while ago but now that I've read through some relevant theory I'm in a better position to find out about the following.

In the notes I have, the behaviour of a function f:U \subset R^n \to R near a critical point point \mathop a\limits^ \to is considered. To do this, a 'quadratic' approximation to f near a is considered.

<br /> f\left( {\mathop a\limits^ \to + \mathop h\limits^ \to } \right) \approx f\left( {\mathop a\limits^ \to } \right) + \frac{1}{2}\left( {D_{\mathop h\limits^ \to } } \right)^2 f\left( {\mathop a\limits^ \to } \right)<br />

Where D_h is the directional derivative of f in the direction of h.

Then there's some stuff on matrix representation, eigenvalues and Hessians. The important point is when the determinant of the Hessian is equal to zero in which case the test of the nature of the critical point is inconclusive. Consequently, there is a need to look at higher derivatives.

My questions are; why is a 'quadratic' approximation considered in stead of for instance a 'linear' or a 'quartic' approximation? Is it because the 2nd derivative test is supposed to use 2nd derivatives? Also, how does looking at higher derivatives (when the 2nd derivative test is inconclusive) help?

Does the math just work out? If the 2nd derivative test is inconclusive then how does considering higher derivatives help? If I use a 3rd degree taylor polynomial to approximate f near a then that's just a very rough approximation. How can I be sure about the nature of the stationary point from the approximation? Any help would be good thanks.

 

[HallsofIvy]

A 'quadratic' approximation is considered instead of 'linear' approximation precisely because in this case the first derivatives are 0. The 'linear' approximation is no different from the constant value and gives no more information. As long as the second derivatives are not 0 then the 'quadratic' approximation is the next best. Of course, it might happen that both first and second derivatives are 0- then you would have to go to a cubic approximation.

1 dimensional examples: y= x² has derivative 0 at x= 0 and so a critical point there. The second derivative is 2 which is positive and so x= 0 is a minimum. The "Taylor's polynomial" expansion about x= 0 is, of course, just 0+ 0x+ x². If y= x³, again the first derivative is 0 and so there is a critical point there. Checking the second derivative, we see that it is also 0 so we need to go to the 3rd derivative. Here we see that the Taylor's polynomial about x= 0 is
0+ 0x+ 0x²+ x³ and so x= 0 is a "saddle point".

You use the derivative where you finally get something other than 0.

 

[Benny]

Thanks for the help. But I'm not sure how you concluded that f(x) = x^3 has saddle point at x = 0 from the fact that you need to differentiate 3 times to get to a derivative which is non-zero at x = 0. If f(x) = x^4 then then I would need to differentiate 4 times to get to a derivative which is non-zero. But in this case x = 0 corresponds to a local minimum.

 

[TD]

In these cases, when the first non-zero derivative is of odd order, then we have a point of inflection (if I'm not mistaking) and not an extremum.

 

[matt grime]

I would imagine the conclusion was reached because it's obvious what that cubic looks like, ie use your common sense: if it locally looks like y=x^3 then it is a saddle, if locally it looks like x^4 then it will be a maximum.

 

[Benny]

I asked the question because I am not sure what to do with a function of two variables when the Hessian matrix corresponding to a critical point is the zero matrix. I got the impression that there was a reason for using f(x) = x^3 as an example. Usually, I don't find it to be all that easy to visualise a 3 dimensional surface.

Of course its obvious what the nature of the critical point of f(x) = x^3 at x = 0 is. The nature of the critical point of f(x,y) = sin(x) + sin(y) + sin(x+y) at (x,y) = (pi,pi) is not so obvious. I could 'plug' that point into the equation and I'll find that f = 0 at that point. But what do I do then? Consider the behaviour of f along lines like x = 0, y = 0, y = x? I don't know how to utilise that method and even then it seems like a fudge method to determine the nature of the critical point.

By the way, if f(x,y) is continuous in all of R^2 and I have a critical point which isn't a local maximum or minimum then is that point necessarily a saddle?

 

[HallsofIvy]

Look at the graphs! f(x,y)= a+ x²+ y² has a minimum at (0,0), f(x,y)= a- x²- y² has a maximum at (0,0), f(x,y)= a- x²+ y² or f(x,y)= a+ x²- y[/sup]2[/sup] has a saddle point at (0,0). For a more complicated function, write out the Taylor's series. At a critical point, the linear terms will be 0 so look at the quadratic terms- they must be one of the above. If the second derivatives are also 0 then the quadratic terms are also 0 so you need to look at the third derivatives. But of course x³ or x²y or xy² or y³ are all less than 0 on one side and greater than 0 on the other- so saddle point. If it happens that all first, second, and third go on to the fourth derivatives to look at the fourth powers.

By the way, if f(x,y) is continuous in all of R^2 and I have a critical point which isn't a local maximum or minimum then is that point necessarily a saddle?

Except, of course, if f(x,y) is a constant.

 

[Benny]

Ok thanks for the help.