Proof: extremum has a semi definitie Hessian matrix

[Coffee_]

Consider a function $f : U \subseteq \mathbb{R}^{n} -> \mathbb{R}$ that is an element of $C^{2}$ which has an minimum in $p \in U$.

According to Taylor's theorem for multiple variable functions, for each $h \in U$ there exists a $t \in ]0,1[$ such that :

$f(p+h)-f(p) = h^{T}.D^{2}(p+th).h$

Where $D^{2}$ is the Hessian or the matrix with the mixed partial derivatives, $D^{2}(p+th)$ means it's not in point p. I already assume it's a minimum so the Jacobian matrix that should be there is 0.

Now I should take a limit and somehow show that in both cases where p is a strict or not strict minimum that is $f(p)<f(x)$ of $f(p)\leq f(x)$ that in both cases $D^{2}$ will be negative semi definite. Can someone help me finish/understand this final step formally? Because I'm not sure about how a limit will work in the equation above, the left term just goes to zero.

 

[wabbit]

You could divide both sides by $||h||^2$.

 

[Coffee_]

You could divide both sides by $||h||^2$.

Not sure what that would give me. Left I'd get a similar expression to the derivative but since it's in multiple variables I'm not sure. It would be equal to the limit of $\frac{D*h}{||h||}$ I guess?

 

[wabbit]

Well you can then have $h\rightarrow 0$ but $h/||h||$ remains finite, and you could set this to be any vector $u=h/||h||$ of your choice and see what this tells you about D when h approaches 0 but u is fixed.

Or if you prefer that formulation, set $h=\theta u$ and take the limit as $\theta\rightarrow 0$

I don't know what your "*" is but if it is just usual matrix-vector multiplication, no this isn't what you get.

 

[Coffee_]

Well you can then have $h\rightarrow 0$ but $h/||h||$ remains finite, and you could set this to be any vector $u=h/||h||$ of your choice and see what this tells you about D when h approaches 0 but u is fixed.

Or if you prefer that formulation, set $h=\theta u$ and take the limit as $\theta\rightarrow 0$

I don't know what your "*" is but if it is just usual matrix-vector multiplication, no this isn't what you get.

I reasoned that that's what's it supposed to be according to $lim \frac{f(p+h)-f(p)-D*h}{||h||}=0$ where $D$ is the derivative matrix.

 

[wabbit]

But as you stated in your op, the first derivative is 0 and D is the Hessian, so suddenly using D as the jacobian instead is truly bizarre reasonning.

Oh sorry I see you were using $D^2$ to denote the hessian - please read my previous statement as referring to that.

 

[Coffee_]

But as you stated in your op, the first derivative is 0 and D is the Hessian, so suddenly using D as the jacobian instead is truly bizarre reasonning.

Oh right, should have thought a bit longer before replying.