Proving $f(x_k+εp)<f(x_k)$ with Taylor Series

[i_a_n]

Prove that if $p^T▽f(x_k)<0$, then $f(x_k+εp)<f(x_k)$ for $ε>0$ sufficiently small.

I think we can expand $f(x_k+εp)$ in a Taylor series about the point $x_k$ and look at $f(x_k+εp)-f(x_k)$, but what's then? (Taylor series: $f(x_0+p)=f(x_0)+p^T▽f(x_0)+(1/2)p^T▽^2f(x_0)p+...$
=> here is what's $p$)

 

[i_a_n]

can anyone help me?

 

[MarkFL]

We ask that you do not bump a topic unless you have something to add, such as further information or other things you have tried to work the problem.

With most people busy with family and other things during the weekends, you will probably get responses beginning tomorrow, but I make no promises you understand. I am just trying to explain that our helpers are not online as much during the weekends.

No topics get ignored here, it just takes the right person (i.e. who knows how to help) to come along when they have time to be online.

 

[I like Serena]

Prove that if $p^T▽f(x_k)<0$, then $f(x_k+εp)<f(x_k)$ for $ε>0$ sufficiently small.

I think we can expand $f(x_k+εp)$ in a Taylor series about the point $x_k$ and look at $f(x_k+εp)-f(x_k)$, but what's then? (Taylor series: $f(x_0+p)=f(x_0)+p^T▽f(x_0)+(1/2)p^T▽^2f(x_0)p+...$
=> here is what's $p$)

You have:

$f(x_k+εp)=f(x_k)+(εp)^T▽f(x_0)+(1/2)(εp)^T▽^2f(x_k)(εp)+... \qquad (1)$​

Let $b = p^T▽f(x_k)$, so $b < 0$.
Let $R=(1/2)(εp)^T▽^2f(x_k)(εp)+...$.

Then (1) becomes:

$f(x_k+εp)=f(x_k) + εb + R \qquad (2)$​

The absolute value of the remainder terms R is less than the absolute value of the first order term for $ε>0$ sufficiently small:

$|R| < |εb|$

$R < -εb$

$εb + R < 0 \qquad (3)$​

Combining (2) and (3):

$f(x_k+εp)=f(x_k) + εb + R < f(x_k)$ $\qquad \blacksquare$​

Btw, before I did not understand what p was, nor how $\nabla$ was intended.
Seeing no effort either I ignored this thread.
Apparently you added an extra explanation later, so here you go. :)

 

[i_a_n]

You have:

$f(x_k+εp)=f(x_k)+(εp)^T▽f(x_0)+(1/2)(εp)^T▽^2f(x_k)(εp)+... \qquad (1)$​

Let $b = p^T▽f(x_k)$, so $b < 0$.
Let $R=(1/2)(εp)^T▽^2f(x_k)(εp)+...$.

Then (1) becomes:

$f(x_k+εp)=f(x_k) + εb + R \qquad (2)$​

The absolute value of the remainder terms R is less than the absolute value of the first order term for $ε>0$ sufficiently small:

$|R| < |εb|$

$R < -εb$

$εb + R < 0 \qquad (3)$​

Combining (2) and (3):

$f(x_k+εp)=f(x_k) + εb + R < f(x_k)$ $\qquad \blacksquare$​

Btw, before I did not understand what p was, nor how $\nabla$ was intended.
Seeing no effort either I ignored this thread.
Apparently you added an extra explanation later, so here you go. :)

why $|R| < |εb|$? Is that simply because $ε>0$ is sufficiently small?

 

[I like Serena]

why $|R| < |εb|$? Is that simply because $ε>0$ is sufficiently small?

Yes.
The remainder terms consist of higher powers in ε than the εb-term.
As long as the series converges the εb-term (which is non-zero) will be larger than R if ε is small enough.