Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Gradient Ascent question

  1. Feb 4, 2008 #1
    hi all,

    Couple of months ago I had an entrance exam wherein this problem appeared. (I hope this is what it was).

    For a scalar function [tex]f\left(x\right)=f\left(x_{1},x_{2},...,x_{n}\right)[/tex] the gradient is given as
    [tex]\nabla f=\left(\frac {\partial f \left(x\right)} {\partial x_1},\frac {\partial f \left(x\right)} {\partial x_2},...,\frac {\partial f \left(x\right)} {\partial x_n}\right)[/tex]
    Then show that for any small change [tex]\Delta x[/tex], [tex]f\left(x+\Delta x\right)[/tex] is maximum if [tex]\Delta x[/tex] lies along [tex]\nabla f\left(x\right)[/tex].

    Frankly, I did not get the question then. So I did few preliminary steps as follows.
    [tex]f\left(x+\Delta x\right)=f\left(x\right)+\nabla f\left(x\right)\cdot\Delta x + O\left(\left(\Delta x\right)^2\right)[/tex]

    [tex]\nabla f\left(x+\Delta x\right)=\nabla f\left(x\right)+\nabla \left(\nabla f\left(x\right)\cdot\Delta x\right) [/tex]

    For [tex]f\left(x+\Delta x\right)[/tex] to be maximum, [tex]\nabla f\left(x+\Delta x\right)=0[/tex] is maximum

    [tex]\nabla f\left(x\right)=-\nabla \left(\nabla f\left(x\right)\cdot\Delta x\right) [/tex]

    This is where I gave up then. Bu intuition I could see that the dot product would give a maximum answer if [tex]\Delta x[/tex] lies along [tex]\nabla f\left(x\right)[/tex]. But I could not prove it.

    So can somebody help me with this as this question is haunting me for last two months now.

    Thanks in advance.
  2. jcsd
  3. Feb 4, 2008 #2


    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    Dearly Missed

    Let [tex]g_{n}(t)=f(x_{1}+tn_{1},x_{2}+tn_{2},....,x_{n}+tn_{n})[/tex]
    where [tex]\vec{n}=(n_{1},n_{2},....,n_{n})[/tex] is a unit vector in direction n.

    Thus, g_{n}(t) measures the increase of f along some direction.

    At t=0, we are at the point [tex]\vec{x}_{0}=(x_{1},x_{2}...,x_{n})[/tex]

    Now, the direction for maximal growth of f at [itex]\vec{x}_{0}[/itex] will be finding the maximum of [tex]\frac{dg_{n}}{dt}|_{t=0}[/tex], considered as a function of [itex]\vec{n}[/tex]

    By aid of the chain rule, this transforms to finding the maximum of:

    Clearly, [itex]\vec{n}[/itex] must be parallell to the gradient in order to maximize the expression, which proves the result.
  4. Feb 4, 2008 #3
    Thanks for the reply

    Thank you so much for the reply. I appreciate that. We can now say that this question has been resolved.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook