Gradient Ascent: Finding Maximum of a Scalar Function

[gaganaut]

hi all,

Couple of months ago I had an entrance exam wherein this problem appeared. (I hope this is what it was).

For a scalar function f\left(x\right)=f\left(x_{1},x_{2},...,x_{n}\right) the gradient is given as
\nabla f=\left(\frac {\partial f \left(x\right)} {\partial x_1},\frac {\partial f \left(x\right)} {\partial x_2},...,\frac {\partial f \left(x\right)} {\partial x_n}\right)
Then show that for any small change \Delta x, f\left(x+\Delta x\right) is maximum if \Delta x lies along \nabla f\left(x\right).

Frankly, I did not get the question then. So I did few preliminary steps as follows.
f\left(x+\Delta x\right)=f\left(x\right)+\nabla f\left(x\right)\cdot\Delta x + O\left(\left(\Delta x\right)^2\right)

Hence,
\nabla f\left(x+\Delta x\right)=\nabla f\left(x\right)+\nabla \left(\nabla f\left(x\right)\cdot\Delta x\right)

For f\left(x+\Delta x\right) to be maximum, \nabla f\left(x+\Delta x\right)=0 is maximum

Hence,
\nabla f\left(x\right)=-\nabla \left(\nabla f\left(x\right)\cdot\Delta x\right)

This is where I gave up then. Bu intuition I could see that the dot product would give a maximum answer if \Delta x lies along \nabla f\left(x\right). But I could not prove it.

So can somebody help me with this as this question is haunting me for last two months now.

Thanks in advance.

 

[arildno]

Let g_{n}(t)=f(x_{1}+tn_{1},x_{2}+tn_{2},...,x_{n}+tn_{n})
where \vec{n}=(n_{1},n_{2},...,n_{n}) is a unit vector in direction n.

Thus, g_{n}(t) measures the increase of f along some direction.

At t=0, we are at the point \vec{x}_{0}=(x_{1},x_{2}...,x_{n})

Now, the direction for maximal growth of f at \vec{x}_{0} will be finding the maximum of \frac{dg_{n}}{dt}|_{t=0}, considered as a function of \vec{n}[/tex]<br /> <br /> By aid of the chain rule, this transforms to finding the maximum of:<br /> \nabla{f}_{\vec{x}=\vec{x}_{0}}\cdot\vec{n}<br /> <br /> Clearly, \vec{n} must be parallell to the gradient in order to maximize the expression, which proves the result.

 

[gaganaut]

Thanks for the reply

Thank you so much for the reply. I appreciate that. We can now say that this question has been resolved.