Gradient Ascent: Finding Maximum of a Scalar Function

[gaganaut]

hi all,

Couple of months ago I had an entrance exam wherein this problem appeared. (I hope this is what it was).

For a scalar function $$f\left(x\right)=f\left(x_{1},x_{2},...,x_{n}\right)$$ the gradient is given as
$$\nabla f=\left(\frac {\partial f \left(x\right)} {\partial x_1},\frac {\partial f \left(x\right)} {\partial x_2},...,\frac {\partial f \left(x\right)} {\partial x_n}\right)$$
Then show that for any small change $$\Delta x$$, $$f\left(x+\Delta x\right)$$ is maximum if $$\Delta x$$ lies along $$\nabla f\left(x\right)$$.

Frankly, I did not get the question then. So I did few preliminary steps as follows.
$$f\left(x+\Delta x\right)=f\left(x\right)+\nabla f\left(x\right)\cdot\Delta x + O\left(\left(\Delta x\right)^2\right)$$

Hence,
$$\nabla f\left(x+\Delta x\right)=\nabla f\left(x\right)+\nabla \left(\nabla f\left(x\right)\cdot\Delta x\right)$$

For $$f\left(x+\Delta x\right)$$ to be maximum, $$\nabla f\left(x+\Delta x\right)=0$$ is maximum

Hence,
$$\nabla f\left(x\right)=-\nabla \left(\nabla f\left(x\right)\cdot\Delta x\right)$$

This is where I gave up then. Bu intuition I could see that the dot product would give a maximum answer if $$\Delta x$$ lies along $$\nabla f\left(x\right)$$. But I could not prove it.

So can somebody help me with this as this question is haunting me for last two months now.

Thanks in advance.

 

[arildno]

Let $$g_{n}(t)=f(x_{1}+tn_{1},x_{2}+tn_{2},...,x_{n}+tn_{n})$$
where $$\vec{n}=(n_{1},n_{2},...,n_{n})$$ is a unit vector in direction n.

Thus, g_{n}(t) measures the increase of f along some direction.

At t=0, we are at the point $$\vec{x}_{0}=(x_{1},x_{2}...,x_{n})$$

Now, the direction for maximal growth of f at $\vec{x}_{0}$ will be finding the maximum of $$\frac{dg_{n}}{dt}|_{t=0}$$, considered as a function of [itex]\vec{n}[/tex]

By aid of the chain rule, this transforms to finding the maximum of:
$$\nabla{f}_{\vec{x}=\vec{x}_{0}}\cdot\vec{n}$$

Clearly, $\vec{n}$ must be parallell to the gradient in order to maximize the expression, which proves the result.

 

[gaganaut]

Thanks for the reply

Thank you so much for the reply. I appreciate that. We can now say that this question has been resolved.