# Proving gradient points in the direction of maximum increase

1. May 6, 2012

### hivesaeed4

How do we prove that the gradient points in the direction of the maximum increase? Would it be enough to simply state that the gradient is just the derivates of a function w.r.t all the variables a function depends upon. Since the derivative of a term w.r.t a certain variable gives the maximum increase of that term and since in gradient not only do we account for all possible terms and all the variables upon which a term depends upon but also includes the direction, so it is logical to conclude that the gradient points in the direction of the maximum increase.

Note: I do hope that the above proof is adequate but somehow feel that the proof has to be mathematical. Could someone tell me whether or not my intuition is correct?

2. May 6, 2012

### arildno

"Since the derivative of a term w.r.t a certain variable gives the maximum increase of that term ".
Does it?

3. May 6, 2012

### hivesaeed4

Sorry. It gives the rate of change of a term.

Then how do I prove that the gradient points in the direction of the maximum increase?

4. May 6, 2012

### arildno

Have you learnt about directional derivatives yet?

5. May 6, 2012

### arildno

Ok:
I just saw that you DO know about directional derivatives, on basis of another thread posted by you!

Now, you now know that the directional derivative is the dot/scalar product between the gradient vector and the direction vector.

Can you rewrite a dot/scalar product between two vectors in another way, a way involving the magintudesof the two vectors?

6. May 7, 2012

### hivesaeed4

Yes. But that would involve the cos of the angle between the 2 vectors. But how does that prove that the gradient gives the us maximum rate of increase?

7. May 7, 2012

### Skrew

I havent done this but as a suggestion.

Make the directional derivative a function of theta and maximize it.

8. May 7, 2012

### arildno

For what angle between the vectors does the cosine function receive its maximum value?

9. May 7, 2012

### hivesaeed4

zero.

10. May 7, 2012

### arildno

Indeed!

And, therefore:
What angle between the direction vector and the gradient vector maximizes the dot product between them?

11. May 9, 2012

### hivesaeed4

zero?

12. May 9, 2012

### arildno

Right!

And, if two vectors have zero angle between them, then they are parallell vectors, meaning that the direction of maximal increase is in the direction of the.....?

13. May 9, 2012

### hivesaeed4

In the direction of both vectors.

14. May 9, 2012

### hivesaeed4

So is that the required proof?

15. May 9, 2012

### arildno

Yes.
In particular, in the direction of the gradient.

16. May 9, 2012

### arildno

What do you think is lacking/unsatisfactory?

17. May 9, 2012

### mathwonk

the answer depends on your definition of the gradient. if you think the gradient is the vector that dots with a direction vector to give the directional derivative, then the previous explanation is all you need.

I you think the gradient is a vector whose entries are the partial derivatives, then you need to make sense out of that a bit more. i.e. you need the chain rule to get the previous statement.

18. May 9, 2012

### arildno

That is why I asked OP what, if anything, he found lacking.

However, since he had already posted other threads in which he showed familiarity with the concept of the directional derivative, I assumed he was also familiar with how that is derived.
Therefore, I did not address that point explicitly.