## Main Question or Discussion Point

What is "Negative Gradient" ? and what is "Gradient Descent Method" ? What is the difference and relationship between them ?
What is the benefit each of them ?

Last edited:

jtbell
Mentor
Do you know how to calculate the gradient of a function, in vector calculus, and what it means geometrically?

Do you know how to calculate the gradient of a function, in vector calculus, and what it means geometrically?
thanks a lot

No, I am not so good in math , I know this is a stupid question, but if you can answer it, I will appreciated that

jtbell
Mentor
You can find out how to calculate the gradient in any calculus textbook that includes multivariable calculus (vector calculus), and probably on hundreds of web sites including Wikipedia (http://en.wikipedia.org/wiki/Gradient), so I won't do that here. I'll just talk about the meaning of the gradient.

Suppose you have a function h(x,y) that tells you the elevation (height) of the land at horizontal coordinates (x,y). The gradient of this function, ##\vec \nabla h(x,y)##, is a vector function that gives you a vector for each point (x,y). This gradient vector points in the direction of steepest uphill slope, and its magnitude is the value of that slope (like the slope of a straight-line graph).

The opposite direction, the negative gradient ##-\vec \nabla h(x,y)## tells you the direction of steepest downhill slope.

If you want to find the location (x,y) at which h(x,y) is minimum (e.g. the bottom of a valley), one way is to follow the negative gradient vector downhill. Calculate ##-\vec \nabla h## at your starting point (x0, y0), take a step downhill in that direction to the point (x1, y1), calculate ##-\vec \nabla h## at that point, take a step in the new downhill direction, etc. Keep going until you find yourself at a higher elevation at the end of a step, indicating that you have gone past the bottom.