Why is a function decreasing the fastest in dir of neg of the gradient?

Click For Summary
SUMMARY

The discussion clarifies that a function decreases fastest in the direction of the negative gradient under specific conditions, primarily when the function lacks saddle points. The analogy of a mountain illustrates that steepest descent applies only when the function's surface is symmetric and positive definite. The validity of steepest descent methods, such as the conjugate gradient method, is contingent on the presence of a nonhorizontal tangent plane at the point of interest. The reference to Jonathan Shewchuk's work emphasizes the importance of understanding these conditions for accurate application of gradient-based optimization techniques.

PREREQUISITES
  • Understanding of gradient vectors and their significance in optimization
  • Familiarity with concepts of saddle points and positive definiteness
  • Knowledge of steepest descent and conjugate gradient methods
  • Basic comprehension of tangent planes in multivariable calculus
NEXT STEPS
  • Read 'Introduction to the conjugate gradient method without the agonizing pain' by Jonathan Shewchuk
  • Study the properties of symmetric positive definite matrices
  • Explore the implications of saddle points in optimization problems
  • Learn about the geometric interpretation of gradient vectors in multivariable functions
USEFUL FOR

Mathematicians, data scientists, and machine learning practitioners interested in optimization techniques, particularly those utilizing gradient-based methods for function minimization.

toofle
Messages
20
Reaction score
0
Why is the function decreasing the fastest in the direction of the negative of the gradient?

Just because it increases the fastest in the direction of the positive of the gradient why does this have to mean it has to decrease the fstest in the negative of the gradient?
If you stand facing a steep mountain and ityou have flats behind you and a huge drop on your sides, it would decrease faster to 90 degress than behind you. What am I not understanding with gradients?
 
Physics news on Phys.org
toofle said:
Why is the function decreasing the fastest in the direction of the negative of the gradient?

It doesn't (always), exactly for the reason you are giving. Your mountain and your function are not allowed to have saddle points.
There cannot be valleys where straight ahead and behind you, you go up, and left and right of you, you go down. Symmetric (positive) definiteness is often a requirement to guarantee this.

If this is about steepest descent or conjugate gradient methods, You should read 'Introduction to the conjugate gradient method without the agonizing pain' by Jonathan Shewchuk.
 
This is only claimed to be valid at points where the function surface has a nonhorizontal tangent plane. If there is such a plane, walking at the surface near the point can be approximated with walking at the tangent plane. Wouldn't you then agree that the directions of steepest increase and decrease are opposite?
 

Similar threads

  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 7 ·
Replies
7
Views
8K
Replies
15
Views
5K
  • · Replies 32 ·
2
Replies
32
Views
4K
  • · Replies 27 ·
Replies
27
Views
8K
  • · Replies 13 ·
Replies
13
Views
3K
  • · Replies 9 ·
Replies
9
Views
3K
Replies
12
Views
6K
Replies
2
Views
1K