Use of a derivative or a gradient to minimize a function

Click For Summary
SUMMARY

The discussion focuses on the methods for minimizing a function f(x), specifically contrasting the traditional approach of setting the derivative to zero (referred to as 'method I') with the gradient descent technique. While 'method I' is effective for functions that can be differentiated analytically, it fails in scenarios where analytical differentiation is impossible, such as when dealing with empirical data represented as pairs. An example provided illustrates the limitations of 'method I' when faced with a non-analytic function, necessitating the use of gradient descent for effective minimization.

PREREQUISITES
  • Understanding of calculus, specifically derivatives
  • Familiarity with optimization techniques
  • Knowledge of gradient descent algorithm
  • Basic concepts of numerical approximation methods
NEXT STEPS
  • Study the principles of gradient descent and its applications in optimization
  • Learn about numerical approximation techniques for functions
  • Explore scenarios where empirical data is used for function minimization
  • Investigate advanced optimization algorithms beyond gradient descent
USEFUL FOR

Mathematicians, data scientists, and engineers involved in optimization problems, particularly those working with empirical data and requiring numerical methods for function minimization.

onako
Messages
86
Reaction score
0
Given certain function f(x), a standard way to minimize it is to set its derivative to zero, and solve for x. However, in certain cases the method of gradient descent is used; compared to the previous method (call it 'method I')that simply sets the derivative to zero and solves for x, the gradient descent takes multiple steps.

Why could not one use only the 'method I' for minimization? Could you give an example illustrating the difficulty of applying 'mehtod I'?
 
Physics news on Phys.org
The standard situation is where you cannot differentiate the function analytically and have to use a numerical approximation.
 
Could you provide a simple example?
 
Any situation in which your data is given as a set of pairs from an experient rather than as an analytic function.
 

Similar threads

Undergrad Understand the delta rule increment in gradient descent
  • · Replies 18 ·
Replies
18
Views
3K
Undergrad What Exactly is Step Size in Gradient Descent Method?
  • · Replies 3 ·
Replies
3
Views
3K
Undergrad Gradient descent algorithm and optimizers
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad Gradient Vectors: Perpendicular to Level Curves
  • · Replies 4 ·
Replies
4
Views
2K
Graduate On uniform boundedness of the GD algorithm
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Why is this closed line integral zero?
  • · Replies 8 ·
Replies
8
Views
3K
Graduate How to take the spatial derivative of quaternions
  • · Replies 8 ·
Replies
8
Views
3K
Undergrad Question about gradient, tangent plane and normal line
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Gradient With Respect to a Set of Coordinates
  • · Replies 3 ·
Replies
3
Views
2K
MHB Calculus & Diff. Eqn: Beginner Qs on Function, Derivative & Gradient
  • · Replies 14 ·
Replies
14
Views
3K