Question about Newton's method for solving a function

Click For Summary
SUMMARY

This discussion focuses on Newton's method for optimization, specifically addressing the necessity of the full Hessian matrix. The user, Ed, inquires whether it is feasible to use only the diagonal elements of the Hessian for approximating Newton's method. The response confirms that approximations to the Hessian are valid and introduces the concept of quasi-Newton methods as a solution. This indicates that one can effectively utilize simplified Hessian calculations in optimization tasks.

PREREQUISITES
  • Understanding of optimization theory
  • Familiarity with gradient descent algorithms
  • Knowledge of Hessian matrix and its role in optimization
  • Basic concepts of quasi-Newton methods
NEXT STEPS
  • Research quasi-Newton methods and their applications
  • Study the role of the Hessian matrix in optimization
  • Learn about different approximation techniques for the Hessian
  • Explore advanced optimization algorithms beyond Newton's method
USEFUL FOR

Students and professionals in optimization theory, data scientists, and anyone interested in advanced mathematical methods for solving optimization problems.

edwardnash
Messages
2
Reaction score
0
Hi there,
I am new to optimization theory. I just went thru solving linear equations using gradient descent. I am looking into Newton's method now which calculates second order derivatives. I was wondering if we really need the hessian matrix for this method to work. Can we just compute the diagonal elements in the hessian and not all elements in the hessian and approximate the Newton's method. I was wondering if anybody familiar with these methods could help me out.

thanks,
ed
 
Physics news on Phys.org
You are right, you can work with approximations to the Hessian matrix. If you stick with your course (or textbook) a bit longer, you will probably soon find out about some of them.

Methods using this idea are sometimes called quasi-Newton methods.
 
Cool. That answers my question. Thanks!
 

Similar threads

Undergrad Looking for a creative or quick method for finding roots in GF(p^n)
  • · Replies 3 ·
Replies
3
Views
2K
Undergrad Gradient descent algorithm and optimizers
  • · Replies 5 ·
Replies
5
Views
2K
Graduate How to use the Newton-Raphson method while keeping the sum constant?
  • · Replies 2 ·
Replies
2
Views
2K
Using Newton's Method to Solve and Using Fsolve in Matlab
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Make Intelligent Initial Guesses for Newton-Raphson Programmatically
  • · Replies 16 ·
Replies
16
Views
5K
Stationary points classification using definiteness of the Lagrangian
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Error bounds for Newton's Method
  • · Replies 13 ·
Replies
13
Views
3K
Why is this method valid to show function is PSD?
  • · Replies 3 ·
Replies
3
Views
2K
Undergrad Number of Binary Operations on a Set with a Special Property
  • · Replies 2 ·
Replies
2
Views
3K
Graduate Optimization with Newton's method
  • · Replies 3 ·
Replies
3
Views
2K