Question about Newton's method for solving a function

In summary, the conversation discusses the use of Newton's method in optimization theory. The speaker is new to this topic and has just learned about solving linear equations using gradient descent. They are now exploring Newton's method, which involves calculating second order derivatives. The question is raised about the necessity of using the full Hessian matrix or if only diagonal elements can be used. The response states that approximations can be used and these methods are known as quasi-Newton methods. The speaker expresses gratitude for the answer.
  • #1
edwardnash
3
0
Hi there,
I am new to optimization theory. I just went thru solving linear equations using gradient descent. I am looking into Newton's method now which calculates second order derivatives. I was wondering if we really need the hessian matrix for this method to work. Can we just compute the diagonal elements in the hessian and not all elements in the hessian and approximate the Newton's method. I was wondering if anybody familiar with these methods could help me out.

thanks,
ed
 
Physics news on Phys.org
  • #2
You are right, you can work with approximations to the Hessian matrix. If you stick with your course (or textbook) a bit longer, you will probably soon find out about some of them.

Methods using this idea are sometimes called quasi-Newton methods.
 
  • #3
Cool. That answers my question. Thanks!
 

1. What is Newton's method for solving a function?

Newton's method is an iterative algorithm used to find the roots of a function. It involves making an initial guess for the root, then using the function's derivative to find a better approximation of the root. This process is repeated until the desired level of accuracy is achieved.

2. How does Newton's method work?

Newton's method works by using the slope of the tangent line at a given point on the function to estimate the location of the root. This estimate is then refined by finding the intersection of the tangent line with the x-axis, which gives a new guess for the root. This process is repeated until the desired level of accuracy is reached.

3. What are the advantages of using Newton's method?

One advantage of Newton's method is that it is typically faster than other root-finding methods. Additionally, it can handle complex functions with multiple roots. It also provides a high level of accuracy when the initial guess is close to the actual root.

4. What are the limitations of Newton's method?

One limitation of Newton's method is that it may fail to converge or give inaccurate results if the initial guess is not close enough to the actual root. It also requires knowledge of the function's derivative, which may not always be available or easy to calculate.

5. Can Newton's method be used to solve any type of function?

No, Newton's method works best for continuous and differentiable functions. It may not work for functions with discontinuities, sharp turns, or multiple roots that are too close together. In these cases, other root-finding methods may be more suitable.

Similar threads

I Looking for a creative or quick method for finding roots in GF(p^n)
    • Linear and Abstract Algebra
Replies
3
Views
748
I Gradient descent algorithm and optimizers
    • General Math
Replies
5
Views
841
Using Newton's Method to Solve and Using Fsolve in Matlab
    • Engineering and Comp Sci Homework Help
Replies
2
Views
1K
I Make Intelligent Initial Guesses for Newton-Raphson Programmatically
    • General Math
Replies
16
Views
1K
Stationary points classification using definiteness of the Lagrangian
    • Calculus and Beyond Homework Help
Replies
4
Views
834
Why is this method valid to show function is PSD?
    • Calculus and Beyond Homework Help
Replies
3
Views
811
I Error bounds for Newton's Method
    • Calculus
Replies
13
Views
1K
I What Exactly is Step Size in Gradient Descent Method?
    • Calculus
Replies
3
Views
1K
Optimization with Newton's method
    • Mechanics
Replies
3
Views
1K
Optimization using Newton's method gradient hessian
    • Differential Equations
Replies
1
Views
2K

Hot Threads

Recent Insights

Back
Top