Question about Newton's method for solving a function

[edwardnash]

Hi there,
I am new to optimization theory. I just went thru solving linear equations using gradient descent. I am looking into Newton's method now which calculates second order derivatives. I was wondering if we really need the hessian matrix for this method to work. Can we just compute the diagonal elements in the hessian and not all elements in the hessian and approximate the Newton's method. I was wondering if anybody familiar with these methods could help me out.

thanks,
ed

 

[AlephZero]

You are right, you can work with approximations to the Hessian matrix. If you stick with your course (or textbook) a bit longer, you will probably soon find out about some of them.

Methods using this idea are sometimes called quasi-Newton methods.

 

[edwardnash]

Cool. That answers my question. Thanks!