Question about Newton's method for solving a function

edwardnash
Messages
2
Reaction score
0
Hi there,
I am new to optimization theory. I just went thru solving linear equations using gradient descent. I am looking into Newton's method now which calculates second order derivatives. I was wondering if we really need the hessian matrix for this method to work. Can we just compute the diagonal elements in the hessian and not all elements in the hessian and approximate the Newton's method. I was wondering if anybody familiar with these methods could help me out.

thanks,
ed
 
Physics news on Phys.org
You are right, you can work with approximations to the Hessian matrix. If you stick with your course (or textbook) a bit longer, you will probably soon find out about some of them.

Methods using this idea are sometimes called quasi-Newton methods.
 
Cool. That answers my question. Thanks!
 

Thread 'Trouble understanding an online solution to an exercise in Dummit & Foote'

##\textbf{Exercise 10}:## I came across the following solution online: Questions: 1. When the author states in "that ring (not sure if he is referring to ##R## or ##R/\mathfrak{p}##, but I am guessing the later) ##x_n x_{n+1}=0## for all odd $n$ and ##x_{n+1}## is invertible, so that ##x_n=0##" 2. How does ##x_nx_{n+1}=0## implies that ##x_{n+1}## is invertible and ##x_n=0##. I mean if the quotient ring ##R/\mathfrak{p}## is an integral domain, and ##x_{n+1}## is invertible then...
View full post »

Thread 'Questions about non existence of GCDs vs (coimages, cokernels)'

The following are taken from the two sources, 1) from this online page and the book An Introduction to Module Theory by: Ibrahim Assem, Flavio U. Coelho. In the Abelian Categories chapter in the module theory text on page 157, right after presenting IV.2.21 Definition, the authors states "Image and coimage may or may not exist, but if they do, then they are unique up to isomorphism (because so are kernels and cokernels). Also in the reference url page above, the authors present two...
View full post »

Thread 'Decomposition into irreps of compact Lie group'

When decomposing a representation ##\rho## of a finite group ##G## into irreducible representations, we can find the number of times the representation contains a particular irrep ##\rho_0## through the character inner product $$ \langle \chi, \chi_0\rangle = \frac{1}{|G|} \sum_{g\in G} \chi(g) \chi_0(g)^*$$ where ##\chi## and ##\chi_0## are the characters of ##\rho## and ##\rho_0##, respectively. Since all group elements in the same conjugacy class have the same characters, this may be...
View full post »

Similar threads

I Looking for a creative or quick method for finding roots in GF(p^n)
Replies
3
Views
1K
I Gradient descent algorithm and optimizers
Replies
5
Views
2K
A How to use the Newton-Raphson method while keeping the sum constant?
Replies
2
Views
2K
Stationary points classification using definiteness of the Lagrangian
Replies
4
Views
2K
Using Newton's Method to Solve and Using Fsolve in Matlab
Replies
2
Views
2K
I Make Intelligent Initial Guesses for Newton-Raphson Programmatically
Replies
16
Views
3K
Why is this method valid to show function is PSD?
Replies
3
Views
1K
I Error bounds for Newton's Method
Replies
13
Views
2K
B What unique contributions to math did linear algebra make?
Replies
19
Views
3K
Optimization with Newton's method
Replies
3
Views
2K

Hot Threads

Recent Insights

Back
Top