Basic question about numerical hessian

Is it really the vector of second derivatives?In summary, The conversation is about computing the hessian for a 1-D signal and using a transformation to write it as a quadratic form. There is confusion about the vector h being a vector of second derivatives.
  • #1
pamparana
128
0
Hello all,

Suppose I have a simple 1-D signal and I want to compute the hessian. In that case, it should generalise for second derivative for normal scalar functions.



So, I observe the signal as [itex]v = [x_1, x_2, x_3, x_4...][/itex]. Then, numerically the hessian is given as (assuming I am only conputing it at interior points):

[itex]h = [0, (x_1+x_3 -2x_2), (x_2+x_4-2x_3), 0] [/itex].

Now, according to the document for example here (http://planetmath.org/HessianMatrix), I should be able to write this as:

[itex]h = v * H * v'[/itex]. Where H is some transformation. I am trying to figure out what this H should be for my simple 1-D case without any luck.

I would greatly appreciate any help anyone can give me with computing this Hessian operator.

Thanks,
Luca
 
Physics news on Phys.org
  • #2
pamparana said:
Now, according to the document for example here (http://planetmath.org/HessianMatrix), I should be able to write this as:

[itex]h = v * H * v'[/itex]. Where H is some transformation.

The page at PlanetMath says [itex] v H v^T [/itex] is a quadratic form, so it is a scalar valued function. The thing you are calling [itex] h [/itex] looks like a vector.
 

1. What is a numerical Hessian?

A numerical Hessian is a mathematical tool used in optimization and machine learning to calculate the shape or curvature of a function at a specific point. It is a matrix containing second-order partial derivatives of a function with respect to its variables.

2. How is a numerical Hessian calculated?

A numerical Hessian is typically calculated using the finite difference method, which involves taking small steps in each direction from a specific point on a function and calculating the change in gradient. These values are then used to construct the Hessian matrix.

3. What is the importance of the numerical Hessian in optimization?

The numerical Hessian provides information about the shape of a function at a specific point, which is crucial in optimization. It can help determine the direction of steepest descent and whether a point is a local minimum or maximum. It is also used in second-order optimization algorithms to improve convergence speed and accuracy.

4. How can the numerical Hessian be used in machine learning?

In machine learning, the numerical Hessian is used to calculate the curvature of the cost or error function, which can help improve the performance of training algorithms. It is also used in regularization techniques to prevent overfitting and in parameter tuning to speed up convergence.

5. Are there any limitations of the numerical Hessian?

One limitation of the numerical Hessian is that it can be computationally expensive to calculate, especially for high-dimensional functions. It is also not suitable for discontinuous or non-differentiable functions. Additionally, the accuracy of the Hessian can be affected by the step size used in the finite difference method.

Similar threads

Replies
3
Views
733
  • Introductory Physics Homework Help
Replies
4
Views
739
Replies
1
Views
4K
Replies
1
Views
2K
  • Programming and Computer Science
Replies
3
Views
854
  • Introductory Physics Homework Help
Replies
1
Views
753
  • Linear and Abstract Algebra
Replies
9
Views
2K
  • Calculus and Beyond Homework Help
Replies
5
Views
2K
  • Linear and Abstract Algebra
Replies
11
Views
956
  • High Energy, Nuclear, Particle Physics
Replies
6
Views
1K
Back
Top