Example of Levenberg-Marquardt-Method

[Sitewinder]

Hi,

I've to cope a little problem concerning general nonlinear regression. To cut a long story short: I've got a very complicated function y=f(a,x), and a lot of measured values for x and y. The goal is to fit the free parameter a.

Therefor, I wrote a simple c++ program that calculates the sums of the quadratic differences between f(a,x) and the measured y-values, using a lot of values in a large interval for a:

$$\sum_i{(f(a_j,x_i)-y_i)^2}$$

The value a_j which causes the smallest sum, is taken as the fitted parameter a.

Though it works, this method is extremely inefficient due to the high number of measured values, the function's complexity and the large interval needed for a.

What I'm searching for is an introduction or an explanation for a more efficient least-square-method, for example the Levenberg-Marquardt-Method. Unfortunately I didn't find anything useful in the net

I hope that someone can help me.
Thanks in advance
Site

 

[Dr Transport]

do a google search on levenberg-marquardt, i found approximately 200 sites for it in about 5 seconds. also look on the numerical recipies website, www.nr.com, they have a decent explanation.

 

[M98Ranger]

Point User

The Levenberg-Marquardt method is a commonly used algorithm for solving nonlinear least squares problems. It is an iterative method that combines the Gauss-Newton method and the steepest descent method to find the minimum of a sum of squares function. The main idea behind this method is to adjust the step size of the steepest descent method by adding a damping factor that takes into account the local curvature of the function. This allows for a more efficient and stable convergence towards the minimum.

To use the Levenberg-Marquardt method for your problem, you would need to modify your existing program to incorporate the new algorithm. This would involve calculating the Jacobian matrix, which is a matrix of partial derivatives of the function with respect to the parameters, and using it to update the parameter vector in each iteration.

There are many resources available online that provide explanations and implementations of the Levenberg-Marquardt method. One useful resource is the book "Numerical Recipes: The Art of Scientific Computing" by Press et al., which provides a detailed explanation of the method and its implementation in various programming languages.

I hope this helps and good luck with your problem!