Non linear curve fitting

[a.mlw.walker]

so i have the equation attached which i want to solve for a,b and Co.

Tk is a time, and k is a whole number above 0. Using an iterative non linear method i need to solve for a,b and Co. I have heard of the Levenberg-Marquardt method, but it is quite complicated, and I wondered whether there was another way.

The reason for this is, I will be implementing it into a C program to solve it for me, and it will be hard to do this with Leveneberg i think. (Unless someone is willing to share some Leveneberg-Marquardt source code?)

Is there another way that is friendlier to computer programs?

Thanks

Alex

 

[Topher925]

No matter which non-linear least squares method you use, its going to be complicated, especially with an equation like that where you will have to solve 3 coefficients. Since you are solving three coefficients you will have to perform three dimensional optimization (extrema) as well.

I would look into using software such as Matlab that already has functions like this written for you. It will save you a lot of time. The curve-fitting toolbox has special functions created for solving problems just like this.

 

[whome9]

I would look into using software such as Matlab that already has functions like this written for you. It will save you a lot of time. The curve-fitting toolbox has special functions created for solving problems just like this.

Yes, otherwise the solution search may not converge, . . .ever.

 

[a.mlw.walker]

OK, so I have boudght Numerical Recipes in C, which provides the functions to use the levenberg marquardt method but I can't quite work out how to apply it to my function. Has anyone used it, or does anyone have a little zip file using it so that i can see how to integrate my function to minimize into it?

thanks

 

[Topher925]

How far have you gotten? What differential equation are you using to min/max-imize your error function?

 

[a.mlw.walker]

Not far, i have read a lot about how it works if you planned on doing it by hand, but haven't actually attempted to do it, because I know that I will need to get a computer to do it for me as it would take far to long by hand, so I was just trying to learn how to use the functions that have already been written. Its a sub topic within a project I am doing, and didnt really want to spend too much time on it, as it is not directly what I am researching, apart from the fact that I need to curve fit a non linear function...

 

[Cyrus]

What exactly are you trying to curve fit, and why is this method necessary? I curve fit nonlinear data and I have not used anything like what you have in the OP.

 

[chroot]

Why are you attempting to write the curve-fitting software from scratch? Are you required to do so?

I recommend http://ccsl.mae.cornell.edu/eureqa.

- Warren

 

[Cyrus]

Why are you attempting to write the curve-fitting software from scratch? Are you required to do so?

I recommend http://ccsl.mae.cornell.edu/eureqa.

- Warren

Interesting program, but can definitely lead to some serious pitfalls if one is not careful in how its used. I am always weary of 'plug-n-chug' algorithms because in flight dynamics this can get you into trouble quickly since models have to be physically meaningful. This program can give you a 99.99999% R^2 value fit with a totally nonsense model. My only point: user beware.

 

[Cyrus]

I was going to say why don't you just expand out the sigma series and least squares it, but I now see you have a nonlinear form of the coefficients, which means you need a nonlinear estimation method. This is going to get nasty quickly!

You could use a modified Newton-Raphson method, but this ant going to be pretty! (I feel your pain now)

 

[Topher925]

You also may want to look into using a Fourier transform since the function is periodic.

Pardon my ignorance, but how would you apply the Newton-Rhapson method to this problem?

 

[hotvette]

so i have the equation attached which i want to solve for a,b and Co

Based on your post a few months ago:

https://www.physicsforums.com/showthread.php?t=338309

The function you post appears to be of the form S = sum[y_i - f(x_i,a,b,C₀)]², which means you have a least squares situation and are really trying to minimize S. Is that correct? Presummably, you have n data points (x_i, y_i) where n > 2. Correct?

If all of this is true, then the biggest task in solving the problem is deriving expressions for the partial derivatives of f with respect to a, b, and C₀. You'll need that for using any canned program (in a pinch, numerical differentiation can be used instead).

By the way, there is a good description of Levenberg-Marquardt here:

http://en.wikipedia.org/wiki/Levenberg-Marquardt_algorithm

And, you can get Levenberg-Marquardt routines in many languages (including C++) here:

http://www.alglib.net/optimization/levenbergmarquardt.php

Lastly, Levenberg-Marquardt is only needed if the problem has poor conditioning or is numerically unstable. It could very well be that standard Newton-Gauss method will work just fine.

 

[Cyrus]

Based on your post a few months ago:

https://www.physicsforums.com/showthread.php?t=338309

The function you post appears to be of the form S = sum[y_i - f(x_i,a,b,C₀)]², which means you have a least squares situation and are really trying to minimize S. Is that correct? Presummably, you have n data points (x_i, y_i) where n > 2. Correct?

If all of this is true, then the biggest task in solving the problem is deriving expressions for the partial derivatives of f with respect to a, b, and C₀. You'll need that for using any canned program (in a pinch, numerical differentiation can be used instead).

By the way, there is a good description of Levenberg-Marquardt here:

http://en.wikipedia.org/wiki/Levenberg-Marquardt_algorithm

And, you can get Levenberg-Marquardt routines in many languages (including C++) here:

http://www.alglib.net/optimization/levenbergmarquardt.php

Lastly, Levenberg-Marquardt is only needed if the problem has poor conditioning or is numerically unstable. It could very well be that standard Newton-Gauss method will work just fine.

It's not a least squares problem, because its a nonlinear form of the coefficients.

 

[hotvette]

It's not a least squares problem, because its a nonlinear form of the coefficients.

Least Sqaures problems can be nonlinear. Here are a couple of references:

http://en.wikipedia.org/wiki/Non-linear_least_squares

http://www.itl.nist.gov/div898/strd/nls/data/boxbod.shtml

 

[Cyrus]

Least Sqaures problems can be nonlinear. Here are a couple of references:

http://en.wikipedia.org/wiki/Non-linear_least_squares

http://www.itl.nist.gov/div898/strd/nls/data/boxbod.shtml

A least squares state estimator comes from an assumption of a linear model in the parameter estimates, with a white noise process, i.e.

$$z=X\theta+\nu$$​

According to your wiki link, it appears to be doing exactly as I said:

The basis of the method is to approximate the model by a linear one and to refine the parameters by successive iterations.

You can see the book: "System identification for aircraft", p. 102 for more information on how the least-squares method is linear. What you are describing is a maximum likelihood method.

 

[hotvette]

Questions for a.mlw.walker:

1. How many values do you have for t_k? Pls post them.

2. Is the value of S known?

 

[Cyrus]

Least Sqaures problems can be nonlinear. Here are a couple of references:

http://en.wikipedia.org/wiki/Non-linear_least_squares

http://www.itl.nist.gov/div898/strd/nls/data/boxbod.shtml

You are absolutely correct. I was caught up in using linear least squares I forgot about the nonlinear aspects to it.

 

[a.mlw.walker]

Hi sorry, got behind on this thread exams and all that. Its something I was reading about, and thought I would try and solve, Its one of those charity boxes, where you drop a coin in and it spirals around and around and around until it drops into the bottom...

To start with for the first couple of revolutions you can assume the spiral is very small (also the slope at the start is small) Therefore the original equation is the equation of the coin moving around the track as though the radius is not changing (or the slope angle). The Tk values are the times for consectutive 'laps'. After a couple of spins, there is then another equation for when the coin is spiralling, but I wasnt going to move onto that, this is just me broadening my understanding of non linear curve fitting. Cyrus mentioned that he/she has never used such a method for non linear curve fitting, so what method would you recommend I solve this with

In answer to another post (which I cannot see now because cos on page 2) I am writing it from scratch because I want to write a C program to solve it, if i told it the values for consectutive Tk's, but if someone (cyrus) knows a better way for me to solve this, then there is probably opensource C code for non linear curve fitting. There is some for the Levenburg-Marquardt method, but trying to comprehend how to use such complicated code that someone else has written is beyond me, it would have been easier to write it myself.

Thanks guys. Non-linear curve fitting is something this Mechanical engineering degree seems to avoid...

 

[hotvette]

Its something I was reading about

Can you post the doc or article you were reading?

 

[a.mlw.walker]

Unfortunately no. I took the governing equations out, and planned to learn how to use them at a later date (thats now).

 

[a.mlw.walker]

Oh My Physics forums didnt keep emailing me telling me people were replying.

I could have as many as say 50 values for tk, the values are just recorded timings of something happening, the value of s, from my memory of doing least squares fitting is set to 0 isn't it to make the error minimum?

does anyone have any C sourcecode to curve fit a set of data to an equation?

 

[abc007]

give us the points and I may able to interpolate it with different method.

 

[a.mlw.walker]

I currently don't have any data, this is all going onto a microchip which will then be part of a circuit that will be left for days to collect the readings, and curve fit it on the fly. the rest of the code is done, its just this curve fitting that's the problem, sorry not to be able to provide much, i just know i need to curve fit the data to that original equation i posted.

 

[hotvette]

does anyone have any C sourcecode to curve fit a set of data to an equation?

In this thread I posted a link to source codes last December:

And, you can get Levenberg-Marquardt routines in many languages (including C++) here:

http://www.alglib.net/optimization/l...gmarquardt.php

Though I haven't used them, my guess is all you have to do is supply a routine to evaluate the Jacobian matrix (i.e. matrix of partial derivatives of S wrt the unknowns). I agree that using canned routines can be troublesome if you really want to know what's going on. Non-linear curve fitting (in this case least squares becuase you have far more data points than unknowns) isn't all that difficult conceptually. The thing is, you need to understand linear curve fitting before non-linear can make any sense.

Re non-linear curve fitting in ME, I learned it in grad school (ME) in a numerical methods course.

 

[a.mlw.walker]

i need levenberg marquardt in C. There are a couple around like lmfit, but I am not sure how to use it, if someone can have a look, and help me work out what i need to supply then i may be able to implement it. (attached)

 

[a.mlw.walker]

Guys Hi.
I know this post is ages old. But I am coming back to the problem now. I decided to wait until I had finished my degree to continue with this project. I have finished now. Hopefully you will also come back to the post 2 years later!

I have timings for tk: 1.994, 1.806, 1.632, 1.493, 1.2
x tk
1 1.2
2 1.493
3 1.632
4 1.806
5 1.994

I have used alglib to do a parameter estimation. It gives me values, but values I don't think are correct.

As far as I know S is not known. I understand what the parameter estimation does, and based on that, I don't think S is required?

Does anyone have the code to do a MATLAB curve fit on this to see what it reckons?
I used the initial guesses of 1,1,1 for each parameter.

I have managed to get lmfit working but i can't get convergence