Non linear curve fitting

1. Dec 20, 2009

a.mlw.walker

so i have the equation attached which i want to solve for a,b and Co.

Tk is a time, and k is a whole number above 0. Using an iterative non linear method i need to solve for a,b and Co. I have heard of the Levenberg-Marquardt method, but it is quite complicated, and I wondered whether there was another way.

The reason for this is, I will be implementing it into a C program to solve it for me, and it will be hard to do this with Leveneberg i think. (Unless someone is willing to share some Leveneberg-Marquardt source code???)

Is there another way that is friendlier to computer programs?

Thanks

Alex

Attached Files:

• equation.bmp
File size:
104.4 KB
Views:
247
2. Dec 20, 2009

Topher925

No matter which non-linear least squares method you use, its going to be complicated, especially with an equation like that where you will have to solve 3 coefficients. Since you are solving three coefficients you will have to perform three dimensional optimization (extrema) as well.

I would look into using software such as Matlab that already has functions like this written for you. It will save you a lot of time. The curve-fitting toolbox has special functions created for solving problems just like this.

3. Dec 20, 2009

whome9

Yes, otherwise the solution search may not converge, . . .ever.

4. Dec 21, 2009

a.mlw.walker

OK, so I have boudght Numerical Recipes in C, which provides the functions to use the levenberg marquardt method but I cant quite work out how to apply it to my function. Has anyone used it, or does anyone have a little zip file using it so that i can see how to integrate my function to minimize into it?

thanks

5. Dec 21, 2009

Topher925

How far have you gotten? What differential equation are you using to min/max-imize your error function?

6. Dec 21, 2009

a.mlw.walker

Not far, i have read alot about how it works if you planned on doing it by hand, but havent actually attempted to do it, because I know that I will need to get a computer to do it for me as it would take far to long by hand, so I was just trying to learn how to use the functions that have already been written. Its a sub topic within a project I am doing, and didnt really want to spend too much time on it, as it is not directly what I am researching, apart from the fact that I need to curve fit a non linear function...

7. Dec 21, 2009

Cyrus

What exactly are you trying to curve fit, and why is this method necessary? I curve fit nonlinear data and I have not used anything like what you have in the OP.

8. Dec 21, 2009

chroot

Staff Emeritus
Why are you attempting to write the curve-fitting software from scratch? Are you required to do so?

I recommend Eureqa.

- Warren

9. Dec 21, 2009

Cyrus

Interesting program, but can definitely lead to some serious pitfalls if one is not careful in how its used. I am always weary of 'plug-n-chug' algorithms because in flight dynamics this can get you into trouble quickly since models have to be physically meaningful. This program can give you a 99.99999% R^2 value fit with a totally nonsense model. My only point: user beware.

10. Dec 22, 2009

Cyrus

I was going to say why don't you just expand out the sigma series and least squares it, but I now see you have a nonlinear form of the coefficients, which means you need a nonlinear estimation method. This is going to get nasty quickly!

You could use a modified Newton-Raphson method, but this ant going to be pretty! (I feel your pain now)

11. Dec 22, 2009

Topher925

You also may want to look into using a Fourier transform since the function is periodic.

Pardon my ignorance, but how would you apply the Newton-Rhapson method to this problem?

12. Dec 22, 2009

hotvette

Based on your post a few months ago:

The function you post appears to be of the form S = sum[yi - f(xi,a,b,C0)]2, which means you have a least squares situation and are really trying to minimize S. Is that correct? Presummably, you have n data points (xi, yi) where n > 2. Correct?

If all of this is true, then the biggest task in solving the problem is deriving expressions for the partial derivatives of f with respect to a, b, and C0. You'll need that for using any canned program (in a pinch, numerical differentiation can be used instead).

By the way, there is a good description of Levenberg-Marquardt here:

http://en.wikipedia.org/wiki/Levenberg-Marquardt_algorithm

And, you can get Levenberg-Marquardt routines in many languages (including C++) here:

http://www.alglib.net/optimization/levenbergmarquardt.php

Lastly, Levenberg-Marquardt is only needed if the problem has poor conditioning or is numerically unstable. It could very well be that standard Newton-Gauss method will work just fine.

Last edited: Dec 22, 2009
13. Dec 22, 2009

Cyrus

It's not a least squares problem, because its a nonlinear form of the coefficients.

14. Dec 22, 2009

hotvette

15. Dec 22, 2009

Cyrus

A least squares state estimator comes from an assumption of a linear model in the parameter estimates, with a white noise process, i.e.

$$z=X\theta+\nu$$​

According to your wiki link, it appears to be doing exactly as I said:

You can see the book: "System identification for aircraft", p. 102 for more information on how the least-squares method is linear. What you are describing is a maximum likelihood method.

Last edited: Dec 22, 2009
16. Dec 24, 2009

hotvette

Questions for a.mlw.walker:

1. How many values do you have for tk? Pls post them.

2. Is the value of S known?

17. Jan 15, 2010

Cyrus

18. Jan 16, 2010

a.mlw.walker

Hi sorry, got behind on this thread exams and all that. Its something I was reading about, and thought I would try and solve, Its one of those charity boxes, where you drop a coin in and it spirals around and around and around until it drops into the bottom...

To start with for the first couple of revolutions you can assume the spiral is very small (also the slope at the start is small) Therefore the original equation is the equation of the coin moving around the track as though the radius is not changing (or the slope angle). The Tk values are the times for consectutive 'laps'. After a couple of spins, there is then another equation for when the coin is spiralling, but I wasnt going to move onto that, this is just me broadening my understanding of non linear curve fitting. Cyrus mentioned that he/she has never used such a method for non linear curve fitting, so what method would you recommend I solve this with

In answer to another post (which I cannot see now because cos on page 2) I am writing it from scratch because I want to write a C program to solve it, if i told it the values for consectutive Tk's, but if someone (cyrus) knows a better way for me to solve this, then there is probably opensource C code for non linear curve fitting. There is some for the Levenburg-Marquardt method, but trying to comprehend how to use such complicated code that someone else has written is beyond me, it would have been easier to write it myself.

Thanks guys. Non-linear curve fitting is something this Mechanical engineering degree seems to avoid...

19. Jan 16, 2010

hotvette

Can you post the doc or article you were reading?

20. Jan 17, 2010

a.mlw.walker

Unfortunately no. I took the governing equations out, and planned to learn how to use them at a later date (thats now).

21. Aug 27, 2010

a.mlw.walker

Oh My Physics forums didnt keep emailing me telling me people were replying.

I could have as many as say 50 values for tk, the values are just recorded timings of something happening, the value of s, from my memory of doing least squares fitting is set to 0 isnt it to make the error minimum???

does anyone have any C sourcecode to curve fit a set of data to an equation?

22. Aug 27, 2010

abc007

give us the points and I may able to interpolate it with different method.

23. Aug 27, 2010

a.mlw.walker

I currently dont have any data, this is all going onto a microchip which will then be part of a circuit that will be left for days to collect the readings, and curve fit it on the fly. the rest of the code is done, its just this curve fitting thats the problem, sorry not to be able to provide much, i just know i need to curve fit the data to that original equation i posted.

24. Aug 27, 2010

hotvette

In this thread I posted a link to source codes last December:

Though I haven't used them, my guess is all you have to do is supply a routine to evaluate the Jacobian matrix (i.e. matrix of partial derivatives of S wrt the unknowns). I agree that using canned routines can be troublesome if you really want to know what's going on. Non-linear curve fitting (in this case least squares becuase you have far more data points than unknowns) isn't all that difficult conceptually. The thing is, you need to understand linear curve fitting before non-linear can make any sense.

Re non-linear curve fitting in ME, I learned it in grad school (ME) in a numerical methods course.

Last edited by a moderator: May 4, 2017
25. Aug 27, 2010

a.mlw.walker

i need levenberg marquardt in C. There are a couple around like lmfit, but I am not sure how to use it, if someone can have a look, and help me work out what i need to supply then i may be able to implement it. (attached)

File size:
23.2 KB
Views:
77