Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Non linear curve fitting

  1. Dec 20, 2009 #1
    so i have the equation attached which i want to solve for a,b and Co.

    Tk is a time, and k is a whole number above 0. Using an iterative non linear method i need to solve for a,b and Co. I have heard of the Levenberg-Marquardt method, but it is quite complicated, and I wondered whether there was another way.

    The reason for this is, I will be implementing it into a C program to solve it for me, and it will be hard to do this with Leveneberg i think. (Unless someone is willing to share some Leveneberg-Marquardt source code???)

    Is there another way that is friendlier to computer programs?



    Attached Files:

  2. jcsd
  3. Dec 20, 2009 #2
    No matter which non-linear least squares method you use, its going to be complicated, especially with an equation like that where you will have to solve 3 coefficients. Since you are solving three coefficients you will have to perform three dimensional optimization (extrema) as well.

    I would look into using software such as Matlab that already has functions like this written for you. It will save you a lot of time. The curve-fitting toolbox has special functions created for solving problems just like this.
  4. Dec 20, 2009 #3
    Yes, otherwise the solution search may not converge, . . .ever.
  5. Dec 21, 2009 #4
    OK, so I have boudght Numerical Recipes in C, which provides the functions to use the levenberg marquardt method but I cant quite work out how to apply it to my function. Has anyone used it, or does anyone have a little zip file using it so that i can see how to integrate my function to minimize into it?

  6. Dec 21, 2009 #5
    How far have you gotten? What differential equation are you using to min/max-imize your error function?
  7. Dec 21, 2009 #6
    Not far, i have read alot about how it works if you planned on doing it by hand, but havent actually attempted to do it, because I know that I will need to get a computer to do it for me as it would take far to long by hand, so I was just trying to learn how to use the functions that have already been written. Its a sub topic within a project I am doing, and didnt really want to spend too much time on it, as it is not directly what I am researching, apart from the fact that I need to curve fit a non linear function...
  8. Dec 21, 2009 #7
    What exactly are you trying to curve fit, and why is this method necessary? I curve fit nonlinear data and I have not used anything like what you have in the OP.
  9. Dec 21, 2009 #8


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Why are you attempting to write the curve-fitting software from scratch? Are you required to do so?

    I recommend Eureqa.

    - Warren
  10. Dec 21, 2009 #9
    Interesting program, but can definitely lead to some serious pitfalls if one is not careful in how its used. I am always weary of 'plug-n-chug' algorithms because in flight dynamics this can get you into trouble quickly since models have to be physically meaningful. This program can give you a 99.99999% R^2 value fit with a totally nonsense model. My only point: user beware.
  11. Dec 22, 2009 #10
    I was going to say why don't you just expand out the sigma series and least squares it, but I now see you have a nonlinear form of the coefficients, which means you need a nonlinear estimation method. This is going to get nasty quickly!

    You could use a modified Newton-Raphson method, but this ant going to be pretty! (I feel your pain now)
  12. Dec 22, 2009 #11
    You also may want to look into using a Fourier transform since the function is periodic.

    Pardon my ignorance, but how would you apply the Newton-Rhapson method to this problem?
  13. Dec 22, 2009 #12


    User Avatar
    Homework Helper

    Based on your post a few months ago:


    The function you post appears to be of the form S = sum[yi - f(xi,a,b,C0)]2, which means you have a least squares situation and are really trying to minimize S. Is that correct? Presummably, you have n data points (xi, yi) where n > 2. Correct?

    If all of this is true, then the biggest task in solving the problem is deriving expressions for the partial derivatives of f with respect to a, b, and C0. You'll need that for using any canned program (in a pinch, numerical differentiation can be used instead).

    By the way, there is a good description of Levenberg-Marquardt here:


    And, you can get Levenberg-Marquardt routines in many languages (including C++) here:


    Lastly, Levenberg-Marquardt is only needed if the problem has poor conditioning or is numerically unstable. It could very well be that standard Newton-Gauss method will work just fine.
    Last edited: Dec 22, 2009
  14. Dec 22, 2009 #13
    It's not a least squares problem, because its a nonlinear form of the coefficients.
  15. Dec 22, 2009 #14


    User Avatar
    Homework Helper

  16. Dec 22, 2009 #15
    A least squares state estimator comes from an assumption of a linear model in the parameter estimates, with a white noise process, i.e.


    According to your wiki link, it appears to be doing exactly as I said:

    You can see the book: "System identification for aircraft", p. 102 for more information on how the least-squares method is linear. What you are describing is a maximum likelihood method.
    Last edited: Dec 22, 2009
  17. Dec 24, 2009 #16


    User Avatar
    Homework Helper

    Questions for a.mlw.walker:

    1. How many values do you have for tk? Pls post them.

    2. Is the value of S known?
  18. Jan 15, 2010 #17
  19. Jan 16, 2010 #18
    Hi sorry, got behind on this thread exams and all that. Its something I was reading about, and thought I would try and solve, Its one of those charity boxes, where you drop a coin in and it spirals around and around and around until it drops into the bottom...

    To start with for the first couple of revolutions you can assume the spiral is very small (also the slope at the start is small) Therefore the original equation is the equation of the coin moving around the track as though the radius is not changing (or the slope angle). The Tk values are the times for consectutive 'laps'. After a couple of spins, there is then another equation for when the coin is spiralling, but I wasnt going to move onto that, this is just me broadening my understanding of non linear curve fitting. Cyrus mentioned that he/she has never used such a method for non linear curve fitting, so what method would you recommend I solve this with

    In answer to another post (which I cannot see now because cos on page 2) I am writing it from scratch because I want to write a C program to solve it, if i told it the values for consectutive Tk's, but if someone (cyrus) knows a better way for me to solve this, then there is probably opensource C code for non linear curve fitting. There is some for the Levenburg-Marquardt method, but trying to comprehend how to use such complicated code that someone else has written is beyond me, it would have been easier to write it myself.

    Thanks guys. Non-linear curve fitting is something this Mechanical engineering degree seems to avoid...
  20. Jan 16, 2010 #19


    User Avatar
    Homework Helper

    Can you post the doc or article you were reading?
  21. Jan 17, 2010 #20
    Unfortunately no. I took the governing equations out, and planned to learn how to use them at a later date (thats now).
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook