Formulating Curves: Techniques for Nonlinear Data Fitting

[S&S]

This is just a general asking. It doesn't related with my current study. If someone can point out for me, just give me more knowleges.

In my class I learned the least square technique which allow us draw a best fitted line between observed depent and indepent varivables. Then I think if my observation is a curve that can be much more complex than a straight line, how can I formulate it. I think this thing should be very important in scientific research and real life analysis.

Would you guys tell me how to deal with curves? Or you also can figure me some key words which I can do some search and read by myself. I think these things should not be hard. If I can aways know something that my classmates don't know, I will be definitely better than them, right?

Cheers

 

[Physics Monkey]

I can demonstrate by example. Suppose you had a data set that you thought could be well fitted by a parabola. Your data set would look something like $$\{x_n,y_n\}$$ with n running up to N, and your fit function looks something like $$y = a + bx + cx^2$$. So the question then is, how to determine a, b, and c using least squares? The most direct way to do this is to use matrices. Let's take N = 4 just to be definite. Define a matrix $$X$$,
$$X = \left[\begin{array}{ccc} 1 & x_1 & x^2_1 \\ 1 & x_2 & x^2_2 \\ 1 & x_3 & x^2_3 \\ 1 & x_4 & x^2_4 \end{array} \right],$$
a vector $$Y$$,
$$X = \left[\begin{array}{c} y_1 \\ y_2 \\ y_3 \\ y_4 \end{array} \right]$$
and another vector $$C$$,
$$C = \left[\begin{array}{c} a \\ c \\ c \end{array} \right].$$
What are each of these matrices, X is the matrix of x values, Y is the vector of y values, and C is the matrix of unknown coeffecients. You can now convince yourself that the equation you would like to solve is
$$X C = Y,$$
but in general this equation has no exact solution, so instead you find a least squares solution. The least squares solution of this matrix equation gives a vector C that contains the numerical values of your coeffecients that give a least squares fit based on the parabola. Hopefully you can see how to generalize this procedure for larger N or more complicated fitting functions. Of course, this method is only the beginning of a more comprehensive treatment.

 

[Integral]

The least squares method is not limited to linear approximations, it is quite general and can be applied to polynomials in general. The trouble is that given arbitrary order, there is some polynomial that is an EXACT fit to your data.
Because of this ability of higher order polynomials to mold themselves to a data set they are not very useful for THIS purpose. If your data is nonlinear then you need to be able to guess, or know what functional relationship to expect in your data. There is no way that the data alone can revel its true mathematical nature. How do you learn the functional nature of your data? By understanding the system which generated it is the first step. You might say that this is what being a scientist is all about.
We have a https://www.physicsforums.com/showthread.php?p=806673#post806673" for this.

 

[S&S]

thank you guys, this's what I'm talking about.

However, things getting more interesting. I used to use a graph tool which based on MS Excel and it's also programed by some higher stage research students in my school. This graph tool has two modes. First is the linear fit graphing which is drawing a best fit line for input data. The second mode is drawing a smooth curve from data. That thing never asked me to specify a polynomial function.

According to your method, I think, the algorithm they programed the graph tool must can draw the graph without knowing anything from the function which determines the curve. How could that algorithm looks like, any idea?

 

[Physics Monkey]

You're welcome S&S,

Your program was probably doing what is usually called nonparametric fitting which is essentially interpolation. Probably the program finds a smoothing spline by guessing some reasonable value of a smoothness parameter based on your data (this sort of makes this method parametric, but the thing is still using splines rather than a single global function). Probably you could input this number yourself if you knew how. Alternatively, the program may just spit out a cubic spline which is simply a curve made of third order polynomials connecting each data point.

I would like to emphasize what Integral mentioned which is that data fitting is an art, and it requires you to know a lot about the system you are studying. Just asking the computer to draw a smooth curve through your data doesn't really tell you much, although it may look pretty.