Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Uncertainty of linear and non linear least-squares fit

  1. Jan 24, 2010 #1
    1) Linear fit

    I am running an experiment to find the linear relation between displacement (X) and voltage reading (V) of the measurement, when fitting a linear line to my measured data, such that:

    V= m*X + b;

    Based on the following links http://en.wikipedia.org/wiki/Regression_analysis (under Linear regression)


    http://en.wikipedia.org/wiki/R-squared ( under Definitions)

    I can find the The standard errors of 'm' and 'b' parameters, Sm and Sb, respectively. and R-squared

    So the Uncertainty of my measurement is :

    m -/+ Sm


    b -/+ Sb

    But I would like to use the fit in the following way :


    How can I relate the fit error to this form of the equation ?

    2) Non-Linear fit

    In the second experiment I have the following cubic fit:

    X= a3 * (V ^3) + a2 * (V ^2) + a1 * (V) + a0

    How can I find the uncertainty of a3, a2, a1, and a0 ? because I could not find any reference for doing so.

    Can I use the same formula I used in the linear case to find the R-squared here for the non-linear fit?
  2. jcsd
  3. Jan 24, 2010 #2


    User Avatar
    Science Advisor
    Homework Helper

    1) Why don't you estimate x = a0 + a1 V, where a0 = -b/m and a1 = 1/m?
    2) Using matrix notation, X = UA, where U = [V^3 V^2 V 1] and A = [a3 a2 a1 a0]'. Define Z = ((U'U)^-1)U'. The least squares estimator is A* = ZX, a 4x1 vector. Var(A*) = Z' Var(X) Z = (U'U)^-1 Var(X), which is a 4x4 matrix. The diagonal terms are variances; off-diagonal terms are covariances.
    Last edited: Jan 24, 2010
  4. Jan 24, 2010 #3

    I disagree with that:
    if I fit a line as
    V=m1 * X + b1


    X= m2* V + b2;


    m2 IS NOT equal to 1/m1
    and b2 is NOT equal to -b1/m2

    you can run a simple Matlab code and see this.

    Can you please re-write the equation to find the Var of A ( Var[A] ) I was not able to understand it.
    is it
    Var(A*) = Z' Var(X) Z ?
    What does MSE means?
    is (Z'Z)^-1 part of the equation?

    Thank you
    Last edited: Jan 24, 2010
  5. Jan 24, 2010 #4


    User Avatar
    Science Advisor
    Homework Helper

    You are correct; m2 is not equal to 1/m1 and and b2 is not equal to -b1/m2. [However, the standardized coefficients m1/s.e.(m1) and m2/s.e.(m2) are equal, where "s.e." is the standard error of the coefficient. This topic came up before in this forum, see https://www.physicsforums.com/showthread.php?t=316155.] All of this is tangential to your main question, which is, "how do you incorporate measurement error of X into the overall error of the regression?" Sorry to send you in the wrong direction there.

    Var(A*) = Var(ZX) = Z Var(X) Z' is the matrix equivalent of the formula Var(aX) = Var(X)a^2. When you make the substitution Z = ((U'U)^-1)U', Var(A*) simplifies to (U'U)^-1 Var(X). Note that as long as X is the dependent variable, it can be separated into a nonrandom part (UA) and a random part (the error, or the residual); the complete model is X = UA + error. Since UA is nonrandom, Var(X) = Var(error) = MSE.
  6. Jan 25, 2010 #5
    Once again I would like to thank you for your valuable help.

    I am using what you mentioned in your previous posts to solve the following system of equations for a very simple linear case. X= a1*V+a0

    X = UA
    U = [V 1] and A = [a1 a0]'.
    Defining Z = ((U'U)^-1)U'.
    and solving the least squares estimator A = ZX.
    I am comparing my results with what I get from 'polyfit' in Matlab, obviously I get the same result.

    The problem is when I solve the following equation to find the uncertainty of a0 and a1

    Var(A*) = Z' Var(X) Z = (U'U)^-1 Var(X)

    The numbers here are NOT similar what I get from finding the uncertainty using the three equations I mentioned in my first post.

    In addition, Can I use the same formula I used in the linear case ( X=a*V+b ) to find the R-squared for the non-linear fit ( X= a3 * (V ^3) + a2 * (V ^2) + a1 * (V) + a0 )

    Thank you
  7. Jan 26, 2010 #6


    User Avatar
    Science Advisor
    Homework Helper

    Have you tried defining var X = SSE/(N-2)? Do you find a different result using the matrix formula?

    What you are referring as the nonlinear fit is in fact linear, so yes, the same formula applies. The term nonlinear is reserved for nonlinearity in parameters, for example X = ab + V^b.
  8. Jan 26, 2010 #7
    As I already mentioned, I am trying to run a very simple case of linear line (X= a1*V+a0) becuase I already know how to find the uncirtinty in a0 and a1 for this case ( based on the three equations that I have in my first post where var X = SSE/(N-2) )


    Var(A*) = Z' Var(X) Z = (U'U)^-1 Var(X)
    is giving me a different result for the uncertainty in a0 and a1 .

    I can post the MATLAB code if this could be on any help.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook