1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Least-squares estimation

  1. Jan 21, 2015 #1
    1. The problem statement, all variables and given/known data
    Five measurements of fuel consumption of a random Fiat car gives us data:

    v (km/h) l/100 km
    70 --> 3.9+-0.1
    90 --> 4.8+-0.1
    120 --> 6.0+-0.1
    150 --> 8.1+-0.1
    160 --> 9.0+-0.1

    We would like to check if for velocities greater than 70 km/h the fuel consumption obeys ##\Phi =\Phi _0 +\alpha v^2##. Find ##\Phi _0## and ##\alpha ## using least-squares estimation.

    2. Relevant equations

    3. The attempt at a solution
    Ok, I tried to do something but I assume there exist an easier way than mine...

    Firstly, the law that fuel consumption should obey is ##\Phi =\Phi _0 +\beta v +\alpha v^2##. So $$
    \Phi _0\\
    \Phi _0\\
    \end{bmatrix}=(H^TH)^{-1}H^Tz$$ Where IF I am not mistaken $$H=
    1 & 0& 70^2\\
    1& 0 & 90^2\\
    1& 0 & 120^2\\
    1& 0 & 150^2\\
    1& 0 & 160^2
    \end{bmatrix}$$ meaning $$H^TH=
    5 &0 & 75500\\
    0 & 0&0 \\
    75500& 0&14.6\cdot 10^8
    \end{bmatrix}$$ Now here the problems start. Am I really supposed to find the inverse of this? o_O Is there really no better/faster solution to this problem?
  2. jcsd
  3. Jan 21, 2015 #2


    User Avatar
    Staff Emeritus
    Science Advisor
    Homework Helper

    You won't be able to find an inverse to this matrix because one entire row and one entire column is nothing but zeroes. Obviously, if you calculate the determinant of the original matrix, it too will equal zero; therefore, the original matrix is not invertible.

    Also, this is not technically a physics problem, let alone an advanced physics problem, so I'm moving this thread to the Pre-Calc HW forum.
  4. Jan 21, 2015 #3
    Ok... than what should I do?

    Yes, I apologize for misplacing the thread and thank you for moving it!
  5. Jan 21, 2015 #4


    User Avatar
    Education Advisor
    Gold Member

    Mmmmm! Generalized least squares. Powerful with the data analysis is this. Everybody who ever does data analysis should learn this one. It should not be the last method you learn, of course. Not by a wide margin. But it will get you through many a task.

    The problem statement does not have a term linear in speed. Where did your linear term come from? And why did you set it to zero?

    Whenever you do generalized least-squared fit method, your H matrix should never have more than one column that is all one value. Having two columns that were both constant essentially means you are pulling out two constant (phi_0) terms. That does not make sense. Also, you should not have a zero column. That would mean you were pulling out the portion of the data that was some parameter times zero. Again, it does not make sense.

    By the way, once you have got your fitting params as the question suggestions, how will you check that this is a good fit? You will get values for these params from a very wide rang of data. How will you check that the values you have obtained are a good fit? There are several ways to approach that. Presumably your text or your class has told you one or more such methods. For example, if the data were really better fitted by a constant and a linear term, how would you detect that? Or what if it is really a cubic?
  6. Jan 21, 2015 #5

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Either keep all three parameters ##\Phi_0, \alpha, \beta## and use the 3x3 matrix, or else keep only two parameters ##\Phi_0, \alpha## and use the 2x2 matrix obtained by omitting row and column 2 in the one you have above. Then just invert the matrices---or better, still, solve the linear system of equations by straight Gaussian elimination (the method you first learned in high school). Generally, matrix inversion ought to be avoided whenever possible; matrix inversion is just about the worst way to solve equations (due to stability and roudoff error issues).
  7. Jan 22, 2015 #6
    Hmm, ok, I understand the first part, but I don't quite get the part about how to avoid the matrix inversion. Could you be more specific?

    For example in my case the equation is ##\begin{bmatrix}
    \Phi _0\\
    \end{bmatrix}=(H^TH)^{-1}H^Tz## and I can't see how would one solve it without matrix inversion.
  8. Jan 22, 2015 #7
    Aaaa, ok, I found a formula in my notes for that inverse $$
    (H^TH)^{-1}=\frac{1}{n\sum t_i^2-(\sum t_i)^2 }\begin{bmatrix}
    \sum t_i^2 & -\sum t_i\\
    -\sum t_i & n
    \end{bmatrix}$$ and also $$
    \sum t_i\\
    \sum t_iz_i

    Caluclating this (checked with mathematica) leaves me with some rather "weird" results, saying that $$
    \Phi _0\\
    \end{bmatrix}$$ Weird because this means that the fuel consumption is reducing with speed. I find that a bit strange. Or?
  9. Jan 22, 2015 #8

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    I said: just solve the equations.

    Your equations ##(H^T H) (\Phi_0,\alpha)^T = H^T z## are
    [tex] \begin{bmatrix} 5 & 75500\\75500& 1458590000 \end{bmatrix} \begin{bmatrix}\Phi_0 \\\alpha \end{bmatrix} = \begin{bmatrix}31.8 \\557040 \end{bmatrix}[/tex]
    These say
    [tex] 5 \Phi_0 + 75500 \alpha = 31.8\\
    75500 \Phi_0 + 1458590000 \alpha = 557040[/tex]
    These are just 2 linear equations in 2 unknowns, and can be solved by high-school methods.

    BTW: it is always a good idea to scale a problem before setting out trying to solve it. In this case, using ##w = v/10## instead of ##v## produces a much more well-behaved problem. The equation ##\Phi = \Phi_0 + \alpha v^2## can be re-written as ##\Phi = \Phi_0 + \gamma w^2##, where ##\gamma = 100 \alpha##. This produces
    [tex] \begin{bmatrix} 5&755\\755 & 145859 \end{bmatrix} \begin{bmatrix} \Phi_0\\ \gamma \end{bmatrix}
    = \begin{bmatrix} 31.8 \\5570.4 \end{bmatrix} [/tex]
    [tex] 5 \Phi_0 + 755 \gamma = 31.8\\
    755 \Phi_0 + 145869 \gamma = 6670.4[/tex]
    The solution of this system is ##\Phi_0 = 2.71654548879261348 \doteq 2.7165## and ##\gamma = 0.0241288378225654673 \doteq 0.02413##, giving ##\alpha \doteq 0.2413 \times 10^{-3}##.

    BTW: rounding off the matrix entry ##1458590000## to ##14.6 \times 10^8##--as you have done--produces a fair amount of inaccuracy in the solution, perhaps even an unacceptable level of inaccuracy.
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted

Similar Discussions: Least-squares estimation
  1. Least Squares Regression (Replies: 10)

  2. Least Squares (Replies: 1)