1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Least squares proof

  1. Feb 27, 2009 #1
    1. The problem statement, all variables and given/known data

    In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

    [tex]A^T A x^*= A^T b [/tex]

    Prove that there's always a solution to this equation.

    2. Relevant equations

    3. The attempt at a solution
    I distinct 2 situations [tex]A^T A [/tex] is invertible and it isn't invertible. If it's invertible then there's no problem [tex]x^*= (A^T A)^{-1} A^T b [/tex]

    But how I prove that it works in the non-invertible case?
  2. jcsd
  3. Feb 27, 2009 #2
    If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.
  4. Feb 28, 2009 #3
    Don't you mean the columns of [tex]A^T b[/tex] are in the span of the columns of [tex]A^T A[/tex]? If so I don't understand how to prove such a thing.
  5. Mar 1, 2009 #4
    I've thought about it and I seriously don't know how to prove that the A^Tb is in the column space of A^TA. Can someone help me?
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook