Least squares proof

  • Thread starter dirk_mec1
  • Start date
  • #1
761
13

Homework Statement



In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

[tex]A^T A x^*= A^T b [/tex]

Prove that there's always a solution to this equation.

Homework Equations


-

The Attempt at a Solution


I distinct 2 situations [tex]A^T A [/tex] is invertible and it isn't invertible. If it's invertible then there's no problem [tex]x^*= (A^T A)^{-1} A^T b [/tex]

But how I prove that it works in the non-invertible case?
 

Answers and Replies

  • #2

Homework Statement



In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

[tex]A^T A x^*= A^T b [/tex]

Prove that there's always a solution to this equation.

Homework Equations




The Attempt at a Solution


I distinct 2 situations [tex]A^T A [/tex] is invertible and it isn't invertible. If it's invertible then there's no problem [tex]x^*= (A^T A)^{-1} A^T b [/tex]

But how I prove that it works in the non-invertible case?

If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.
 
  • #3
761
13
If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.

Don't you mean the columns of [tex]A^T b[/tex] are in the span of the columns of [tex]A^T A[/tex]? If so I don't understand how to prove such a thing.
 
  • #4
761
13
I've thought about it and I seriously don't know how to prove that the A^Tb is in the column space of A^TA. Can someone help me?
 

Related Threads on Least squares proof

  • Last Post
Replies
7
Views
979
  • Last Post
Replies
1
Views
2K
  • Last Post
Replies
8
Views
2K
  • Last Post
Replies
3
Views
1K
  • Last Post
Replies
3
Views
1K
  • Last Post
Replies
0
Views
2K
  • Last Post
Replies
12
Views
2K
  • Last Post
Replies
10
Views
867
  • Last Post
Replies
2
Views
1K
  • Last Post
Replies
6
Views
2K
Top