Least squares proof (1 Viewer)

Users Who Are Viewing This Thread (Users: 0, Guests: 1)

760
13
1. The problem statement, all variables and given/known data

In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

[tex]A^T A x^*= A^T b [/tex]

Prove that there's always a solution to this equation.

2. Relevant equations
-

3. The attempt at a solution
I distinct 2 situations [tex]A^T A [/tex] is invertible and it isn't invertible. If it's invertible then there's no problem [tex]x^*= (A^T A)^{-1} A^T b [/tex]

But how I prove that it works in the non-invertible case?
 
1. The problem statement, all variables and given/known data

In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

[tex]A^T A x^*= A^T b [/tex]

Prove that there's always a solution to this equation.

2. Relevant equations


3. The attempt at a solution
I distinct 2 situations [tex]A^T A [/tex] is invertible and it isn't invertible. If it's invertible then there's no problem [tex]x^*= (A^T A)^{-1} A^T b [/tex]

But how I prove that it works in the non-invertible case?
If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.
 
760
13
If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.
Don't you mean the columns of [tex]A^T b[/tex] are in the span of the columns of [tex]A^T A[/tex]? If so I don't understand how to prove such a thing.
 
760
13
I've thought about it and I seriously don't know how to prove that the A^Tb is in the column space of A^TA. Can someone help me?
 

The Physics Forums Way

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving
Top