Least squares proof

[dirk_mec1]

Homework Statement

In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

$$A^T A x^*= A^T b$$

Prove that there's always a solution to this equation.

Homework Equations

-

The Attempt at a Solution

I distinct 2 situations $$A^T A$$ is invertible and it isn't invertible. If it's invertible then there's no problem $$x^*= (A^T A)^{-1} A^T b$$

But how I prove that it works in the non-invertible case?

 

[John Creighto]

Homework Statement

In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

$$A^T A x^*= A^T b$$

Prove that there's always a solution to this equation.

Homework Equations

The Attempt at a Solution

I distinct 2 situations $$A^T A$$ is invertible and it isn't invertible. If it's invertible then there's no problem $$x^*= (A^T A)^{-1} A^T b$$

But how I prove that it works in the non-invertible case?

If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.

 

[dirk_mec1]

If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.

Don't you mean the columns of $$A^T b$$ are in the span of the columns of $$A^T A$$? If so I don't understand how to prove such a thing.

 

[dirk_mec1]

I've thought about it and I seriously don't know how to prove that the A^Tb is in the column space of A^TA. Can someone help me?