# Least squares proof

1. Feb 27, 2009

### dirk_mec1

1. The problem statement, all variables and given/known data

In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

$$A^T A x^*= A^T b$$

Prove that there's always a solution to this equation.

2. Relevant equations
-

3. The attempt at a solution
I distinct 2 situations $$A^T A$$ is invertible and it isn't invertible. If it's invertible then there's no problem $$x^*= (A^T A)^{-1} A^T b$$

But how I prove that it works in the non-invertible case?

2. Feb 27, 2009

### John Creighto

If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.

3. Feb 28, 2009

### dirk_mec1

Don't you mean the columns of $$A^T b$$ are in the span of the columns of $$A^T A$$? If so I don't understand how to prove such a thing.

4. Mar 1, 2009

### dirk_mec1

I've thought about it and I seriously don't know how to prove that the A^Tb is in the column space of A^TA. Can someone help me?