# Least squares proof (1 Viewer)

### Users Who Are Viewing This Thread (Users: 0, Guests: 1)

#### dirk_mec1

1. The problem statement, all variables and given/known data

In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

$$A^T A x^*= A^T b$$

Prove that there's always a solution to this equation.

2. Relevant equations
-

3. The attempt at a solution
I distinct 2 situations $$A^T A$$ is invertible and it isn't invertible. If it's invertible then there's no problem $$x^*= (A^T A)^{-1} A^T b$$

But how I prove that it works in the non-invertible case?

#### John Creighto

1. The problem statement, all variables and given/known data

In the least squares method the vector x* that is the best approximation to b statisfies the Least squares equation:

$$A^T A x^*= A^T b$$

Prove that there's always a solution to this equation.

2. Relevant equations

3. The attempt at a solution
I distinct 2 situations $$A^T A$$ is invertible and it isn't invertible. If it's invertible then there's no problem $$x^*= (A^T A)^{-1} A^T b$$

But how I prove that it works in the non-invertible case?
If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.

#### dirk_mec1

If it is not invertible you need another criteria to get a unique solution. For instance you could require that x has minimum norm. In which case use could use the pseudo inverse which is based on singular value decomposition. However, you do not required a unique solution in the above question. So perhaps you could try showing that the columns of A^T form the same column space as the columns of A^TA.
Don't you mean the columns of $$A^T b$$ are in the span of the columns of $$A^T A$$? If so I don't understand how to prove such a thing.

#### dirk_mec1

I've thought about it and I seriously don't know how to prove that the A^Tb is in the column space of A^TA. Can someone help me?

### The Physics Forums Way

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving