Proving the pseudoinverse gives the minimum error solution

[liengen]

Homework Statement

Let's say I have a non invertible system of linear equations: Ax=b
Then the pseudoinverse gives a approximate solution: x'=A^+*b

(1) Given the property: A*A^+*A = A, prove that x' is a vector which lies in the image of A and minimises the error = ||b-Ax'||

(2) show that the magnitude of the error is ||Ax'-b||^2=b^T*(I-A*A^+)*b and that the error vector lies in the kernel of A

Homework Equations

A^+ is the pseudoinverse

The Attempt at a Solution

Let's just consider (1) for now.
I know that since b does not lie in the column space of A, we have to find a approximate solution by projecting b onto this space. Let's denote this projected as proj_b.
Thus: Ax' = proj_b.
It's kinda obvious that x' lies in the image of A. I just cannot see the link between this and the given property A*A^+*A = A.

 

[jbunniii]

Homework Statement

Let's say I have a non invertible system of linear equations: Ax=b
Then the pseudoinverse gives a approximate solution: x'=A^+*b

(1) Given the property: A*A^+*A = A, prove that x' is a vector which lies in the image of A and minimises the error = ||b-Ax'||

(2) show that the magnitude of the error is ||Ax'-b||^2=b^T*(I-A*A^+)*b and that the error vector lies in the kernel of A

Homework Equations

A^+ is the pseudoinverse

The Attempt at a Solution

Let's just consider (1) for now.
I know that since b does not lie in the column space of A, we have to find a approximate solution by projecting b onto this space. Let's denote this projected as proj_b.
Thus: Ax' = proj_b.
It's kinda obvious that x' lies in the image of A. I just cannot see the link between this and the given property A*A^+*A = A.

It is not at all obvious to me that x' is in the image of A. What if A is an $m \times n$ matrix, with $m \neq n$? Then $x'$ doesn't even have the right dimension. Is there an unstated assumption or a typo somewhere?

 

[liengen]

It is not at all obvious to me that x' is in the image of A. What if A is an $m \times n$ matrix, with $m \neq n$? Then $x'$ doesn't even have the right dimension. Is there an unstated assumption or a typo somewhere?

I see your point. Nothing else is stated though.

If the rows or columns are linearly dependent, I still think it's obvious, since then we just have either the same equations (rows) or the unknowns equal each other (columns) and the pseudoinverse just kills these. We still get the right answer. When we have a unsolvable system however (i.e more linearly independent rows than columns) then I agree, x' doesn't have the right dimension compared with A.