Proving Idempotency of a Matrix: A Step-by-Step Guide

[mathmari]

Hey!

We have that a matrix $A$ is idempotent if it holds that $A^2=A$.

We suppose that $X$ is a $m\times n$-matrix and that $(X^TX)^{-1}$ exists.

I want to show that $A=I_m-X(X^TX)^{-1}X^T$ is idempotent. I have done the following:

$$A^2 =A\cdot A=(I_m-X(X^TX)^{-1}X^T)\cdot (I_m-X(X^TX)^{-1}X^T) \\ =I_m(I_m-X(X^TX)^{-1}X^T)-X(X^TX)^{-1}X^T(I_m-X(X^TX)^{-1}X^T) \\ =I_mI_m-I_mX(X^TX)^{-1}X^T-X(X^TX)^{-1}X^TI_m+(X(X^TX)^{-1}X^T)(X(X^TX)^{-1}X^T) \\ =I_m-X(X^TX)^{-1}X^T-X(X^TX)^{-1}X^T+X(X^TX)^{-1}(X^TX)(X^TX)^{-1}X^T \\ =I_m-2X(X^TX)^{-1}X^T+XI_n(X^TX)^{-1}X^T \\ =I_m-2X(X^TX)^{-1}X^T+X(X^TX)^{-1}X^T \\ =I_m-X(X^TX)^{-1}X^T =A$$

Is this correct? (Wondering)

Is the step from the $3$th to the $4$th line correct? Or doesn't it hold that $(X(X^TX)^{-1}X^T)(X(X^TX)^{-1}X^T)=X(X^TX)^{-1}(X^TX)(X^TX)^{-1}X^T$ ? (Wondering)

 

[Opalg]

Hey!

We have that a matrix $A$ is idempotent if it holds that $A^2=A$.

We suppose that $X$ is a $m\times n$-matrix and that $(X^TX)^{-1}$ exists.

I want to show that $A=I_m-X(X^TX)^{-1}X^T$ is idempotent. I have done the following:

$$A^2 =A\cdot A=(I_m-X(X^TX)^{-1}X^T)\cdot (I_m-X(X^TX)^{-1}X^T) \\ =I_m(I_m-X(X^TX)^{-1}X^T)-X(X^TX)^{-1}X^T(I_m-X(X^TX)^{-1}X^T) \\ =I_mI_m-I_mX(X^TX)^{-1}X^T-X(X^TX)^{-1}X^TI_m+(X(X^TX)^{-1}X^T)(X(X^TX)^{-1}X^T) \\ =I_m-X(X^TX)^{-1}X^T-X(X^TX)^{-1}X^T+X(X^TX)^{-1}(X^TX)(X^TX)^{-1}X^T \\ =I_m-2X(X^TX)^{-1}X^T+XI_n(X^TX)^{-1}X^T \\ =I_m-2X(X^TX)^{-1}X^T+X(X^TX)^{-1}X^T \\ =I_m-X(X^TX)^{-1}X^T =A$$

Is this correct? (Wondering)

Is the step from the $3$th to the $4$th line correct? Or doesn't it hold that $(X(X^TX)^{-1}X^T)(X(X^TX)^{-1}X^T)=X(X^TX)^{-1}(X^TX)(X^TX)^{-1}X^T$ ? (Wondering)

Your solution is correct. The associative property of multiplication $A(BC) = (AB)C$ extends to any choice of bracketing in longer expressions. For example $(AB)((CD)E) = A((BC)(DE)).$ The proof of this "general associative law" is messy but not particularly hard or interesting, and consists essentially of using induction to show that any bracketing gives the same result as the standard bracketing where the operations are performed in sequence from left to right, like $(((AB)C)D)E$.

 

[mathmari]

Your solution is correct. The associative property of multiplication $A(BC) = (AB)C$ extends to any choice of bracketing in longer expressions. For example $(AB)((CD)E) = A((BC)(DE)).$ The proof of this "general associative law" is messy but not particularly hard or interesting, and consists essentially of using induction to show that any bracketing gives the same result as the standard bracketing where the operations are performed in sequence from left to right, like $(((AB)C)D)E$.

Ah ok... Thanks a lot! (Happy)