Principal Components and the Residual Matrix

[Jrb599]

I've been reading about principal components and residual matrixs.

It's my understanding if you used every principal component to recalculate your orginal data, then the residual matrix should be 0.

Therefore, I created a fake dataset of two random variables and calculated the principal components.


When I do eigenvector1,1*princomp1,1+Eigenvector1,2*princomp1,2 = var 1
similarly
When I do eigenvector2,1*princomp2,1+Eigenvector2,2*princomp2,2 = var 2

so therefore the residual matrix is 0 which is what I wanted. However, this is only true when I standardize the data.

If I don't standardized the data, the two formulas I listed above aren't true.

What is throwing me for a loop is none of the papers I read said anything about standardizing the data, but it looks like the data must be standardized for this to hold. I don't want to make any assumptions so I thought I would ask. Is this correct?

 

[chiro]

Hey Jrb599 and welcome to the forums.

Did you take into account the eigen-values for the principal component eigen-vectors?

The eigen-values represent the variance component which is related the un-standardized random variables' variance attributes.

 

[Jrb599]

Hi Chiro,

Thanks for the response. Yeah I've taken the eigenvalues into account, and I still can't get it to work

 

[chiro]

Just out of curiosity, what eigen-values do you get from PCA for the standardized data? Are they unit length?

Also you should calculate the PCA matrix and get its inverse to go from PCA space to original space since the PCA is a linear transformation from original space to new space.

Try this to get the original random variables if you are in the initial PCA space.

 

[Jrb599]

Chiro - I realized the program I was using was still doing mean-centering. It's working now