Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Principal Components and the Residual Matrix

  1. Jun 21, 2013 #1
    I've been reading about principal components and residual matrixs.

    It's my understanding if you used every principal component to recalculate your orginal data, then the residual matrix should be 0.

    Therefore, I created a fake dataset of two random variables and calculated the principal components.

    When I do eigenvector1,1*princomp1,1+Eigenvector1,2*princomp1,2 = var 1
    When I do eigenvector2,1*princomp2,1+Eigenvector2,2*princomp2,2 = var 2

    so therefore the residual matrix is 0 which is what I wanted. However, this is only true when I standardize the data.

    If I don't standardized the data, the two formulas I listed above aren't true.

    What is throwing me for a loop is none of the papers I read said anything about standardizing the data, but it looks like the data must be standardized for this to hold. I don't want to make any assumptions so I thought I would ask. Is this correct?
  2. jcsd
  3. Jun 21, 2013 #2


    User Avatar
    Science Advisor

    Hey Jrb599 and welcome to the forums.

    Did you take into account the eigen-values for the principal component eigen-vectors?

    The eigen-values represent the variance component which is related the un-standardized random variables' variance attributes.
  4. Jun 23, 2013 #3
    Hi Chiro,

    Thanks for the response. Yeah I've taken the eigenvalues into account, and I still can't get it to work
  5. Jun 24, 2013 #4


    User Avatar
    Science Advisor

    Just out of curiosity, what eigen-values do you get from PCA for the standardized data? Are they unit length?

    Also you should calculate the PCA matrix and get its inverse to go from PCA space to original space since the PCA is a linear transformation from original space to new space.

    Try this to get the original random variables if you are in the initial PCA space.
  6. Jun 24, 2013 #5
    Chiro - I realized the program I was using was still doing mean-centering. It's working now
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook