Principal Component Analysis: eigenvectors?

evidenso

Hey
Hello, Im dealing with som Principal Component Analysis
Can anyone explain why the first eigenvector of a covariance matrix gives the direction of maximum variability. why this special property of eigenvectors

atyy

http://en.wikipedia.org/wiki/Normal_distribution

http://en.wikipedia.org/wiki/Multiva...l_distribution

A univariate Gaussian has only one variance, which appears in the denominator of the argument of the exponential.
A multivariate Gaussian has a covariance matrix, which appears in the "denominator" of the argument of the exponential.

Principal components analysis essentially assumes a multivariate Gaussian, then rotates the covariance matrix until it is diagonal, so that the diagonal elements are the variances of the rotated variables. The rotated variables are called "eigenvectors" and their variances are called "eigenvalues". The eigenvectors are conventionally arranged so that the one with the largest eigenvalue is "first", which is equivalent the largest variance being "first".