Principal Component Analysis vs Factor Analysis vs regression

lalbatros

I just gave a try to a statistical excel add-in and found PCA and FA quite interresting.
However, I don't see where are the differences between these two analysis, except for the layout of the results.
Additionaly, I see the link with multiple regression, but I don't see the link precisely enough.

I see that both PCA and FA produce factors.
I understand factors as independent and normalized random variables from which the variability of the dataset can be reproduced.
I also appreciate the ranking of the factors according to the their contribution to the global variance.
But where are the difference between PCA and FA?

The comparison with multiple linear regression left me uneasy too.
I first tought I would be able to calculate the slope of simple linear regression from the PCA factor loadings.
This works rather well when dispersion is small, but there are large differences when dispersion is large.

Digging more into the details,
- I observed small differences in the "factor loading" from PCA or FA
- I had difficulty to reproduce all the details by SVD decomposition of the correlation matrix, but numbers are close

Could some of you provide me with helpful comments.
Web links to well-written descriptions and/or comparisons of the methods would be highly appreciated too.

EnumaElish

http://en.wikipedia.org/wiki/Multivariate_statistics