Principal Component Analysis vs Factor Analysis vs regression

by lalbatros
Tags: analysis, component, factor, principal, regression
lalbatros is offline
Oct17-05, 02:54 PM
P: 1,235
I just gave a try to a statistical excel add-in and found PCA and FA quite interresting.
However, I don't see where are the differences between these two analysis, except for the layout of the results.
Additionaly, I see the link with multiple regression, but I don't see the link precisely enough.

I see that both PCA and FA produce factors.
I understand factors as independent and normalized random variables from which the variability of the dataset can be reproduced.
I also appreciate the ranking of the factors according to the their contribution to the global variance.
But where are the difference between PCA and FA?

The comparison with multiple linear regression left me uneasy too.
I first tought I would be able to calculate the slope of simple linear regression from the PCA factor loadings.
This works rather well when dispersion is small, but there are large differences when dispersion is large.

Digging more into the details,
- I observed small differences in the "factor loading" from PCA or FA
- I had difficulty to reproduce all the details by SVD decomposition of the correlation matrix, but numbers are close

Could some of you provide me with helpful comments.
Web links to well-written descriptions and/or comparisons of the methods would be highly appreciated too.
Phys.Org News Partner Science news on
Better thermal-imaging lens from waste sulfur
Hackathon team's GoogolPlex gives Siri extra powers
Bright points in Sun's atmosphere mark patterns deep in its interior
EnumaElish is offline
Oct18-05, 05:49 PM
Sci Advisor
HW Helper
EnumaElish's Avatar
P: 2,483

Register to reply

Related Discussions
[SOLVED] Regression Analysis for a Gamma function Set Theory, Logic, Probability, Statistics 3
power analysis of low frequency component Electrical Engineering 0
Independent Component Analysis (ICA) Calculus 0