- #1
Trentkg
- 3
- 0
This question broadly relates to principle component analysis (PCA)
Say you have some data vector X, and a linear transformation K that maps X to some new data vector Z:
K*X → Z
Now say you have another linear transformation P that maps Z to a new data vector Y:
P*Z → Y
is there a linear transformation, call it M, that maps X to Y?
M*X → Y?
If Y, Z, P and X are known, can we solve for M? I would think we could find M by simple substitution...
M*X → Y,
P*Z → Y,
M → X^-1*P*Z = X^-1 (P*K*X) ?
We'll run into serious problems here if X is not square.
WHY I'M ASKING THIS QUESTION AND HOW IT RELATED TO PCA:
Without going into too much detail, PCA is a dimensional reduction technique. It seeks to find a Linear transformation P that maps Z to Y, such that the matrix Cy, defined as:
Cy == 1/n Y*Y^T is diagonalized.
Cy is the covariance matrix--the diagonal terms represent the covariance of the system while the off diagonal terms represent the variance (To see why this is true, write Y as an MxN matrix with elements i->j. If these elements are mean Zero, what does element 1/n Yi x Yj look like? What about 1/n Yi x Yi? )When Cy is diagonlized, the diagonal terms (variance of the system) is maximized, while the off diagonal terms (the covariance of the system) are minimized (set to zero). Y is Z in an ew basis, with the highest variance of the system is aligned along the first eigenvector, the second highest variance of the system alinged along the second, so on and so forth. The idea is if there are 20 measurements in a ssytem, yet you can express 99% of the variance of the system in only the first 4 eigenvectors, then your 20 dimensional system can probably be reduced to 4. ( a better explanation can be found here: http://www.snl.salk.edu/~shlens/pca.pdf )
Usually X is setup as an mxn matrix where m is the number of different measurements and n the number of trials. The first transformation, K, could be standardization/normlization, or changing units. The fear is that the variance of one measurement will dominate. If m1 has variance 1, and m2 variance 10000, then m2 will dominate the covariance matrix even if m1 is in units of cm and m2 units of km. Hence, we must standardize the variables so they are comparable with a map K.
The problem, then, is that the transformation taking eigenvalues that map Z to Y are in terms of the data set Z. Data set Z may not be of any interest to the experimenter (in this case, ME!). I'm interested in what the eigenvectors/Principle components are of data set X!
Anyways, thanks for any help!
Say you have some data vector X, and a linear transformation K that maps X to some new data vector Z:
K*X → Z
Now say you have another linear transformation P that maps Z to a new data vector Y:
P*Z → Y
is there a linear transformation, call it M, that maps X to Y?
M*X → Y?
If Y, Z, P and X are known, can we solve for M? I would think we could find M by simple substitution...
M*X → Y,
P*Z → Y,
M → X^-1*P*Z = X^-1 (P*K*X) ?
We'll run into serious problems here if X is not square.
WHY I'M ASKING THIS QUESTION AND HOW IT RELATED TO PCA:
Without going into too much detail, PCA is a dimensional reduction technique. It seeks to find a Linear transformation P that maps Z to Y, such that the matrix Cy, defined as:
Cy == 1/n Y*Y^T is diagonalized.
Cy is the covariance matrix--the diagonal terms represent the covariance of the system while the off diagonal terms represent the variance (To see why this is true, write Y as an MxN matrix with elements i->j. If these elements are mean Zero, what does element 1/n Yi x Yj look like? What about 1/n Yi x Yi? )When Cy is diagonlized, the diagonal terms (variance of the system) is maximized, while the off diagonal terms (the covariance of the system) are minimized (set to zero). Y is Z in an ew basis, with the highest variance of the system is aligned along the first eigenvector, the second highest variance of the system alinged along the second, so on and so forth. The idea is if there are 20 measurements in a ssytem, yet you can express 99% of the variance of the system in only the first 4 eigenvectors, then your 20 dimensional system can probably be reduced to 4. ( a better explanation can be found here: http://www.snl.salk.edu/~shlens/pca.pdf )
Usually X is setup as an mxn matrix where m is the number of different measurements and n the number of trials. The first transformation, K, could be standardization/normlization, or changing units. The fear is that the variance of one measurement will dominate. If m1 has variance 1, and m2 variance 10000, then m2 will dominate the covariance matrix even if m1 is in units of cm and m2 units of km. Hence, we must standardize the variables so they are comparable with a map K.
The problem, then, is that the transformation taking eigenvalues that map Z to Y are in terms of the data set Z. Data set Z may not be of any interest to the experimenter (in this case, ME!). I'm interested in what the eigenvectors/Principle components are of data set X!
Anyways, thanks for any help!
Last edited by a moderator:
