(Sequential) Principal Component Analysis

[javicm]

Hello everybody!

Any of you have read the paper "EM algorithms for PCA and SPCA" of Sam Roweis? If so, maybe you could help me! :-) I have some problems with equation 2b, well, in fact my problem is that I don't manage to deduce it. If you didn't read the paper but are curious, how could you prove that
$$\frac{N(Cx,R)|_yN(0,I)|_x}{N(0,CC^T+R)|_y}$$ follows this distribution: $$N(\beta y, I-\beta C)|_x$$, where $$N(A,B)|_c$$ means a normal (Gaussian) distribution with mean A, covariance matrix B and evaluated at c, and $$\beta = C^T(CC^T+R)^{-1}$$.

Thanks a lot!
Javier

 

[mmwave]

Hello Javier,

I have not read the paper "EM algorithms for PCA and SPCA" by Sam Roweis, but I am familiar with the topic of EM algorithms. In order to prove the distribution you mentioned, we can use the properties of normal distributions and the definition of conditional probability.

First, let's rewrite the equation as follows:

\frac{N(Cx,R)|_yN(0,I)|_x}{N(0,CC^T+R)|_y} = N(\beta y, I-\beta C)|_x

We can expand the numerator using the definition of conditional probability:

N(Cx,R)|_yN(0,I)|_x = N(Cx,R) \times \frac{N(0,I)|_x}{N(0,CC^T+R)|_y}

Then, using the properties of normal distributions, we can rewrite the numerator as follows:

N(Cx,R)|_yN(0,I)|_x = N(Cx,R) \times \frac{N(Cx,CC^T+R)}{N(0,CC^T+R)|_y}

We can simplify the fraction by using the definition of \beta:

\frac{N(Cx,CC^T+R)}{N(0,CC^T+R)|_y} = \frac{N(Cx,CC^T+R)}{N(0,CC^T+R)} \times \frac{N(0,CC^T+R)}{N(0,CC^T+R)|_y} = \beta \times \frac{N(0,CC^T+R)}{N(0,CC^T+R)|_y}

Substituting this back into the original equation, we get:

\frac{N(Cx,R)|_yN(0,I)|_x}{N(0,CC^T+R)|_y} = N(Cx,R) \times \beta \times \frac{N(0,CC^T+R)}{N(0,CC^T+R)|_y}

Using the properties of normal distributions again, we can rewrite the fraction as:

\frac{N(0,CC^T+R)}{N(0,CC^T+R)|_y} = \frac{