Suppose vectors X(adsbygoogle = window.adsbygoogle || []).push({}); _{1}, X_{2}, ... , X_{n}whose components are random variables are mutually independent(I mean X_{i}'s are vectors of components with constants which are possible values of random variables labeled by the component indice, and all these labeled random variables are organized as a vector X, hence X_{i}'s just are samples of such a X), and the sample mean is [itex]\hat{M}[/itex] = [itex]\frac{1}{N}[/itex][itex]\sum[/itex]_{[itex]\stackrel{N}{i = 1}[/itex]}X_{i}, and the true mean of all X_{i}'s is M. Then to estimate the covariance matrix of X_{i}, we employ the following formula:

[itex]\hat{Ʃ}[/itex] = [itex]\frac{1}{N}[/itex][itex]\sum[/itex]_{[itex]\stackrel{N}{i = 1}[/itex]}{(X_{i}- [itex]\hat{M}[/itex])(X_{i}- [itex]\hat{M}[/itex])^{T}}

[itex]\ [/itex][itex]\ [/itex][itex]\: [/itex]= [itex]\frac{1}{N}[/itex][itex]\sum[/itex]_{[itex]\stackrel{N}{i = 1}[/itex]}{((X_{i}- M) - ([itex]\hat{M}[/itex] - M))((X_{i}- M) - ([itex]\hat{M}[/itex] - M))^{T}}

[itex]\ [/itex][itex]\ [/itex][itex]\: [/itex]= [itex]\frac{1}{N}[/itex][itex]\sum[/itex]_{[itex]\stackrel{N}{i = 1}[/itex]}(X_{i}- M)(X_{i}- M)^{T}- ([itex]\hat{M}[/itex] - M)([itex]\hat{M}[/itex] - M)^{T}

My question is how does the equal sign hold in the last step?

I did some work about this question, first I note that the transpose is a llinear transformation, i.e., for two vectors V and U, (V + U)^{T}= V^{T}+ U^{T}, then I realize that the following equation islegal.

(V - U)(V - U)^{T}= V[itex]\! [/itex]V^{T}- VU^{T}- UV^{T}+ UU^{T}

Let V = (X_{i}- M) and U = ([itex]\hat{M}[/itex] - M), the terms missing in the last step of [itex]\hat{Ʃ}[/itex] are -VU^{T}and -UV^{T}, OK, I know the entries of E[VU^{T}] actually are covariances of X_{i}and [itex]\hat{M}[/itex], and I assume they are all zero, consequently the terms -VU^{T}and -UV^{T}do miss because of taking the expectation on [itex]\hat{Ʃ}[/itex], but in the last step, they vanished before taking the expectation! Why?

Finally, I also notice that the sign of UU^{T}= ([itex]\hat{M}[/itex] - M)([itex]\hat{M}[/itex] - M)^{T}has been changed from + to -, how does this happen?

**Physics Forums | Science Articles, Homework Help, Discussion**

Dismiss Notice

Join Physics Forums Today!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

# Question about sample covariance matrix

**Physics Forums | Science Articles, Homework Help, Discussion**