Variance-covariance matrix of random vector

**kingwinner** · Jun16-09, 12:31 AM

Notation:
Var(Y) is the variance-covariance matrix of a random vector Y
B' is the tranpose of the matrix B.

1) Let A be a m x n matrix of constants, and Y be a n x 1 random vector. Then Var(AY) = A Var(Y) A'

Proof:
Var(AY)
= E[(AY-A E(Y)) (AY-A E(Y))' ]
= E[A(Y-E(Y)) (Y-E(Y))' A' ]
= A E[(Y-E(Y)) (Y-E(Y))'] A'
= A Var(Y) A'

Now, I don't understand the step in red. What theorem is that step using?
I remember a theorem that says if B is a m x n matrix of constants, and X is a n x 1 random vector, then BX is a m x 1 matrix and E(BX) = B E(X), but this theorem doesn't even apply here since it requries X to be a column vector, not a matrix of any dimension.

2) Theorem: Let Y be a n x 1 random vector, and B be a n x 1 vector of constants(nonrandom), then Var(B+Y) = Var(Y).

I don't see why this is true. How can we prove this?
Is it also true that Var(Y+B) = Var(Y) ?

Any help is greatly appreciated!

**statdad** · Jun16-09, 08:02 AM

For question 1: the matrix

is constant, so it (and

) can be factored outside of the expectation. This is the same type of principal you use with random variables (think

).

For 2: Again,

is a collection of constants, and addition of constants doesn't change the variance of a random variable. In a little more detail:

$LaTeX Code: \\begin{align*} E(Y + B) & = \\mu_Y + B \\\\ Var(Y+B) & = E[((Y+B) - (\\mu_Y + B))((Y+B) - (\\mu_Y+B))single-quote] \\\\ & = E[(Y-\\mu_Y)(Y-\\mu_Y)single-quote] = Var[Y] \\end{align*} $

**kingwinner** · Jun16-09, 04:41 PM

1) But how can we prove it rigorously in the general case of random matrices?
i.e. how can we prove that
E(AZ) = A E(Z)
and E(W A') = E(W) A' ?
where Z and W are any random matrices, and A is any constant matrix such that the product is defined

2) Thanks for the proof! Now I can see more rigorously why that property is true in the multivariate context.

**EnumaElish** · Jun16-09, 04:59 PM

1) You could start with the 2x2 case then generalize; or use induction.

**statdad** · Jun16-09, 05:16 PM

Originally Posted by kingwinner

1) But how can we prove it rigorously in the general case of random matrices?
i.e. how can we prove that
E(AZ) = A E(Z)
and E(W A') = E(W) A' ?
where Z and W are any random matrices, and A is any constant matrix such that the product is defined

2) Thanks for the proof! Now I can see more rigorously why that property is true in the multivariate context.

Suppose your random matrix is (using the definition of matrix multiplication)

$LaTeX Code: Z = \\begin{pmatrix} z_{11} & z_{12} & \\dots & z_{1k} \\\\ z_{21} & z_{22} & \\hdots & z_{2k} \\\\ \\ddots & \\ddots & \\ddots & \\ddots \\\\ z_{m1} & z_{m2} & \\dots & z_{mk} \\end{pmatrix} $

and that your constant matrix is

with similar notation for its entries.
The

entry of the matrix

is the random variable given by

$LaTeX Code: \\sum_{l=1}^m a_{rl} z_{lt} $

so the expected value of the

entry is

$LaTeX Code: E\\left(\\sum_{l=1}^m a_{rl}z_{lt}\\right) = \\sum_{l=1}^m E\\left(a_{rl}z_{lt}\\right) = \\sum_{l=1}^m a_{rl} E\\left(z_{lt}\\right) $

The second equality is true since each

value is a constant number and each

is a random variable , so the ordinary rules of expectation apply. What does the equation mean?

a) The left side is the expected value of the

entry in the matrix

b) The right side is the

entry in the matrix product of

and the expected value of

(call this

)

This shows that corresponding elements of

and

are equal, so

This type of approach works whether you have random variables or random vectors.

**kingwinner** · Jun16-09, 05:34 PM

1) Once again, thanks for the great proof!

And I suppose the proof of E(W A') = E(W) A', with the constant matrix on the right of a random matrix W, can be done similarly, right?

**statdad** · Jun16-09, 05:42 PM

Originally Posted by kingwinner

1) Once again, thanks for the great proof!

And I suppose the proof of E(W A') = E(W) A', with the constant matrix on the right of a random matrix W, can be done similarly, right?

Yes, as can the derivations for the case of random and constant vectors.

**kingwinner** · Jun16-09, 05:59 PM

I am trying to modify your proof to prove that E(ZA) = E(Z) A (assuming ZA is defined), but it doesn't seem to work out...

The

entry of the matrix

is the random variable given by

$LaTeX Code: \\sum_{l=1}^m Z_{rl} a_{lt} $

so the expected value of the

entry is

$LaTeX Code: E\\left(\\sum_{l=1}^m Z_{rl}a_{lt}\\right) = \\sum_{l=1}^m E\\left(Z_{rl}a_{lt}\\right) = \\sum_{l=1}^m a_{lt} E\\left(Z_{rl}\\right) $

?????

**statdad** · Jun16-09, 06:09 PM

[quote=kingwinner;2239839]I am trying to modify your proof to prove that E(ZA) = E(Z) A (assuming ZA is defined), but it doesn't seem to work out...

The

entry of the matrix

is the random variable given by

$LaTeX Code: \\sum_{l=1}^m Z_{rl} a_{lt} $

so the expected value of the

entry is

$LaTeX Code: E\\left(\\sum_{l=1}^m Z_{rl}a_{lt}\\right) = \\sum_{l=1}^m E\\left(Z_{rl}a_{lt}\\right) = \\sum_{l=1}^m a_{lt} E\\left(Z_{rl}\\right) $

Remember you want the matrix

to appear on the right, so factor the constants to the right in the sum (it doesn't matter for constants and variables, but it will make reconstructing the matrix product easier). Also make sure you have the matrices' indexes organized correctly.

**kingwinner** · Jun16-09, 06:27 PM

Thanks a lot, statdad! You are of great help!