Valid Covariance Matrices

Tags:
1. Nov 30, 2014

weetabixharry

I'm trying to understand what makes a valid covariance matrix valid. Wikipedia tells me all covariance matrices are positive semidefinite (and, in fact, they're positive definite unless one signal is an exact linear combination of others). I don't have a very good idea of what this means in practice.

For example, let's assume I have a real-valued covariance matrix of the form:

$$\mathbf{R}=\left[ \begin{array}{ccc} 1 & 0.7 & x \\ 0.7 & 1 & -0.5 \\ x & -0.5 & 1 \end{array} \right]$$
where $x$ is some real number. What range of values can $x$ take?

I can sort of see that $x$ is constrained by the other numbers. Like it can't have magnitude more than 1, because the diagonals are all 1. However, it is also constrained by the off-diagonals.

Of course, for my simple example, I can solve the eigenvalue problem for eigenvalues of zero to give me the range of values (roughly -0.968465844 to 0.268465844)... but this hasn't really given me any insight in a general sense.

I feel like there might be a neat geometrical interpretation that would make this obvious.

Can anyone offer any insight?

Last edited: Nov 30, 2014
2. Dec 1, 2014

mathman

I don't know if this a complete answer. However assume you have three random variables X, Y, Z each with variance 1, cov(X,Y) = 0.7, cov(X,Z) = x, and cov(Y,Z) = -0.5. For simplicity assume all means = 0.
Consider E((X±Y±Z)2)≥0 for all possible sign combinations. This will give you four bounds on x. This may be the best, although I am not sure.

3. Dec 2, 2014

Stephen Tashi

The terminology "covariance matrix" is ambiguous. There is a covariance matrix for random variables and there is a covariance matrix computed from samples of random variables. I don't think it works to claim that the sample covariance matrix is just the covariance matrix of a population consisting of the sample because the usual way to compute the sample covariance involves using denominators of n-1 instead of n.

4. Dec 8, 2014

weetabixharry

I'll have to give this some thought. It's not obvious to me how this works.

5. Dec 8, 2014

weetabixharry

What's the difference between these two? Do either/both have to be Hermitian positive semidefinite? That's the sort I'm interested in.

6. Dec 9, 2014

mathman

1.5+cov(X,Y)+cov(X,Z)+cov(Y,Z)≥0
1.5-cov(X,Y)+cov(X,Z)-cov(Y,Z)≥0
1.5+cov(X,Y)-cov(X,Z)-cov(Y,Z)≥0
1.5-cov(X,Y)-cov(X,Z)+cov(Y,Z)≥0