What Are the Constraints of a Valid Covariance Matrix?

[weetabixharry]

I'm trying to understand what makes a valid covariance matrix valid. Wikipedia tells me all covariance matrices are positive semidefinite (and, in fact, they're positive definite unless one signal is an exact linear combination of others). I don't have a very good idea of what this means in practice.

For example, let's assume I have a real-valued covariance matrix of the form:

$$\mathbf{R}=\left[ \begin{array}{ccc} 1 & 0.7 & x \\ 0.7 & 1 & -0.5 \\ x & -0.5 & 1 \end{array} \right]$$
where $x$ is some real number. What range of values can $x$ take?

I can sort of see that $x$ is constrained by the other numbers. Like it can't have magnitude more than 1, because the diagonals are all 1. However, it is also constrained by the off-diagonals.

Of course, for my simple example, I can solve the eigenvalue problem for eigenvalues of zero to give me the range of values (roughly -0.968465844 to 0.268465844)... but this hasn't really given me any insight in a general sense.

I feel like there might be a neat geometrical interpretation that would make this obvious.

Can anyone offer any insight?

 

[mathman]

I don't know if this a complete answer. However assume you have three random variables X, Y, Z each with variance 1, cov(X,Y) = 0.7, cov(X,Z) = x, and cov(Y,Z) = -0.5. For simplicity assume all means = 0.
Consider E((X±Y±Z)²)≥0 for all possible sign combinations. This will give you four bounds on x. This may be the best, although I am not sure.

 

[Stephen Tashi]

I'm trying to understand what makes a valid covariance matrix valid.

The terminology "covariance matrix" is ambiguous. There is a covariance matrix for random variables and there is a covariance matrix computed from samples of random variables. I don't think it works to claim that the sample covariance matrix is just the covariance matrix of a population consisting of the sample because the usual way to compute the sample covariance involves using denominators of n-1 instead of n.

 

[weetabixharry]

Consider E((X±Y±Z)²)≥0 for all possible sign combinations. This will give you four bounds on x.

I'll have to give this some thought. It's not obvious to me how this works.

 

[weetabixharry]

There is a covariance matrix for random variables and there is a covariance matrix computed from samples of random variables.

What's the difference between these two? Do either/both have to be Hermitian positive semidefinite? That's the sort I'm interested in.

 

[mathman]

I'll have to give this some thought. It's not obvious to me how this works.

1.5+cov(X,Y)+cov(X,Z)+cov(Y,Z)≥0
1.5-cov(X,Y)+cov(X,Z)-cov(Y,Z)≥0
1.5+cov(X,Y)-cov(X,Z)-cov(Y,Z)≥0
1.5-cov(X,Y)-cov(X,Z)+cov(Y,Z)≥0