Can the Covariance of Random Vectors be Bounded by their Norms?

[Pere Callahan]

Hi there,

I am trying to prove the following. For any random vectors X,Y,Z,W in $\mathbb{R}^d$ and deterministic $d\times d$ matrices A,B the covariance
$$\operatorname{\mathbb{C}ov}\left(X^TAY;Z^TBW\right)$$
can in some way be bounded by the covariance
$$\operatorname{\mathbb{C}ov}\left(X^TY;Z^TW\right)$$
and the norms of the matrices A and B. This is trivial if d=1, because then the first covariance is just AB times the second one, but I could not manage to prove something analogous in higher dimensions.

Any hints or tips are deeply appreciated,

Pere

 

[mmwave]

Dear Pere,

Thank you for your question. This is an interesting problem that has applications in various fields such as statistics, signal processing, and machine learning. After some consideration, I believe I have found a way to prove the desired bound.

Firstly, let us define the random vectors X' = AX and Y' = BY. Then, the covariance \operatorname{\mathbb{C}ov}\left(X^TAY;Z^TBW\right) can be rewritten as \operatorname{\mathbb{C}ov}\left(X'^TY';Z^TW\right). This is because the matrices A and B are deterministic and therefore can be moved outside of the covariance operator.

Next, we can use the Cauchy-Schwarz inequality to bound the covariance as follows:

\operatorname{\mathbb{C}ov}\left(X'^TY';Z^TW\right) \leq \sqrt{\operatorname{\mathbb{Var}}(X'^T) \operatorname{\mathbb{Var}}(Y'^T)} \sqrt{\operatorname{\mathbb{Var}}(Z^T) \operatorname{\mathbb{Var}}(W^T)}

= \sqrt{\operatorname{\mathbb{Var}}(X^T) \operatorname{\mathbb{Var}}(Y^T) \operatorname{\mathbb{Var}}(Z^T) \operatorname{\mathbb{Var}}(W^T)} \sqrt{\operatorname{tr}(A^TA) \operatorname{tr}(B^TB)}

= \sqrt{\operatorname{\mathbb{Var}}(X^T) \operatorname{\mathbb{Var}}(Y^T) \operatorname{\mathbb{Var}}(Z^T) \operatorname{\mathbb{Var}}(W^T)} \sqrt{\operatorname{tr}(A^T) \operatorname{tr}(A) \operatorname{tr}(B^T) \operatorname{tr}(B)}

= \sqrt{\operatorname{\mathbb{Var}}(X^T) \operatorname{\mathbb{Var}}(Y^T) \operatorname{\mathbb{Var}}(Z^T) \operatorname{\mathbb{Var}}(W^T)} \sqrt{\operatorname{tr}(A^T) \operatorname