The sample covariance of N observations of K

  • Context: Undergrad 
  • Thread starter Thread starter DUET
  • Start date Start date
  • Tags Tags
    Covariance
Click For Summary

Discussion Overview

The discussion revolves around the properties of the sample covariance matrix derived from N observations of K variables, specifically addressing its dimensions and characteristics. Participants explore whether the covariance matrix should be described as K-by-K or N-by-K, and the implications of its square nature.

Discussion Character

  • Technical explanation
  • Debate/contested

Main Points Raised

  • Some participants assert that the sample covariance matrix is correctly described as K-by-K, indicating that it is a square matrix representing the covariances between K variables.
  • Others question whether the covariance matrix could be N-by-K, seeking clarification on the dimensions based on the number of observations versus the number of variables.
  • It is noted that the covariance matrix must be square, as it contains all possible covariance values between pairs of variables, leading to N*N or N^2 entries.
  • Participants discuss that the off-diagonal elements represent covariances between different variables, while diagonal elements represent variances.
  • There is a consensus that each covariance value appears twice in the matrix due to the symmetry of covariance (Cov(X,Y) = Cov(Y,X)).

Areas of Agreement / Disagreement

Participants generally agree that the covariance matrix is K-by-K and must be square, but there is some debate regarding the initial interpretation of its dimensions and the implications of having N observations.

Contextual Notes

Some assumptions regarding the definitions of covariance and the nature of the observations may not be fully articulated, leading to potential misunderstandings about the matrix dimensions.

DUET
Messages
55
Reaction score
0
http://en.wikipedia.org/wiki/Covariance

I read from the above link that-

The sample covariance of N observations of K variables is the K-by-K matrix.

I am wondering-

Should "K-by-K" be "N-by-K" or not?
 
Physics news on Phys.org
DUET said:
http://en.wikipedia.org/wiki/Covariance

I read from the above link that-

The sample covariance of N observations of K variables is the K-by-K matrix.

I am wondering-

Should "K-by-K" be "N-by-K" or not?
No. K-by-K is correct. Moreover it is symmetric. Each term in the matrix is the estimated covariance between two of the variables.
 
Does it mean covariance matrix must be in square size, 3x3, 4x4, 5x5 etc?

It can never be 3x4, 4x7, 5x8 etc.
Correct?
 
Last edited:
Yes it has to be square. Remember that you need all possible values of Cov(X,Y) where X and Y are random variables. If you have N different variables then you have N*N or N^2 possibilities and since Cov(X,Y) = Cov(Y,X) you will have a lot of repeated values.
 
  • Like
Likes   Reactions: 1 person
chiro said:
you will have a lot of repeated values.
I think every value will repeat twice. Am I correct?
 
In general yes unless you are looking at the same random variable since Cov(X,X) = Var(X) on the diagonal elements.
 
Yes! I am just taking off diagonal elements into account.
 
Last edited:

Similar threads

Undergrad Seeking an intuition linking definitions of contravariant and covariant
  • · Replies 1 ·
Replies
1
Views
3K
Undergrad Sample mean and sample covariance
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Proving inequality about dimension of quotient vector spaces
  • · Replies 25 ·
Replies
25
Views
4K
Undergrad Uniqueness of reduced row echelon form (proof verification)
  • · Replies 4 ·
Replies
4
Views
3K
Undergrad Proof regarding congruence relation
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad ##S_{n}(\alpha)=\sum_{k=1}^{n} (-1)^{\lfloor{k\alpha\rfloor}}##
  • · Replies 2 ·
Replies
2
Views
3K
Undergrad Explanation of a Line of a proof in Axler Linear Algebra Done Right 3r
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Analogy question for algebraists
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Showing ##k[x_1,\ldots,x_n]/\mathfrak{a}## is finite dimensional
  • · Replies 40 ·
2
Replies
40
Views
5K
Graduate First eigenvalue not matching, but all others are
  • · Replies 8 ·
Replies
8
Views
2K