Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Var(X) = Cov(X,X) ?

  1. Jan 24, 2013 #1
    Var(X) = Cov(X,X) ??

    [tex]Var(X)=\sum_{i=1}^N P(X_i)(X_i-EX)^2.[/tex]

    [tex]Cov(X,Y) = \sum_{i=1}^N\sum_{j=1}^M P(X_i,Y_j)(X_i - EX)(Y_j - EY).[/tex]

    If, for instance, [itex]P(X_i) = 1/N[/itex] and [itex]X = Y = (1,2,3)[/itex], then

    [tex]Var(X) = \frac{1}{3} ((1-2)^2 + (2-2)^2 + (3-2)^2) = \frac{2}{3},[/tex]


    [tex]Cov(X,X) = \sum_{i=1}^3 \sum_{j=1}^3 \frac{1}{9} (X_i - EX)(X_j - EX)[/tex]

    [tex]=\frac{1}{9}((1-2)^2+(3-2)^2+2(1-2)(3-2)) = 2 - 2 = 0??[/tex]

    There are 9 values of (X,Y); each occurs with equal probability. I've omitted the terms that contain (2-2) from the summation. Apparently I've misunderstood something about the definition of covariance, but what?
  2. jcsd
  3. Jan 24, 2013 #2
    Re: Var(X) = Cov(X,X) ??

    This formula is wrong.

    Here is how you calculate it. By definition, the covariance is

    [tex]Cov(X,X)=E[ (X-EX)(X-EX) ][/tex]

    So define the random variable [itex]Z=(X-EX)(X-EX)=(X-2)^2[/itex]. The covariance is EZ. Now, if X takes on the values 1,2 and 3. Then Z takes on the values 0,1. Furthermore [itex]P\{Z=0\}=P\{X=2\}=1/3[/itex] and [itex]P\{Z=1\}=P(\{X=1\}\cup \{X=3\}) = 2/3[/itex].


    [tex]Cov(X,X)=EZ = \sum_{k=0}^1 k P\{Z=k\} = 2/3[/tex]
  4. Jan 24, 2013 #3


    User Avatar
    Science Advisor

    Re: Var(X) = Cov(X,X) ??

    Hey Rasalhague.

    I don't know what you did, but Ill use the expanded form of covariance in your definition.

    = E[(X - E[X])(X - E[X])]
    = E[X^2] - E[X]^2.

    You are not applying the expectation operator correctly since you are need to apply the definition of the expectation to the whole definition (i.e (X-E[X])(X-E[X) and this means taking into account shifts by the mean.

    If you expand the Covariance operator you get:

    Cov(X,Y) = E[XY] - E[X]E[Y] and this is done using some simple algebra which leaves us with

    Cov(X,X) = E[X^2] - E[X]^2 which is the same as the variance.

    You are not calculating the variance or covariance but something that I have absolutely no idea with.
  5. Jan 24, 2013 #4
    Re: Var(X) = Cov(X,X) ??

    The formula defines covariance for discrete variables in Simon & Blume (1994): Mathematics for Economists, end of section A5.4, and in Robert J. Serfling's online intro 'Covariance and Correlation', formula (1) which he identifies with E[(X-EX)(Y-EY)P(X,Y)] in the formula which follows that. Serfling also states that P(X,Y) means

    [tex]P_X(X)\cdot P_Y(Y)[/tex]

    which in my example makes P(X,X) = (1/3)(1/3) = 1/9. Perhaps you could explain how you would calculate an example where [itex]X \neq Y[/itex], e.g. X = (1, 2, 3) and Y = (1, 4, 9).
  6. Jan 24, 2013 #5
    Re: Var(X) = Cov(X,X) ??

    I'm not sure how to reconcile Serfling's formula (1) with the way Wolfram Mathworld writes it out explicitly for the case where N = M:


    Are there two somewhat different concepts each called covariance, each corresponding to its own way of defining the mean of the product of two random variables?
  7. Jan 24, 2013 #6
    Re: Var(X) = Cov(X,X) ??

    Ah, reading further on that Mathworld article, it seems one definition concerns real-valued random variables from a finite sample space, another definition concerns tuples of such random variables. But still, there appear to be a variety of concepts here to which the name covariance is attached, with disagreement over certain points, and Mathworld doesn't give an explicit version of the more general definition.
    Last edited: Jan 24, 2013
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook