Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Expected value of sample variance

  1. Jun 23, 2009 #1
    Hi,

    My question is related to this web page. http://en.wikipedia.org/wiki/Estimator_bias

    In the Examples section, note the equation for the expected value of sample variance.

    [tex] {E}(S^2)=\frac{n-1}{n} \sigma^2 [/tex]


    Could anybody please show me the steps to go from the sample variance equation (given below) to the above equation?


    [tex]S^2=\frac{1}{n}\sum_{i=1}^n(X_i-\overline{X}\,)^2[/tex]


    Thanks

    MG.
     
  2. jcsd
  3. Jun 23, 2009 #2
    well, that "sample variance" was defined for the purposes of that page. The usual sample variance divides by n-1 instead of by n, so it is not biased. This page includes a derivation of that fact.
     
  4. Jun 23, 2009 #3

    mathman

    User Avatar
    Science Advisor
    Gold Member

    The essential point for the use of n-1 rather than n is that the sample variance makes use of the sample mean, not the theoretical mean.

    Specifically, let x be one sample, m the theoretical mean and a the statistical average.
    Then E(x-a)2=E(x-m+m-a)2=E(x-m)2+E(m-a)2+2E((x-m)(m-a)).
    When you plow through the details, the factor shows up.
     
  5. Jun 23, 2009 #4
    Thanks folks. However, my question is not about the use of n-1 in the denominator. I understand the concept of the degrees of freedom.

    I wish to know the operations/steps I need to perform on the Sample Variance equation to get the expected value equation.

    Thanks again,

    MG.
     
  6. Jun 24, 2009 #5
    I gave you the answer.
     
  7. Jun 24, 2009 #6

    statdad

    User Avatar
    Homework Helper

    Is this what you're looking for?

    First consider (I'll bring in the 1/n later)

    [tex]
    \sum (x_i - \bar x)^2 = \sum x_i^2 - n\bar{x}^2
    [/tex]

    The expected value of this expression is

    [tex]
    \begin{align*}
    E\left(\sum(x_i - \bar x^2)^2\right) &= \sum E(x_i^2) - n E\left( \bar{x}^2\right)\\
    & = \sum \left(\mu^2 + \sigma^2\right) - n \frac 1 {n^2} \left(\sum E(x_i^2) + \sum_{i<j} x_i x_j \right) \\
    & = n\mu^2 + n \sigma^2 - \frac 1 n \left( n \mu^2 + n \sigma^2 + n(n-1) \mu^2 \right) \\
    & = n\mu^2 + n \sigma^2 - \mu^2 - \sigma^2 - (n-1) \mu^2 \\
    & = n\mu^2 + n\sigma^2 - n \mu^2 - \sigma^2 \\
    & = (n-1) \sigma^2
    \end{align*}
    [/tex]

    Now
    [tex]
    \begin{align*}
    S^2 & = \frac 1 n \sum (x_i - \bar{x})^2) \\
    E(S^2) & = \frac 1 n E\left(\sum (x_i - \bar{x}^2) \right) \\
    & = \left(\frac 1 n \right) (n-1) \sigma^2 = \frac{n-1} n \sigma^2
    \end{align*}
    [/tex]

    and from this last line we see that in order to obtain an unbiased estimate of [tex] \sigma^2 [/tex], the maximum likelihood (for normal distributions) estimator [tex] S^2 [/tex] needs to be multiplied by (n)/(n-1) to get

    [tex]
    \frac 1 {n-1} \sum (x_i - \bar{x})^2)
    [/tex]
     
  8. Jun 24, 2009 #7

    Attached Files:

    Last edited: Jun 24, 2009
  9. Jun 24, 2009 #8
    Statdad

    I am not clear about just one step.

    How do I get

    [tex] (\left(\mu^2 + \sigma^2\right) [/tex] from [tex] E (x_i^2) [/tex]


    Thanks

    MG.

    P.S. How do you manage to write so many equations efficiently using LaTex? Do you have an advanced editor?
     
  10. Jun 24, 2009 #9

    statdad

    User Avatar
    Homework Helper

    First:
    Since
    [tex]
    Var(X) = \sigma^2 = E(X - \mu)^2 = E(X^2) - \mu^2
    [/tex]

    a simple re-arrangement gives

    [tex]
    E(X^2) = \sigma^2 + \mu^2
    [/tex]

    Second question: if you want to have several equations nicely aligned inside a display, use the \begin{align*} and \end{align*} pair inside the tex delimiters. Without the tex info, if i have

    f(x) & = x^2 + 5x + 6 \\
    & = (x+3)(x+2)

    inside the delimiters, the compiled result is

    [tex]
    \begin{align*}
    f(x) & = x^2 + 5x + 6 \\
    & = (x+3)(x+2)
    \end{align*}
    [/tex]

    * the "&" sign causes the equations to be aligned at the start of the next symbol ("=" in my
    example)
    * the "\\" terminates a line and tells tex to begin a new line

    If you click on any displayed formula you should see, in a pop-up window, the underlying code.

    Edited to note: some older tex manuals will discuss the use of the "eqarray" (I think I have the name correct, but since I don't use it I'm not going to claim 100% accuracy here) environment for doing what I've done
    with align*. Don't use eqarray - the spacing is (to state it as nicely as possible) horrific.
     
  11. Jun 25, 2009 #10
    Statdad,

    Thanks a lot. I really appreciate your help.

    Also,

    [tex]
    \begin{align*}

    Var (X) & = E [ X - E (X) ]^2 \\
    & = E [ X^2 - 2X E(X) + E(X)^2] \\
    & = E(X^2) - 2 E(X) E(X) + E(X)^2 \\
    & = E(X^2) - 2 E(X)^2 + E(X)^2 \\
    & = E (X^2) - E(X)^2.


    \end{align*}

    [/tex]
     
    Last edited: Jun 25, 2009
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Expected value of sample variance
Loading...