Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Chi-square test: why does it follow a Chi-square distribution

  1. Apr 14, 2014 #1

    it is well-known that the Chi-square test between an observed distribution O and an expected distribution E can be interpreted as a test based on (twice) the second order Taylor approximation of the Kullback-Leibler divergence, i.e.: [tex]2\,\mathcal{D}_{KL}(O \| E) \approx \sum_i \frac{(O_i-E_i)^2}{E_i} = \chi^2[/tex]
    where i is the bin of the histogram (or contigency table). A proof is given here (page 5).

    The question is: how do we know that each of the error terms [itex]\frac{(O_i-E_i)^2}{E_i}[/itex] on the right side of the above equation follows a normal distribution N(0,1)? There is probably some some assumption to be made...?
  2. jcsd
  3. Apr 15, 2014 #2

    Stephen Tashi

    User Avatar
    Science Advisor

    [itex] \frac{ (O_i - E_i)^2}{E_i} [/itex] is nonnegative, so it doesn't follow a normal distribution.

    If [itex] X [/itex] is a binomial random variable representing the number of "successes" n independent trials with probability of success [itex] p [/itex] on each trial then the distribution of [itex]Y = \frac {X-np}{\sqrt{np(1-p)}
    } [/itex] can be approximated by a [itex] N(0,1) [/itex] distribution.
    Last edited: Apr 15, 2014
  4. Apr 15, 2014 #3
    I see. There it is our assumption!
    It seems to me that such an assumption automatically implies that the data in the cells of the contingency table are assumed to follow a multinomial distribution.

    So in the end, although the formula for calculating the [itex]\chi^2[/itex] value is just an approximation of the Kullback-Leibler divergence, if we are willing to perform a decision test we still need the assumption that we are dealing with a multinomial distribution, otherwise the [itex]\chi^2[/itex] value that we calculated according to the formula above, does not necessarily follow a chi2-distribution.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook