Chi-square test: why does it follow a Chi-square distribution

  • Thread starter mnb96
  • Start date
  • #1
713
5
Hello,

it is well-known that the Chi-square test between an observed distribution O and an expected distribution E can be interpreted as a test based on (twice) the second order Taylor approximation of the Kullback-Leibler divergence, i.e.: [tex]2\,\mathcal{D}_{KL}(O \| E) \approx \sum_i \frac{(O_i-E_i)^2}{E_i} = \chi^2[/tex]
where i is the bin of the histogram (or contigency table). A proof is given here (page 5).

The question is: how do we know that each of the error terms [itex]\frac{(O_i-E_i)^2}{E_i}[/itex] on the right side of the above equation follows a normal distribution N(0,1)? There is probably some some assumption to be made...?
 

Answers and Replies

  • #2
Stephen Tashi
Science Advisor
7,707
1,515
The question is: how do we know that each of the error terms [itex]\frac{(O_i-E_i)^2}{E_i}[/itex] on the right side of the above equation follows a normal distribution N(0,1)?

[itex] \frac{ (O_i - E_i)^2}{E_i} [/itex] is nonnegative, so it doesn't follow a normal distribution.

If [itex] X [/itex] is a binomial random variable representing the number of "successes" n independent trials with probability of success [itex] p [/itex] on each trial then the distribution of [itex]Y = \frac {X-np}{\sqrt{np(1-p)}
} [/itex] can be approximated by a [itex] N(0,1) [/itex] distribution.
 
Last edited:
  • Like
Likes 1 person
  • #3
713
5
[itex] [/itex]

If [itex] X [/itex] is a binomial random variable ...

I see. There it is our assumption!
It seems to me that such an assumption automatically implies that the data in the cells of the contingency table are assumed to follow a multinomial distribution.

So in the end, although the formula for calculating the [itex]\chi^2[/itex] value is just an approximation of the Kullback-Leibler divergence, if we are willing to perform a decision test we still need the assumption that we are dealing with a multinomial distribution, otherwise the [itex]\chi^2[/itex] value that we calculated according to the formula above, does not necessarily follow a chi2-distribution.
 

Related Threads on Chi-square test: why does it follow a Chi-square distribution

Replies
4
Views
2K
Replies
1
Views
8K
Replies
2
Views
46K
  • Last Post
Replies
3
Views
2K
  • Last Post
Replies
1
Views
2K
  • Last Post
Replies
8
Views
2K
  • Last Post
Replies
1
Views
2K
  • Last Post
Replies
5
Views
4K
Replies
7
Views
997
Top