Understanding the Uniform Distribution of P-Values in Hypothesis Testing

[chowpy]

I read the following statement from wiki,but I don't know how to get this.

"when a p-value is used as a test statistic for a simple null hypothesis, and the distribution of the test statistic is continuous, then the test statistic (p-value) is uniformly distributed between 0 and 1 if the null hypothesis is true."

anyone can explain it more?
thanks~~

 

[Mapes]

Hi chowpy, welcome to PF!

Imagine that you have a data set A of one or more experimental observations. You also have a null hypothesis in mind (a possible distribution of results that data set A may or may not have come from). Say you're comparing the means of these two distributions (but it could be any parameter that you're comparing).

The p-value is always defined as the expected frequency of obtaining your actual data set A from the null hypothesis. (If the p-value is incredibly low, we might decide that A came from another distribution, and therefore reject the null hypothesis; that's what hypothesis testing is all about.)

If the null hypothesis is actually true, then we'd expect to get a p-value anywhere from 0% to 100%, distributed evenly. In other words, if the data set A (or a more extreme* data set) would only arise 20% of the time, then we'd expect a p-value of 0.20. *By more extreme I mean a data set with a mean farther away from the mean of the null hypothesis, in the example we're using.

Does this answer your question?

 

[statdad]

Remember what it means for a random variable X to be uniformly distributed on (0,1)

P(X <=a) = a for any a in (0,1)
Let P denote the p-value as a random variable

T stand for a generic Test statistic that has a continuous distribution.

Pick an a in (0,1). Since T has a continuous distribution, there is a number ta that satisfies

$$\Pr(T \le ta) = a$$

Now, the events $$P \le a$$ and $$T \le ta$$ are equivalent, so that

$$\Pr(P \le a) = \Pr(T \le ta) = a$$

comparing this to the meaning of "uniformly distributed on (0,1) shows the result.

 

[chowpy]

Thanks Mapes and statdad~
I understand it now~