Understanding the Uniform Distribution of P-Values in Hypothesis Testing

  • Context: Undergrad 
  • Thread starter Thread starter chowpy
  • Start date Start date
  • Tags Tags
    Distribution P-value
Click For Summary

Discussion Overview

The discussion centers around the concept of p-values in hypothesis testing, specifically addressing the statement that p-values are uniformly distributed between 0 and 1 when the null hypothesis is true. Participants explore the implications of this statement and seek clarification on its meaning and significance in statistical analysis.

Discussion Character

  • Exploratory
  • Technical explanation
  • Conceptual clarification

Main Points Raised

  • One participant seeks clarification on the uniform distribution of p-values as stated in a wiki article, expressing confusion about its implications.
  • Another participant explains that the p-value represents the expected frequency of obtaining the observed data under the null hypothesis, suggesting that if the null hypothesis is true, p-values should be evenly distributed from 0% to 100%.
  • A third participant provides a mathematical explanation of uniform distribution, relating the p-value as a random variable to a test statistic with a continuous distribution, and demonstrating the equivalence of events involving p-values and test statistics.
  • A later reply indicates that the initial question has been resolved, suggesting that the explanation was satisfactory.

Areas of Agreement / Disagreement

Participants generally agree on the interpretation of p-values and their distribution under the null hypothesis, but the initial confusion indicates that some aspects of the concept may require further clarification for different participants.

Contextual Notes

The discussion does not resolve all potential uncertainties regarding the implications of p-value distribution, nor does it address all assumptions related to hypothesis testing and the conditions under which the uniform distribution holds.

Who May Find This Useful

Readers interested in statistical hypothesis testing, p-value interpretation, and the mathematical foundations of statistical distributions may find this discussion relevant.

chowpy
Messages
4
Reaction score
0
I read the following statement from wiki,but I don't know how to get this.

"when a p-value is used as a test statistic for a simple null hypothesis, and the distribution of the test statistic is continuous, then the test statistic (p-value) is uniformly distributed between 0 and 1 if the null hypothesis is true."

anyone can explain it more?
thanks~~
 
Physics news on Phys.org
Hi chowpy, welcome to PF!

Imagine that you have a data set A of one or more experimental observations. You also have a null hypothesis in mind (a possible distribution of results that data set A may or may not have come from). Say you're comparing the means of these two distributions (but it could be any parameter that you're comparing).

The p-value is always defined as the expected frequency of obtaining your actual data set A from the null hypothesis. (If the p-value is incredibly low, we might decide that A came from another distribution, and therefore reject the null hypothesis; that's what hypothesis testing is all about.)

If the null hypothesis is actually true, then we'd expect to get a p-value anywhere from 0% to 100%, distributed evenly. In other words, if the data set A (or a more extreme* data set) would only arise 20% of the time, then we'd expect a p-value of 0.20. *By more extreme I mean a data set with a mean farther away from the mean of the null hypothesis, in the example we're using.

Does this answer your question?
 
Remember what it means for a random variable X to be uniformly distributed on (0,1)

P(X <=a) = a for any a in (0,1)
Let P denote the p-value as a random variable

T stand for a generic Test statistic that has a continuous distribution.

Pick an a in (0,1). Since T has a continuous distribution, there is a number ta that satisfies

<br /> \Pr(T \le ta) = a<br />

Now, the events P \le a and T \le ta are equivalent, so that

<br /> \Pr(P \le a) = \Pr(T \le ta) = a<br />

comparing this to the meaning of "uniformly distributed on (0,1) shows the result.
 
Thanks Mapes and statdad~
I understand it now~
 

Similar threads

  • · Replies 24 ·
Replies
24
Views
7K
  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 43 ·
2
Replies
43
Views
6K
  • · Replies 11 ·
Replies
11
Views
3K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 1 ·
Replies
1
Views
10K
Replies
3
Views
12K
Replies
2
Views
3K