Understanding the Role of P-Values in Hypothesis Testing

  • Context: Undergrad 
  • Thread starter Thread starter sodium.dioxid
  • Start date Start date
  • Tags Tags
    Value
Click For Summary

Discussion Overview

The discussion centers on the role of p-values in hypothesis testing, particularly regarding their interpretation and implications for the null hypothesis. Participants explore the conceptual understanding of p-values, their significance, and the assumptions underlying their calculation.

Discussion Character

  • Conceptual clarification
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • Some participants question why a lower p-value is considered stronger evidence against the null hypothesis, suggesting that it may indicate a rare event rather than a rejection of the null hypothesis.
  • Others argue that the p-value represents the probability of observing the data assuming the null hypothesis is true, and that a small p-value implies a lower probability of the null hypothesis being true.
  • A participant presents an analogy to argument by contradiction to explain how a small p-value leads to suspicion that the null hypothesis may be false.
  • One participant proposes a specific example involving a car manufacturer's claim about fuel efficiency, suggesting that a low p-value indicates something is wrong with the null hypothesis.
  • Another participant cautions that while the intuitive concept of a low p-value suggests issues with the null hypothesis, it cannot be definitively concluded that the null hypothesis is wrong or that its probability can be quantified.

Areas of Agreement / Disagreement

Participants express differing views on the interpretation of p-values and their implications for the null hypothesis. There is no consensus on the conclusions that can be drawn from low p-values, and the discussion remains unresolved regarding the subjective nature of hypothesis testing.

Contextual Notes

Participants highlight limitations in the interpretation of p-values, including the dependence on the assumption that the null hypothesis is true and the subjective nature of hypothesis testing as a procedure rather than a definitive proof.

sodium.dioxid
Messages
50
Reaction score
0
I don't understand why a lower p value is stronger evidence against the null hypothesis. P value is a probability; so, wouldn't a lower p value mean that your statistic was very lucky (rare)?
 
Physics news on Phys.org
Well yes, the p-value is a probability; but a probability of what? Without answering that, you'll be lost. The p-value of a statistic is the probability that the null hypothesis is true when you reject it. Focus on the fact that it is related to the probability that the null hypothesis is true, so that when p gets small, the probability of the null hypothesis, in a sense, is lowered.
 
Also, yes, a low p-value could mean that your observation was lucky or rare assuming that the null hypothesis is true. However, when your p-value is very small, you tend not to believe that the answer is that the observation was lucky or rare. Because that small p-value is based on the assumption that the null hypothesis was true, if the p-value is very small then you start to suspect that it's not because you were lucky, but because your assumption was wrong: The null hypothesis was false.

It's analogous to an argument by contradiction. Those go: "Assume X is true. But if X is true then we can derive a contradiction. Therefore X must not be true."

The p-value argument is similar: "Assume the null hypothesis. But if the null hypothesis is true, then something extremely unlikely occurs. Therefore the null hypothesis is extremely unlikely and so we reject it."
 
Thanks for the reply. I am still a bit confused. Can we work through an example? Suppose that a car manufacturer claims its cars to run at 30mpg.

Null: u = 30

I take a random sample of 6 cars and find the mpg to be 21mpg; pretend this correlates to a very low p value. The low p value means that we did not expect for it to be such a low mpg, right? Since it was so unlikely for it to be 21mpg and it came out to be 21 mpg nevertheless, there must be something wrong with the null hypothesis. <--- Is my analysis correct?
 
Last edited:
sodium.dioxid said:
there must be something wrong with the null hypothesis. <--- Is my analysis correct?

You have grasped the basic intuitive concept. The technical truth is that you can't deduce that "there must be something wrong with the null hypothesis". You can't say the null hypothesis is probably wrong and you can't state a number that quantifies the probability that the null hypothesis is correct. The statistical method of hypothesis testing is a subjective procedure. It's a procedure, not a proof of something.
 

Similar threads

  • · Replies 24 ·
Replies
24
Views
7K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 10 ·
Replies
10
Views
3K
Replies
20
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
3
Views
12K
  • · Replies 1 ·
Replies
1
Views
10K
  • · Replies 17 ·
Replies
17
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K