Understanding Statistical Significance: Explaining the 0.05 Level

[jlo2006]

Heres the problem:

When asked to explain the meaning of "statistically significant at the 0.05 level," a student says, "This means there is only probability 0.05 that the null hypothesis is true." Is this an essentially correct explanation of statistical significance? Explain.


Help please. Appreciated.

 

[erik_f]

More specifically, it is the probability that you reject a true null hypothesis.

The difference in the statements is subtle.

The probability of the null hypothesis being true may be 20%, 50%, etc. -- but the probability of rejecting it is .05.

 

[0rthodontist]

Well, you can't say anything about the probability of the null hypothesis being 20% or 50% right off the bat because probability is based on the amount of knowledge you have. For one person the probability of the null hypothesis being true might be 20% and for another person with more knowledge the probability of the same hypothesis being true might be 1.

But if the probability of a type I error is the probability that the null hypothesis is true and you reject it, then let A = the null hypothesis is true and let B = you reject the null hypothesis.
Then P(A n B) is the probability of a type I error, and P(A n B) = .05
But you know that P(B) = 1 since the test was significant. So P(A n B) = P(A) = .05 and the probability that the null hypothesis is true is .05.

 

[EnumaElish]

Is this homework? If it is, it is not supposed to have been posted here. If it is not HW, why did you post it under HW help?