Significance Levels and Probability

ihearyourecho

1. The problem statement, all variables and given/known data

A group of psychologists once measured 77 variables on a sample of schizophrenic people and a sample of people who were not schizophrenic. They compared the two samples using 77 separate significance tests. Two of these tests were significant at the 5% level. Suppose that there is in fact no difference in any of the variables between people who are and people who are not schizophrenic, so that all 77 null hypotheses are true.

A) What is the probability that one specific test shows a difference significant at the 5% level?

B) Why is is not surprising that 2 of the 77 tests were significant at the 5% level?

2. Relevant equations

N/A

3. The attempt at a solution

I know this is a conceptual problem, but that's why I'm not getting it. I can do a problem if there are numbers, but I don't understand the concept of it I guess. If someone could just give me a general direction of where I'm supposed to go/what I'm supposed to do, I think I can get it.

ihearyourecho

I've come up with an answer for both A and B, but I'm not sure if they're right.

A) The probability is 5%.
B) It's not surprising because 5% of 77 is 3.85, so we expect almost 4 observations to be significant at the 5% level. These two observations could be two of those four.

If these are right or if these are wrong, I could still use a little explanation because I think I have a loose grasp on what's going on, but not a strong one.

Thanks :)

statdad

Yes to both - one way (not the only way, but the appropriate way for this set of problems) to think of a significance level is that is the "long term" probability of making a type I error. the point of these two questions is to interpret it that way - and you did.

ihearyourecho

Alrighty, thanks!