Hypothesis Testing - Basic Theory Question

[quarklet]

Homework Statement

Karen asks John to guess the date of her birth, not including the year (i.e. the day and month she was born; discount leap years). If John guessed correctly on his first try, would you believe his claim that he made a lucky guess or would you be suspicious that he already knew your birth date? Explain why, relating to chance and hypothesis testing.

2. The attempt at a solution

The probability of John guessing correctly on his first try is 1 correct day/365 possible days, or 0.0027.

I chose to set the alpha level at 0.05, because none is given in the question.

I know the probability of a correct guess due to chance is less than the alpha level (.0027 < 0.05).

Here is where I get confused. Would my alternative hypothesis be that John is psychic, or a lucky guesser? i.e. What exactly am I proving by the fact that "p-value < alpha?" I feel like this should seem obvious, but I am really frustrated.

Please help guide me in the right direction.

 

[clamtrox]

A word of warning: this might just confuse you more.

So hypothesis A says that John guessed the date and hypothesis B says he knew it. Let's denote the conditional probability that John guessed the date, given that he answered it correctly by P(A|C). This is given by Bayes' theorem,

$$P(A|C) = \frac{1}{N} P(C|A) P(A)$$

where P(A|C) is the conditional probability that A is true, given that you know C (that he guessed right) and P(C|A) is the reversed conditional probability: if you know that John doesn't know the answer, what are the odds of guessing. P(A) is the prior probability you give to the hypothesis and N is a normalization constant that does not depend on the hypothesis.

Now, P(C|A) is just 1/365, and P(A) is probability you assign to the hypothesis. If they are complementary (as in this case) you'd have P(A) + P(B) = 1, etc.

Then take hypothesis B: now you have P(C|B) = 1, so P(B|C) = [1-P(A)]/N. You can compare the two cases easily

$$\frac{P(A|C)}{P(B|C)} = \frac{P(A)}{365 (1-P(A))}.$$

You see that the answer depends on whether or not you think John might have known the answer. If you think that it's completely impossible that John could have known it, you'd assign P(A) = 1 and then you'd have P(B|C) = 0 (since P(B) = 0). If you think there's a one in million chance of John being a psychic, you'd plug in P(A) = 0.999999, and so forth.

Now, in this case you obviously don't have any knowledge on P(A). In this case, the usual assumption is just that the hypotheses are equally likely, P(A) = P(B) = 1/2. In that case, you'd end up with P(A|C) = 0.0027 P(B|C), which is clearly very significant no matter what alpha you choose.