Determining the fairness of a penny

[caffeine]

I've been reviewing probability and came across this problem:

A penny may be fair or it may have two heads. It's tossed n times and it comes up heads on each occaision. If our initial judgement was that both options for the coin ("fair" or "both sides heads") were equally probable, what is our revised judgement in light of the data?
​

The book's answer is P(fair)$= \frac{1}{1+2^n}$, but I don't understand this. Certainly, the book's answer makes sense for n=0 (before we toss the coin).

Suppose we toss the coin once and it comes up heads. The probability for this outcome is 1/2. According to the book's answer, P(fair)=1/3.

I guess what I'm having trouble with is connecting the probability for an event to happen (say, tossing n heads) with the probability that the coin is fair.

What is the connection between these two probabilities? How is the book's answer justified?

Thanks!

 

[EnumaElish]

After stating that "our initial judgement was that both options for the coin ("fair" or "both sides heads") were equally probable" why do you say "the book's answer makes sense for n=0 (before we toss the coin)"? It makes perfect sense -- the prior probability of having a fair coin is 1/2 and that's what the formula equals when n = 0.

Does this help?

Anyway, is this homework?

 

[RandallB]

why do you say "the book's answer makes sense for n=0 (before we toss the coin)"?

It makes perfect sense -- the prior probability of having a fair coin is 1/2 and that's what the formula equals when n = 0.

I think you may have missed the point of his question.
His statement you quoted and questioned agrees completely with your explanation.

Caffine try this.
What are the odds of the fair coin NOT showing Tails ---- and the same for the the unfair coin.
Now total the odds for NOT showing Tails and divide that into the odds for just the fair coin.

Do the same for two flips.
How does it compare to what the book said?
RB

 

[caffeine]

After stating that "our initial judgement was that both options for the coin ("fair" or "both sides heads") were equally probable" why do you say "the book's answer makes sense for n=0 (before we toss the coin)"?

Because it DOES make sense. For n=0, the book's answer gives P(fair)=.5 and P(unfair)=.5. That's what we mean by "equally likely". So, as I stated, the book's answer makes sense for n=0.

It makes perfect sense -- the prior probability of having a fair coin is 1/2 and that's what the formula equals when n = 0.
Does this help?

Unfortunately not because you've stated what I have already stated:

I said: The book's answer makes sense for n=0.

You said: It makes perfect sense for n=0..

Are we not saying the same thing? My question was: How to relate the probability between the event of tossing n heads with the probability that the coin is fair. Since I already said the book's answer makes sense for n=0, I was actually interested in n>0.

Anyway, is this homework?

Did I not say that I'm reviewing probability?

But to answer your question explicitly, no. It's not a homework question. I've finished my degree. I'm not even in school any longer. We didn't even *have* homework past the 2nd year of grad school.

Also, the rules specifically say to NOT post homework questions on the forum, so why would I do so?

Probability has always been a weak point for me, and it shouldn't be. So, as I stated in my post, I'm reviewing it.

 

[caffeine]

I think you may have missed the point of his question.
His statement you quoted and questioned agrees completely with your explanation.
Caffine try this.
What are the odds of the fair coin NOT showing Tails ---- and the same for the the unfair coin.
Now total the odds for NOT showing Tails and divide that into the odds for just the fair coin.
Do the same for two flips.
How does it compare to what the book said?
RB

Wow. OK, so the sample space you're using is:

A: Fair coin shows heads
B: Unfair coin shows heads

and then, the next logical step would be:

P(fair coin) = P(A) / ( P(A) + P(B) )

and you can do this because our entire sample space consists of events where the coin shows heads. That's very nice.

Now I have to figure out why that stymied me in the first place. How did you think of this problem? Can you tell me anything about the thought process that lead you to the answer? Did you think about partitioning the sample space? How did you approach it?

I know that's a hard question, but even more than learning probability, this is exactly the kind of thing I need to get better at when approaching probability problems.

Thanks!

 

[RandallB]

Wow. OK, so the sample space you're using is:
A: Fair coin shows heads
B: Unfair coin shows heads

because our entire sample space consists of events where the coin shows heads.

Now I have to figure out why that stymied me in the first place.
How did you think of this problem?

Not like I thought of it, first time I’ve ever seen it.
The key of course is understanding the problem and both you’re A: B: statement and your ‘because’ statement do that.

Getting there for me, I need to visualize a problem which I find you generally cannot do satisfactorily until you understand it. And understanding correctly is more than half the solution in most cases.
Cranking down though formulas seems so unsatisfying (did I reach the answer by accident) without visualizing what they mean somehow.
Even more so when I cannot “see” how an answer or theory is reached.
I suppose that’s why I’m so uncomfortable with QM which no one claims to really “see”.

Anyway for me the key here was “Not Tails”. Just to focus the visualization on what the real sample space was.
RB

 

[D H]

This problem is a simple application of Bayes' Theorem:

$$P(H_i|E) = \frac{P(E|H_i)P(H_i)}{\sum_j P(E|H_j)P(H_j)}$$

where

$H_i$ is the $i^{th}$ hypothesis explaining some event $E$.
$E$ is the observed event (e.g., flipping $n$ heads in a row).
$P(H_i|E)$ is the posterior probability of $i^{th}$ hypothesis.
$P(H_i)$ is the prior probability of $i^{th}$ hypothesis.
$P(E|H_i)$ is the probability of observing event $E$ assuming the $i^{th}$ hypothesis is true.

The sum is taken over all hypotheses. There are two hypotheses in this problem: $H_1$: coin is fair and $H_2$coin has two heads.

Four items are needed to evaluate the probability a coin is fair after flipping $n$ heads in a row: the two prior probabilities $P(H_i)$ and the probabilities of observing the event given each of the two hypotheses. The prior probabilities are based on the principle of ignorance: all explanations are equally likely, so $P(H_1)=P(H_2)=1/2$.

What are $P(E|H_i)$? These are the probabilities of getting $n$ heads in a row with a fair coin and a two-headed coin, which are $\frac{1}{2^n}$ and $1$ respectively.

Applying Bayes' Theorem to compute the probability that the coin is fair after flipping $n$ heads in a row,
$$\begin{align*} P(H_1|E) &= \frac{\frac{1}{2^n}\frac 1 2} {\frac{1}{2^n}\frac 1 2 + 1 \frac 1 2} \\ &= \frac{1}{1+2^n} \end{align*}$$

 

[EnumaElish]

Believe it or not, I read "Certainly, the book's answer makes sense for n=0" as "the book's answer makes no sense for n=0".

 

[Hurkyl]

I'm going to be rather bold, and claim that every "real-life" probability question is conditional.

Whether or not that's true, it's often the piece people are missing when they're confused by a question like this!


What you have originally is:

P(fair coin | my initial judgement) = 0.5

and you're eventually interested in calculating

P(fair coin | my initial judgement and the result of the experiment)

and to do that, you'll probably want to use

P(this experimental outcome | fair coin)

and

P(this experimental outcome | double-headed coin)

(and the fact that these probabilities are independent of your judgement)

 

[D H]

I'm going to be rather bold, and claim that every "real-life" probability question is conditional.

That claim makes you a strict Bayesianist, a position against which frequentists strongly disagree.

 

[D H]

All of the solutions to the "is the penny fair" problem (including mine) have been Bayesian in nature. A frequentist solution to this problem that avoids the prior probabilities also exists.

There are $2^n$ possible outcomes from a sequence of $n$ coin tosses of a fair coin. Of these, only one has all heads. Now think of marking our two-headed coin in some way to distinguish the two sides. There are similarly $2^n$ possible outcomes from a sequence of $n$ coin tosses of this marked two-headed coin, but all of the outcomes are all heads.

Thus the universe consisting of a sequence of $n$ successive heads in a coin-tossing experiment comprises $2^n+1$ events: $2^n$ from the two-headed coin and $1$ from the fair coin. Proceeding from the principle of ignorance, all of these events are equally likely, and thus the odds the coin is fair is $1/(2^n+1)$.

 

[Hurkyl]

Proceeding from the principle of ignorance, all of these events are equally likely

The thing is, I consider this to be a specification of prior probabilities, so I'm unconvinced that this is avoids using such things.

(I take this approach because thinking in this way solved some interpretational issues for me, and makes it easier for me to set up a problem!)


Of course, the original problem specified that there were prior probabilities to use.

 

[D H]

I was playing devil's advocate. I use whatever works, and often, a Bayesian approach works quite nicely. Maximum likelihood (a frequentist method of choice) is a poor cousin to Bayes' Law in my view.

An added plus is that Bayes' Law is so ammenable to variations. Suppose the original problem was phrased as "a coin stamping machine on occasion produces double-headed coins. How many successive tosses of heads would lead you to believe a coin produced by this machine has two heads?" Answering that is a snap with Bayes' Law.

 

[EnumaElish]

There is usually something counterintuitive in a Bayesian formula. When I try to write D H's explanation as P(H|E) = P(H and E)/P(E), it leads to the conclusion that P(E) = 1 + 1/2ⁿ > 1.

That's because we know P(H|E) = 1/(2ⁿ+1) and P(H and E) = 1/2ⁿ.

I have got to be missing something, what?

 

[Hurkyl]

That's because we know ... P(H and E) = 1/2ⁿ.

Why do we know that?

(I suspect that you mistakenly interpreted P(H and E) to mean P(E|H). At least, that's a mistake I make every now and then)