Probability: Flawed Assumptions in Picking M&M's

[bpet]

No. that is P(R|H_i), the probability of getting a red assuming hypothesis i. P(R|H_i,E) is the conditional probability of getting a red assuming hypothesis i and given the evidence and given the empirical priors:

Agreed, and this is i/5 (the probability that 11th is red given first 10 not red and i reds altogether).

P(R|H_i,E) = P(R|H_i)P(H_i|E)

Not true (e.g. by a Venn diagram argument). Perhaps you meant to write

P(R,H_i|E) = P(R|H_i,E)P(H_i|E)

so that

P(R|E) = \sum_i P(R,H_i|E)

 

[Cexy]

<br /> \begin{tabular}{clllll}<br /> $ i $ &amp; $ P(H_i) $ &amp; $ P(E|H_i) $ &amp; $ P(H_i)P(E|H_i) $ &amp; $ P(H_i|E)\qquad $ &amp; $ P(R|H_i,E) $ \\<br /> \hline<br /> 0 &amp; 0.24 &amp; 1 &amp; 0.24 &amp; 0.64070834 &amp; 0 \\<br /> 1 &amp; 0.33 &amp; 0.33333333 &amp; 0.11 &amp; 0.29365799 &amp; 0.05873160 \\<br /> 2 &amp; 0.23 &amp; 0.09523810 &amp; 0.02190476 &amp; 0.05847735 &amp; 0.02339094 \\<br /> 3 &amp; 0.11 &amp; 0.02197802 &amp; 0.00241758 &amp; 0.00645402 &amp; 0.00387241 \\<br /> 4 &amp; 0.07 &amp; 0.00366300 &amp; 0.00025641 &amp; 0.00068452 &amp; 0.00054761 \\<br /> 5 &amp; 0.02 &amp; 0.00033300 &amp; 0.00000666 &amp; 0.00001778 &amp; 0.00001778 \\<br /> \hline<br /> Total &amp;&amp;&amp; 0.37458541 &amp; 1 &amp; 0.08656034 \\<br /> \end{tabular}

Do you have a mistake in calculating the final probabiity P(R|E)? It seems as though you've calculated

P(R|E) = \sum_i P(R|Hi, E)

whereas you should really be calculating

P(R|E) = \sum_i P(R|Hi,E) P(Hi|E)

It seems intuitively obvious (and I'm pretty sure I have a proof) that if we let p be the average proportion of red candies in each bag, Y be the number of red candies in the first n draws and R be the event "the next candy is red" then

P(R | Y=m) = p

i.e. the probability that the next candy is red is independent of what colors the first m candies were. Note that if p=10% and we've drawn 14 out of the 15 candies in the bag, this tells us that the hypothesis "all the candies are green" is 9 times more likely than the hypothesis "all but one of the candies are green, and we've chosen all the green ones so far", which seems reasonable.

 

[Jobrag]

It would be interesting to know what questions a and b are, the questions and answers might give sufficient information as to whether the bag contained a fixed ratio of sweets to start with just a random selection.

 

[Eero]

I think, things are very clear,actually.

Lets denote the number of candies in a bag by N, the number of red candies by m and the probability that the first red will appear at k-th draw by P(k).
Because we have here a sampling without replacement, the following holds:

P(k+1)/P(k)=(N+1-k-m)/(N-k)=1-(m-1)/(N-k)

So P(k+1)<P(k). The probability decreases in any case until m>1.
In case of m=1 the probability remains the same.

There is no need any reference to Bayes' Theorem or to the particularity of the production of M&M's etc.

 

[D H]

P(R|H_i,E) is the conditional probability of getting a red assuming hypothesis i and given the evidence and given the empirical priors:

P(R|H_i,E) = P(R|H_i)P(H_i|E)

Not true (e.g. by a Venn diagram argument). Perhaps you meant to write

P(R,H_i|E) = P(R|H_i,E)P(H_i|E)

so that

P(R|E) = \sum_i P(R,H_i|E)

Do you have a mistake in calculating the final probabiity P(R|E)? It seems as though you've calculated

P(R|E) = \sum_i P(R|Hi, E)

whereas you should really be calculating

P(R|E) = \sum_i P(R|Hi,E) P(Hi|E)

No and no.

What I should be calculating is exactly what is in that table.

Suppose you have partitioned the sample space Ω into subsets {A_i} such that
1. Each A_i is a subset of Ω,
2. The union of all A_i is Ω, and
3. The intersection of any two members of {A_i} is the empty set.
Then the probability of some event B is

P(A) = \sum_i P(A|B_i)P(B_i)

This is the total probability theorem. See http://www.cs.cornell.edu/courses/cs280/2004sp/probability3.pdf (Note: This reference also discusses Bayes' Theorem.)

In the problem at hand, we are randomly drawing one M&M from a partially emptied bag of candies that contains five candies of which zero to five are red. There are six mutually exclusive possibilities: The contents of the bag now include exactly zero (A₀), one (A₁), two (A₂), three (A₃), four (A₄), of five (A₅) red candies. Because ∪A_i=Ω and A_i∩A_j ∀ i≠j, this a partition of the sample space. Thus the total probability that the next candy drawn from the bag is red is

P(R) = \sum_{i=0}^5 P(R|A_i)P(A_i)

The conditional probabilities P(R|A_i) are easily calculated: They are i/5. The only remaining issue is to calculate the probabilities P(A_i). This has already been done: The conditional probabilities P(H_i|E) are the best guesses available regarding the probabilities P(A_i). Thus

<br /> P(R) =<br /> \sum_{i=0}^5 P(R|A_i)P(A_i) =<br /> \sum_{i=0}^5 P(R|H_i\,\text{and}\,E)P(H_i|E)<br /> \,\,\text{where}\,P(R|H_i\,\text{and}\,E)=i/5

It is often handy to give those somewhat awkward P(R|H_i and E)P(H_i|E) a name. In many texts and articles this name is P(R|H_i,E).

 

[bpet]

...It is often handy to give those somewhat awkward P(R|H_i and E)P(H_i|E) a name. In many texts and articles this name is P(R|H_i,E).

Ok thanks for taking the time to explain that subtle point about the notation. It's unfortunate that the literature has chosen this convention when we have P(R|H_i \mbox{ and } E)P(H_i|E) = P(R \mbox{ and } H_i|E)

 

[diggy]

The bag is superfluous in this problem.

So yes, original-poster, you are specifically correct, every time you don't find a red, your chance of drawing a red from a finite set that has a finite set of reds increases.

This is not the same as a gambler's fallacy, where the picks are reset every time (like a coin or a roulette wheel).

But your teacher is big-picture correct that the difference is so small, that it is effectively zero.

{{Another way of thinking about the bag being meaningless (if that bothers you) is that every time it was filled 1 M&M at a time, the odds of what the next M&M would be changed. So as its filled it becomes more and more improbable that the bag will be filled with all non-red M&M's. But this is the same as you picking directly from the factory set, not the bag.}}

 

[bpet]

...There is no need any reference to Bayes' Theorem or to the particularity of the production of M&M's etc.

The bag is superfluous in this problem...

That was my first thought as well, but it's incorrect. See e.g. posts #23 and #28 to see why the mixing model does matter.

Edit: see also posts #35, 36 regarding the difference in notation between D H's table and mine (our posts are too old to edit now).

 

[D H]

The bag is superfluous in this problem.

So yes, original-poster, you are specifically correct, every time you don't find a red, your chance of drawing a red from a finite set that has a finite set of reds increases.

Try reading the replies. The bag is anything but superfluous in this problem. It is central to it.

 

[diggy]

Try reading the replies. The bag is anything but superfluous in this problem. It is central to it.

Which post are you considering the correct interpretation?

 

[diggy]

Just to simplify life -- assume the factory that made (10) M&M's, (1) of which was red. They are put in (2) bags with (5) M&M's in each. You eat your first M&M and its blue.

I assume we all agree on the next M&M being more likely to be red, yes?

 

[CRGreathouse]

Just to simplify life -- assume the factory that made (10) M&M's, (1) of which was red. They are put in (2) bags with (5) M&M's in each. You eat your first M&M and its blue.

Alternate assumption: there are 3486784401 bags with 0 red M&Ms in them, 3874204890 bags with 1 red M&M in them, 1937102445 bags with 2 red M&Ms in them, 573956280 bags with red 3 M&Ms in them, 111602610 bags with 4 red M&Ms in them, 14880348 bags with 5 red M&Ms in them, 1377810 bags with 6 red M&Ms in them, 87480 bags with 7 red M&Ms in them, 3645 bags with 8 red M&Ms in them, 90 bags with 9 red M&Ms in them, and 1 bag with red 10 M&Ms in it. (All bags have 10 M&Ms in total.) You pick a bag and eat the first M&M and it's blue. Is the second one more likely to be red?

 

[diggy]

Alternate assumption: there are 3486784401 bags with 0 red M&Ms in them, 3874204890 bags with 1 red M&M in them, 1937102445 bags with 2 red M&Ms in them, 573956280 bags with red 3 M&Ms in them, 111602610 bags with 4 red M&Ms in them, 14880348 bags with 5 red M&Ms in them, 1377810 bags with 6 red M&Ms in them, 87480 bags with 7 red M&Ms in them, 3645 bags with 8 red M&Ms in them, 90 bags with 9 red M&Ms in them, and 1 bag with red 10 M&Ms in it. (All bags have 10 M&Ms in total.) You eat the first M&M and it's blue. Is the second one more likely to be red?

Yes.

 

[bpet]

Just to simplify life -- assume the factory that made (10) M&M's, (1) of which was red. They are put in (2) bags with (5) M&M's in each. You eat your first M&M and its blue.

I assume we all agree on the next M&M being more likely to be red, yes?

Now try the case where the factory makes 20 M&M's of which 2 are red. There are two possibilities: (a) the 2 reds go into one bag or (b) the 2 reds go into separate bags.

Without knowing how the bagging machine works (i.e the probabilities of (a) vs (b)), we've got a kind of Schrödinger's cat scenario where the probability of next being red can either increase or decrease.

 

[D H]

Yes.

Try again.

Before you open the bag, what can you say about the probability of getting a red M&M as the first candy from that bag? (No peeking now!) Does opening the bag but not peeking inside change that number one iota?

Hint: Per CRGreathouse's setup, there are 10 billion bags each containing 10 candies (100 billion candies total). Of these 100 billion candies, 10 billion are red.



Now you draw a blue candy as the first candy from that bag. What is the probability the next candy is red?

 

[diggy]

Ah -- so you are saying *if the bags aren't randomly filled (in the i.i.d. sense)* then it's possible that the likelihood of drawing a red goes down once you draw a blue. Yes totally agree.

Edit:
Sorry I thought you were just getting caught up in the bayesian inversion and not accounting for the density of red states increasing as you filtered out the blue-rich bags.