Mixed random variables problem

[baseballfan_ny]

Homework Statement

See below. I got parts (a) and (b) but still working on (c)

Relevant Equations

Total probability theorem, Baye's Rule for Mixed variables

I got (a) and (b) but I'm still working on (c). The solutions can be found here for your reference: https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/unit-ii/lecture-9/MIT6_041SCF13_assn05_sol.pdf. But I'm trying my hardest not to look at them :)

My first idea was that since the problem says successive tosses are independent, then the conditional probability of the 2nd toss being heads should just be the probability of the 1st toss being heads... I know that's wrong I'm not sure how to refute it though.

So then I did

Let Yi be the outcome of the ith toss.

$ p_{Y_i|P} = \begin{cases} p & \text{if } y_i = 1 \\ 1 - p & \text{if } y_i = 0 \end{cases} ${pif yi=11−pif

And then by Baye's rule,
$ P(Y_2 = 1 | Y_1 = 1) = \frac {P(Y_2 = 1) P(Y_1 = 1 | Y_2 = 1)} {P(Y_1 = 1)} $

But that doesn't do me too much good, since I believe P(Y2=1) should equal P(Y1=1), and both would simply be the answer to (b) and they would cancel each other, giving me

$ P(Y_2 = 1 | Y_1 = 1) = P(Y_1 = 1) $

Edit: LaTeX is giving me a lot of trouble (or I'm typing something wrong in the commands). Please let me know if you cannot see what I have written. Thanks.

 

[andrewkirk]

In (b) you've worked out the PDF of P conditional on Y1 = 1. Call that PDF $f_{P,1}$.

What is the expected value of $Y_2$ given $P=x$? Whatever expression in x that is (quite easy), integrate that multiplied by $f_{P,1}(x)$ between the appropriate limits, to obtain the conditional probability that $Y_2=1$.

 

[FactChecker]

My first idea was that since the problem says successive tosses are independent, then the conditional probability of the 2nd toss being heads should just be the probability of the 1st toss being heads... I know that's wrong I'm not sure how to refute it though.

Given a known value of the parameter, p, the coin tosses are independent. But p is an unknown random variable and the result of the first coin toss tends to indicate a value of p, which then tends to indicate the result of a second coin toss. So the two coin tosses are not independent.
Consider an extreme case of a very biased coin that only turns up heads once in a million tosses. Suppose you do not know that it is biased. It is true that all the coin toss results are independent in one context, but after a few thousand tails in a row, you will decide that the coin is biased and correctly anticipate that the next toss will also be tails. So the tosses are not independent in the context where the probability parameter is also an unknown value.

 

[baseballfan_ny]

Thank you for the replies!

Given a known value of the parameter, p, the coin tosses are independent. But p is an unknown random variable and the result of the first coin toss tends to indicate a value of p, which then tends to indicate the result of a second coin toss. So the two coin tosses are not independent.
Consider an extreme case of a very biased coin that only turns up heads once in a million tosses. Suppose you do not know that it is biased. It is true that all the coin toss results are independent in one context, but after a few thousand tails in a row, you will decide that the coin is biased and correctly anticipate that the next toss will also be tails. So the tosses are not independent in the context where the probability parameter is also an unknown value.

Ah, that makes sense. The past history of coin tosses indicates the bias of the coin.

In (b) you've worked out the PDF of P conditional on Y1 = 1. Call that PDF $f_{P,1}$.

What is the expected value of $Y_2$ given $P=x$? Whatever expression in x that is (quite easy), integrate that multiplied by $f_{P,1}(x)$ between the appropriate limits, to obtain the conditional probability that $Y_2=1$.

Ok, so $f_{P,1} = \frac {2\pi x(1 + \sin(2\pi x))} {\pi - 1}$

$$ E[Y_2 | P = x] = (1*x) + (0*(1-p)) = x. $$

So then, following your recommendations, $$ f_{Y_2 | Y_1} = \frac {2\pi} {\pi - 1} \int_0^1 x^2(1 + \sin(2\pi x)) \, dx $$

I'm having a bit of trouble grasping how to get here though. What principle should I use to obtain this expression?

 

[andrewkirk]

Just chain-link conditional probabilities:

\begin{align*}
E[Y_2\ |\ Y_1=1] &=
\int_0^1 E[Y_2\ |\ P=x\wedge Y_1=1]\cdot Pr(P=x\wedge Y_1=1 \ |\ Y_1=1)\,dx
\\&=
\int_0^1 x\cdot f_{P,1}(x)\,dx
\end{align*}

 

[baseballfan_ny]

Just chain-link conditional probabilities:

\begin{align*}
E[Y_2\ |\ Y_1=1] &=
\int_0^1 E[Y_2\ |\ P=x\wedge Y_1=1]\cdot Pr(P=x\wedge Y_1=1 \ |\ Y_1=1)\,dx
\\&=
\int_0^1 x\cdot f_{P,1}(x)\,dx
\end{align*}

Oh I think I see that now! So I get that
\begin{align*}

E[Y_2\ |\ Y_1=1] &=

\int_0^1 E[Y_2\ |\ P=x\wedge Y_1=1]\cdot Pr(P=x\wedge Y_1=1 \ |\ Y_1=1)\,dx

\\&=

\int_0^1 x\cdot f_{P,1}(x)\,dx

\end{align*}

Basically, the expectation of $Y_2$ given $Y_1$ is the expectation of $Y_2$ given you choose a coin of bias $x$ that gave you heads on the 1st toss integrated over all the probabilities of x that give you heads. Something like a total expectation theorem but the expectation we're looking for is already a conditional expectation. In that case, isn't the first expectation in your integral really just "the expectation of $Y_2$ given $Y_1 = 1$ given $P = x$ times the probability $P = x$ given $Y_1 = 1$"?

ie something more like...
$$ E[Y_2 | Y_1 = 1] = \int_0^1 E[Y_2 | Y_1 = 1 | P = x] * Pr(P = x | Y_1 = 1) \,dx $$

My other question is why are we allowed to interpret the expectation values as the probabilities? (c) asks for a probability, but the expectation value is giving us the same answer. My guess is that because the value of $Y_2$ can only take on values from 0 to 1, the expectation of $Y_2$ given $Y_1$ is the same as its probability? Ie if we expect $Y_2$ to be 0.6 (which physically won't happen since it's 0 or 1), then we could interpret it as 0.6 times out of 1 we get heads?

 

[andrewkirk]

why are we allowed to interpret the expectation values as the probabilities? (c) asks for a probability, but the expectation value is giving us the same answer. My guess is that because the value of $Y_2$ can only take on values from 0 to 1, the expectation of $Y_2$ given $Y_1$ is the same as its probability? Ie if we expect $Y_2$ to be 0.6 (which physically won't happen since it's 0 or 1), then we could interpret it as 0.6 times out of 1 we get heads?

That's basically it. For any conditioning subset $A$ of the sample space we have:
$$E[Y_2|A] =
\sum_u u\times Pr(Y_2=u|A) =
1\times Pr(Y_2=1|A) + 0\times Pr(Y_2=0|A) =
Pr(Y_2=1|A)$$