A Coin flipping problem (Markov chain)

[WMDhamnekar]

TL;DR Summary

Suppose that coin 1 has probability 0.7 of coming up heads, and coin 2 has probability 0.6 of coming up heads.
If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, and if it comes up tails, then we select coin 2 to flip tomorrow.
If the coin initially flipped is equally likely to be coin 1 or coin 2, then

a)what is the probability that the coin flipped on the third day after the initial flip is coin 1?

b)Suppose that the coin flipped on Monday comes up heads. What is the probability that the coin flipped on Friday of the same week also comes up heads?

My attempt to answer this question:

 

[Office_Shredder]

The probability of getting a heads on Friday has to be at least 0.6 (why? You can actually conclude 0.65 before doing any real math) so I think you've done it wrong.

My suggestion is to track as your state which coin you are flipping, and to compute the probability of getting a heads on a step, just do that from the game state as a second calculation.

 

[FactChecker]

EDIT: I have changed my mind about the need for more than two states.
My simple brain can not follow your logic. You don't seem to have enough states. I think that each state consists of two parts: What coin is flipped? What is the result?
So I would start with 4 states (C1,H), (C1,T), (C2,H), (C2,T), and go from there. This might reduce to what you have, but I don't see it immediately.

EDIT: A better and more direct set of states is: (Coin 1 before flip), (Coin 1 flip H), (Coin 1 flip T), (Coin 2 before flip), (Coin 2 flip H), (Coin 2 flip T).

 

[pbuk]

Why have you posted an image of your workings? This makes it harder to reply, we have to type everything from scratch.

You start to go wrong in your third sentence: you should not be adding together probabilities for different initial states, this is the job of the transition matrix. Before you write the transition matrix you need to work out what states you are transitioning between. I would choose state 1 = coin 1, state 2 = coin 2 (although it doesn't matter if you choose state 1 = heads, state 2 = tails).

So the first entry in the transition matrix is the probability of transitioning from coin 1 to coin 1: this is the probability of tossing heads on coin 1 = 0.7.

 

[pbuk]

So I would start with 4 states (C1,H), (C1,T), (C2,H), (C2,T), and go from there. This might reduce to what you have, but I don't see it immediately.

Not necessary, the matrix still only has 2 non-zero values in each row and column because e.g. you cannot transition from (C1, T) to (C1, X).

 

[FactChecker]

Not necessary, the matrix still only has 2 non-zero values in each row and column because e.g. you cannot transition from (C1, T) to (C1, X).

Yes, but it doesn't hurt to keep it simple. I would prefer a more methodical development with simple, obvious, steps than trying to compress the process. To each his own.

 

[WMDhamnekar]

Yes, but it doesn't hurt to keep it simple. I would prefer a more methodical development with simple, obvious, steps than trying to compress the process. To each his own.

I think the correct transition matrix is $P = \begin{array}{c c} & \begin{array}{c c } coin1 & coin2 \\ \end{array} \\ \begin{array}{c c } coin 1 \\ coin 2 \end{array} & \begin{bmatrix} 0.65 & 0.35 \\ 0.65 & 0.35 \end{bmatrix} \end{array}$

So, the answer to a) The probability that the coin flipped on third day is coin1 given that the initial flip is coin 1 = 0.65 $P^2 = \begin{bmatrix} \underline{0.65} & 0.35 \\ 0.65 & 0.35 \end {bmatrix}$

Answer to b): The probability that the coin flipped on Friday of the same week also comes up head given that the coin flipped on monday comes up head = 0.65 $P^4 = \begin{bmatrix} \underline{0.65} & 0.35 \\ 0.65 & 0.65 \end {bmatrix}$

 

[pbuk]

That is not correct - it isn't even a transition matrix: the row labels and column labels should be the same.

The first entry should be the probability that if the current coin to be thrown is coin 1 the next coin to be thrown will be coin 1.

Also the answers are not transition matrices, they are probabilities.

 

[WMDhamnekar]

That is not correct - it isn't even a transition matrix: the row labels and column labels should be the same.

The first entry should be the probability that if the current coin to be thrown is coin 1 the next coin to be thrown will be coin 1.

Also the answers are not transition matrices, they are probabilities.

I edited my answer. I think my answer in #7 is fair enough.

 

[Office_Shredder]

I still think you haven't written down the correct probability that if you flip coin 1, your next flip is also coin 1.

 

[FactChecker]

I edited my answer. I think my answer in #7 is fair enough.

I think the correct transition matrix is $P = \begin{array}{c c} & \begin{array}{c c } coin1 & coin2 \\ \end{array} \\ \begin{array}{c c } coin 1 \\ coin 2 \end{array} & \begin{bmatrix} 0.65 & 0.35 \\ 0.65 & 0.35 \end{bmatrix} \end{array}$

But the problem states that if you are in state "Coin 2" then the probability of flipping a head and going to state "Coin 1" is 0.6. That is not the number you have in the table. You do not have a transition matrix between the states that you indicate.
EDIT I suggest a simple and direct set of states (I will edit my post #3 accordingly):
(Coin 1 before flip), (Coin 1 flip H), (Coin 1 flip T), (Coin 2 before flip), (Coin 2 flip H), (Coin 2 flip T).

 

[pbuk]

I suggest a simple and direct set of states (I will edit my post #3 accordingly):
(Coin 1 before flip), (Coin 1 flip H), (Coin 1 flip T), (Coin 2 before flip), (Coin 2 flip H), (Coin 2 flip T).

I do not understand why you think that introducing 6 states where 2 are sufficient is a simplification. It is not, it is an unnecessary complication and one that results in workings that will not relate well to what the OP has been taught.

The selection of coin1 and coin2 as the states is perfectly correct, but the probabilities are all wrong as @Orodruin has said.

 

[Orodruin]

The selection of coin1 and coin2 as the states is perfectly correct, but the probabilities are all wrong as @Orodruin has said.

I think you mean @Office_Shredder , this is my first post in this thread.

 

[FactChecker]

I do not understand why you think that introducing 6 states where 2 are sufficient is a simplification. It is not, it is an unnecessary complication and one that results in workings that will not relate well to what the OP has been taught.

I guess I agree with you. I was confused about what steps he was combining.

The OP should show a simple transition matrix, with the probabilities stated in the problem (NO CALCULATION NEEDED). Then he should apply that matrix to the initial distribution of [0.5, 0.5] to get the probabilities after the first toss. Keep applying the matrix to the intermediate probabilities until the answer for the third toss is obtained.

 

[pbuk]

I think you mean @Office_Shredder , this is my first post in this thread.

Yes, sorry to both of you

 

[WMDhamnekar]

But the problem states that if you are in state "Coin 2" then the probability of flipping a head and going to state "Coin 1" is 0.6. That is not the number you have in the table. You do not have a transition matrix between the states that you indicate.
EDIT I suggest a simple and direct set of states (I will edit my post #3 accordingly):
(Coin 1 before flip), (Coin 1 flip H), (Coin 1 flip T), (Coin 2 before flip), (Coin 2 flip H), (Coin 2 flip T).

I understood what did you mean to say? That's why I computed the probability of head on flipping of any one of the coins (1 or 2).

My method to answer this question:

The probability of head on flipping anyone of the coin 1 or 2, both of which are equally probable, is $\frac12 \times 0.7 + \frac12 \times 0.6 = 0.65$

If on the flipping of the coin 1 or 2 , head comes up with probability 0.65, we select to flip coin 1 tomorrow with probability 0.65 otherwise we select coin 2 tomorrow with probability 0.35.

Your method to answer this question:
Now, as per your suggestion, Transition probability of matrix $P =\begin{array}{c c} & \begin{array}{c c} coin 1 & coin 2 \\ \end{array} \\ \begin{array} {c c }coin 1\\ coin 2 \end{array} & \begin{bmatrix} 0.7 & 0.3 \\ 0.6 & 0.4 \end{bmatrix} \end{array}$

Answer to a) as per your suggestion using P, the third day means a day after tomorrow. Hence the answer is $P^2 = \begin{array}{c c} & \begin{array}{c c} coin 1 & coin 2 \\ \end{array} \\ \begin{array} {c c} coin 1 \\ coin 2 \end{array} & \begin{bmatrix} \underline{0.67} & 0.33 \\ 0.66 & 0.34 \end{bmatrix} \end{array}$

Answer to b) as per your suggestion using P, is $P^4 = \begin{array}{c c} & \begin{array}{c c} H & T \\ \end{array} \\ \begin{array} {c c} H \\ T \end{array} & \begin{bmatrix} \underline{0.65} & 0.35 \\ \underline{0.65} & 0.35 \end {bmatrix}\end{array}$

Answers to both the questions do not differ much using any of the method given above.

 

[FactChecker]

Because matrix multiplication is associative, there are many ways that the individual multiplications can be grouped. I just don't think that it helps to group them. If you do, you need to make sure that you keep things straight. For instance, your multiplication of the transition matrix and the initial probabilities gives only a vector of probabilities, not a matrix. You show a matrix and, I think that you have a misconception about what it represents.

 

[Orodruin]

both of which are equally probable

The coins are not equally probable (except the first day). Since both coins favor heads, it is clear that coin 1 will be flipped more frequently.

 

[WMDhamnekar]

In this problem, your answer is close to the correct answer, but it is definitely wrong. In other problems, your method might give very different answers from the correct ones.

While answering the question b), my method is useful because question is asking what is the probability of coming up head on the flipping of a coin on friday if on monday, coin flip shows up head. Here the states are not coin 1 and coin 2 but Heads and Tails.

So, on monday, the probability of Head on coin flipping is 0.65, not 0.7. Because, on any coin flipping(coin 1 or coin 2), the probability of head on initial (Monday) coin flipping is 0.65.

Please read my answer to b) in my post #16.

 

[FactChecker]

You will have problems later on if you do not routinely distinguish between the transition matrix and the initial probability distribution and do not keep them separate. When you do multiply them, you must recognize that the result is a vector of probabilities, not a transition matrix.

 

[Orodruin]

While answering the question b), my method is useful because question is asking what is the probability of coming up head on the flipping of a coin on friday if on monday, coin flip shows up head. Here the states are not coin 1 and coin 2 but Heads and Tails.

Those states are absolutely equivalent. Using heads and tails is just mildly confusing for the intial state.

Answer to b) as per your suggestion using P, is P4=HTHT[0.65―0.350.65―0.35]

No, it is not. You have calculated the exponential incorrectly.

Answers to both the questions do not differ much using any of the method given above.

The answers differ a lot considering they could obviously only be in the range 0.6-0.7.

 

[pbuk]

The answer to part (a) is not 0.65.

Spoiler

If the initial state vector is $\tau_0$ and the transition matrix is $P$ the state vector for the coin to be thrown on the 1st day after the initial throw is $\tau_0 P$ and so the state vector for the coin to be thrown on the 3rd day after the initial throw is $\tau_0 P^3$. Your answer is a single entry in that matrix element of that state vector and it should be given exactly (it has 4 fractional digits).

The answer to part (b) is not 0.X (I'm not sure what you think it is).

Spoiler

If the coin flipped on Monday comes up heads the coin flipped on Tuesday will be coin 1 so the initial state vector is $\tau_{tue} = (1, 0)$. The the state vector for the coin to be flipped on Wednesday is $\tau_{tue} P^1$ and so the state vector for the coin to be thrown on the Friday is $\tau_{fri} = \tau_{tue} P^3$. The probability that this coin turns up heads is $\tau_{fri} \begin{pmatrix} 0.7 \\ 0.6 \end{pmatrix}$ which is a scalar and should be given exactly (it has 4 digits after the decimal point).

 

[pbuk]

The answers differ a lot considering they could obviously only be in the range 0.6-0.7.

Yes, the answers should be given exactly.

 

[PeroK]

By diagonalising the transition matrix, $T$, you can obtain a simple expression for $T^nv$ where $v$ is the transpose of $[\frac 1 2, \frac 1 2]$.

Apologies if Markov things are normally done with post matrix multiplication, but I find it easier to do things the usual way.

PS and for $v = [1,0]^T$ for part b).

 

[WMDhamnekar]

Answer to a) is $\begin{bmatrix}0.7 & 0.3 \end{bmatrix} \times P^3 = \begin{array}{c c} & \begin{array}{c c} coin 1 & coin 2 \\ \end{array} \\ \begin{array}{c c} coin 1 \\ coin 2 \end{array} & \begin{bmatrix} 0.667 & 0.333 \\ 0.666 & 0.334 \end{bmatrix}\end{array} = \begin{bmatrix} 0.6667 & 0.3333 \end{bmatrix}$
Hence, the probability that the coin flipped on the third day is coin 1 after the initial flip is coin 1 is 0.6667Looking at the great unfavourable response to my answer to question b) I changed my amswer to b) as follows:
$\begin{bmatrix} 1 & 0 \end{bmatrix} \times P^3 = \begin{array}{c c} & \begin{array}{c c} Head & Tail \\ \end{array} \\ \begin{array}{c c} Head \\ Tail \end{array} & \begin{bmatrix} 0.667 & 0.333 \\ 0.666 & 0.334 \end{bmatrix}\end{array} = \begin{bmatrix} 0.667 & 0.333 \end{bmatrix}$
Hence, The probability that the coin flipped on Friday of the same week also comes up head given that the coin flipped on monday comes up head = 0.667

Are the above answers correct? I can't verify them because author has not given answers to both questions( a & b ).

 

[pbuk]

Neither of those answers is correct; compare them with the hints given in #22.

 

[FactChecker]

By diagonalising the transition matrix, $T$, you can obtain a simple expression for $T^nv$ where $v$ is the transpose of $[\frac 1 2, \frac 1 2]$.

And @WMDhamnekar should be aware that fantastic properties like this make it important to keep the transition matrix and the initial probability vector separate.

Apologies if Markov things are normally done with post matrix multiplication, but I find it easier to do things the usual way.

Good advice. It's wise to do this the way that makes it easiest to keep things straight. I was confused for a while because I had my multiplication mixed up.

 

[WMDhamnekar]

Neither of those answers is correct; compare them with the hints given in #22.

I corrected my answers in my post#25. Please, have a look at those answers.

 

[FactChecker]

Answer to a) is $\begin{bmatrix}0.7 & 0.3 \end{bmatrix} \times P^3 = \begin{array}{c c} & \begin{array}{c c} coin 1 & coin 2 \\ \end{array} \\ \begin{array}{c c} coin 1 \\ coin 2 \end{array} & \begin{bmatrix} 0.667 & 0.333 \\ 0.666 & 0.334 \end{bmatrix}\end{array} = \begin{bmatrix} 0.6667 & 0.3333 \end{bmatrix}$
Hence, the probability that the coin flipped on the third day is coin 1 after the initial flip is coin 1 is 0.6667Looking at the great unfavourable response to my answer to question b) I changed my amswer to b) as follows:
$\begin{bmatrix} 1 & 0 \end{bmatrix} \times P^3 = \begin{array}{c c} & \begin{array}{c c} Head & Tail \\ \end{array} \\ \begin{array}{c c} Head \\ Tail \end{array} & \begin{bmatrix} 0.667 & 0.333 \\ 0.666 & 0.334 \end{bmatrix}\end{array} = \begin{bmatrix} 0.667 & 0.333 \end{bmatrix}$
Hence, The probability that the coin flipped on Friday of the same week also comes up head given that the coin flipped on monday comes up head = 0.667

Are the above answers correct? I can't verify them because author has not given answers to both questions( a & b ).

The more I see of this problem, the more I hate it. The fact that this problem allows the same transition matrix for states of coins and states of toss results of the prior coin is annoying. In a more general case, the state set would be larger, but it would be easier to keep track of which coin was tossed for the H, T, versus which coin resulted from that. (The transition matrix to the H,T result of the third coin toss is the same as the transition matrix to the fourth coin. It hurts my head, but I was never the sharpest knife in the drawer.)
I also greatly dislike your habit of equating a matrix to a vector. It would be better to separately say what $P^3$ is in one equation with no probability vector and then use $P^3$ symbolically times the probability vector in a separate equation. Also, normally the H,T transitions would have different values from the coin transitions and the combined transition matrix would be larger.

 

[pbuk]

Answer to a) is $\begin{bmatrix}0.7 & 0.3 \end{bmatrix} \times$ ...

That would be correct if the initial state vector were $\begin{bmatrix}0.7 & 0.3 \end{bmatrix}$, however the question states "If the coin initially flipped is equally likely to be coin 1 or coin 2".

Hence, The probability that the coin flipped on Friday of the same week also comes up head given that the coin flipped on monday comes up head = 0.667

Almost, but if you count through the days you will see that that is the probability that the coin flipped on Thursday of the same week also comes up head given that the coin flipped on Monday comes up head.