Markov chains and production line comprising 3 machines

[PeroK]

Why P[10] = p?
And why P[20] = P[30] = 0? I mean the transition here for P[20] is saying that we have 2 machines today, then a repairman repairs the third machine and so the probability 0 machines are working by the end of the next day is q^3, isn't that right?

Let's take the state $X_0$ where no machines are active at the end of the day. This is represented by the vector $(1, 0, 0, 0)$ (where I've put this in a row to save some Latex). It will be a column vector below.

One machine is repaired overnight. This breaks down the next day with probability $q$, so this is the probability of being back in state $X_0$ the following night. The machine lasts the next day with probability $p$, which is the the probability of having state $X_1$ at the end of the next day. The other states have probability $0$ at the end of the following day. Applying $T$ to $X_0$ gives:
$$
\begin{bmatrix}
q& \dots \\
p& \dots \\
0 & \dots \\
0 & \dots
\end{bmatrix}
\begin{bmatrix}
1 \\
0 \\
0 \\
0
\end{bmatrix} =
\begin{bmatrix}
q \\
p \\
0 \\
0
\end{bmatrix}
$$As required.

 

[PeroK]

This is what I got for the transition matrix, is there anything wrong with it:

P[00] = q P[01] = p P[02] = p^2 P[03] = p^3

P[10] = q P[11] = p P[12] = p^2 P[13] = P^2

P[20] = q^2 P[21] = (p*q)*2 P[22] = p^2 P[23] = p^3

P[30] = q^3 P[31] = p*q^2 P[32] = q*p^2 P[33] = p^3

You've got the rows and columns mixed up. See my example above. That said I don't agree with many of your entries. For example: if there is $1$ working machine at the end of the day, then there are $2$ working machines at the beginning of next day, and the probability there is $1$ working machine at the end of that day is $2pq$. So $P_{11} = 2pq$.

So the reasoning for the values that I got are as follows:

P[00] = q: It means that no working machine exists by the end of one day, then a repairman repairs one machine and then it fails by the end of the next day. Thus, the probability that the one working machine at the start of the day will fail by the end of the day is q.

Okay.

P[01] = p: It means that no working machine exists by the end of one day, then a repairman repairs one machine and then it remains working by the end of the next day. Thus, the probability that the one working machine remains working is p.

Okay, but this is entry $P_{10}$ in the matrix.

P[02] = p^2: It means that no working machine exists by the end of one day, then a repairman repairs one machine and then it remains working by the end of the next day, but also an extra working machine is added to the production line. Thus, the probability that both machines remain working during the day is p^2.

There is nothing in the problem about machines being repaired during the day.

 

[Steve4Physics]

Let me add a few words. I suspect you haven't fully understood the problem, E.g. you wrote:

P[03] = p^3: It means that no working machine exists by the end of one day

No it doesn't. P[03] is the probability of
- changing from the state with 0 machines working at the end of a day to
- the state with 3 machines working at the end of the next day.

According to the rules, only 1 machine can be repaired overnight (and implicitly, additional machines cannot be added). The transition 0→3 is therefore impossible over a full day. So P[03]=0

Edit - reworded to (hopefully) make explanation clearer.

 

[pbuk]

There have been a few diversions in this thread due at least in part to ambiguous wording in the question. For anyone looking for a well-written introduction to Markov chains I can recommend the freely available notes and (unambiguously worded) exercises from this Cambridge University course.

Otherwise please note the solutions to the unambiguous parts below. Please start a new thread for any further discussion.

a-) Specify the state space of (Xn : n ≥ 0).
We are told that $X_n$ denotes the number of working machines at the end of day before the repairman begins any over-night repair. As there are 3 machines we can immediately write down the answer $S = \{0, 1, 2, 3\}$.

b-) Determine the transition matrix in terms of p.
If there are no machines are working at the end of day $t$ then the first thing that happens is exactly 1 machine is repaired overnight, then by the end of the next day either:

• exactly one machine breaks down (the one that was repaired) with probability $q = 1 - p$ and so there are no machines working at the end of day $t + 1$. We can now make the first entry in the transition matrix $a_{0, 0} = 1-p$.
• no machines break down with probability $p$ and so there is exactly 1 machine working at the end of day $t + 1$ (the one that was repaired overnight). We can now make the second entry in the transition matrix $a_{0, 1} = p$.

Because we started the day with only one machine working there are no other possibilities and so we can complete the first row of the transition matrix with zeros.

For the second row we start with exactly 1 machine working at the end of day $t$ so after another machine is repaired overnight there are more possibilities for what can happen during the next day and the second row looks like $\left [(1-p)^2, 2p(1-p), p^2, 0 \dots \right ]$ (note that the sum of each row is always 1: the matrix is said to be a right stochastic matrix and has a number of useful properties).

c-) Given that 2 machines are idle tonight, what is the probability of one idle machine 3 nights later, if p = 0.8 (answer 6 decimal places).

The answer to this question depends on the meaning of the word "idle" and whether we seek the probability of at least one or exactly one "idle" machine. Once we have determined these we can establish the initial state (row) vector $\tau_0$ and calculate the probabilities using $\tau_3 = ((\tau_0 P)P)P = \tau_0 P^3$.

 

[berkeman]

And with that excellent summary, this thread is now tied off. Thanks all.