- #1

mathmari

Gold Member

MHB

- 5,049

- 7

An octopus is trained to chosose from two objects A and B always the object A. Repeated training shows the octopus both objects, if the octopus chooses object A, he will be rewarded. The octopus can be in 3 levels of training:

Level 1: He can not remember which object was rewarded.

Level 2: He briefly remembers, chooses object A, but he can forget it again.

Level 3: He remembers, chooses object A and never forgets it.

The probability that the octopus rises from level 1 to level 2 is the same great, as he learns nothing.

When the octopus reaches level 2, the probability of falling back to level 1 is the same as staying in level 2 and rising to level 3 together.

The probability of moving up from level 2 to level 3 is 5 times greater than staying in level 2.

The octopus can not skip any training level and we assume that he is in level 1 at the beginning of the workout.

- Set up the associated transition matrix of this markov chain and specify the initial distribution.
- What is the probability that the octopus learns nothing in the first 5 attempts and stays completely in level 1?
- How big are the probabilities of the individual training levels after 3 attempts?

- The stochastic matrix is in the following form: $$P=\begin{pmatrix}P_{1,1} & P_{1,2} & P_{1,3} \\ P_{2,1} & P_{2,2} & P_{2,3} \\ P_{3,1} & P_{3,2} & P_{3,3}\end{pmatrix}$$

We have the following information:- $P_{1,2}=P(\text{Learns nothing})$
- $P_{2,1}=P_{2,2}+P_{2,3}$
- $P_{2,3}=5\cdot P_{2,2}$

Which is the probability $P(\text{Learns nothing})$ ? (Wondering)

We have that $P_{2,1}+P_{2,2}+P_{2,3}=1$ and $P_{2,1}=P_{2,2}+P_{2,3}$ and $P_{2,3}=5\cdot P_{2,2}$, so we get the following: \begin{align*}P_{2,2}+P_{2,3}+P_{2,2}+P_{2,3}=1&\Rightarrow 2\cdot P_{2,2}+2\cdot P_{2,3}=1 \\ & \Rightarrow 2\cdot P_{2,2}+2\cdot (5\cdot P_{2,2})=1 \\ & \Rightarrow 2\cdot P_{2,2}+10\cdot P_{2,2}=1 \\ & \Rightarrow 12\cdot P_{2,2}=1 \\ & \Rightarrow P_{2,2}=\frac{1}{12}\end{align*}

So, we get also $\displaystyle{P_{2,3}=5\cdot P_{2,2}=5\cdot \frac{1}{12}=\frac{5}{12}}$ and $\displaystyle{P_{2,1}=P_{2,2}+P_{2,3}=\frac{1}{12}+\frac{5}{12}=\frac{6}{12}=\frac{1}{2}}$.

So, we get the transition matrix: $$P=\begin{pmatrix}P_{1,1} & P_{1,2} & P_{1,3} \\ \frac{1}{2} & \frac{1}{12} & \frac{5}{12} \\ P_{3,1} & P_{3,2} & P_{3,3}\end{pmatrix}$$

How can we complete the remaining matrix? (Wondering)