- #1

- 31

- 0

Hey could someone explain why this is true? I am trying to understand how to solve such a problem but I don't understand the solution.

Given a Markov chain [itex]\left\{X_{n}: n\in\ \mathbb{N}\right\}[/itex] with four states 1,2,3,4 and transition matrix

[itex]P =

\begin{pmatrix}

0 & \frac{1}{2}& \frac{1}{2} & 0 \\

0 & 0 & \frac{1}{2} & \frac{1}{2}\\

\frac{1}{2} & 0 & 0 & \frac{1}{2} \\

\frac{1}{2} & \frac{1}{2} & 0 & 0

\end{pmatrix}[/itex]

We leave from state 3. What is the chance that state 1 will be reached before state 4?

Lets call this chance x. Lets call y the same chance but instead leaving from 2. Lets call:

[itex]f =

\begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}[/itex]

Then f satisfies P'f = f with P' the transition matrix derived from P by making state 1 and 4 absorbing states i.e.,

[itex]

\begin{pmatrix}

1 & 0& 0 & 0 \\

0 & 0 & \frac{1}{2} & \frac{1}{2}\\

\frac{1}{2} & 0 & 0 & \frac{1}{2} \\

0 & 0 & 0 & 1\end{pmatrix}*\begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}= \begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}[/itex]

Solving gives y = 1/2, x = 1/4.

**Problem:**Given a Markov chain [itex]\left\{X_{n}: n\in\ \mathbb{N}\right\}[/itex] with four states 1,2,3,4 and transition matrix

[itex]P =

\begin{pmatrix}

0 & \frac{1}{2}& \frac{1}{2} & 0 \\

0 & 0 & \frac{1}{2} & \frac{1}{2}\\

\frac{1}{2} & 0 & 0 & \frac{1}{2} \\

\frac{1}{2} & \frac{1}{2} & 0 & 0

\end{pmatrix}[/itex]

We leave from state 3. What is the chance that state 1 will be reached before state 4?

**Solution:**Lets call this chance x. Lets call y the same chance but instead leaving from 2. Lets call:

[itex]f =

\begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}[/itex]

Then f satisfies P'f = f with P' the transition matrix derived from P by making state 1 and 4 absorbing states i.e.,

[itex]

\begin{pmatrix}

1 & 0& 0 & 0 \\

0 & 0 & \frac{1}{2} & \frac{1}{2}\\

\frac{1}{2} & 0 & 0 & \frac{1}{2} \\

0 & 0 & 0 & 1\end{pmatrix}*\begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}= \begin{pmatrix}

1 \\

x \\

y \\

0

\end{pmatrix}[/itex]

Solving gives y = 1/2, x = 1/4.

Last edited: