How Do Transition Probabilities Determine the Behavior of a Markov Chain?

[eXorikos]

Homework Statement

Let $$\left( X_n \right)_{n \geq 0}$$ be a Markov chain on {0,1,...} with transition probabilities given by:
$$p_{01} = 1$$, $$p_{i,i+1} + p_{i,i-1} = 1$$, $$p_{i,i+1} = \left(\frac{i+1}{i} \right)^2 p_{i,i-1}$$
Show that if $$X_0 = 0$$ then the probability that $$X_n \geq 1$$ for all $$n \geq 1$$ is 6/$$\pi^2$$

The Attempt at a Solution

I really don't have an attempt. I think I have to use the master equation for discrete time since it's a stationary distribution for n>0. I've been thinking about it more than it seems by these two sentences but I'm quite stuck...

 

[mighty2000]

Hello! You are correct in thinking that the master equation for discrete time can be used to solve this problem. Let's approach it step by step.

First, let's define the state space of our Markov chain. Since the chain is defined on {0,1,...}, we can write it as S = {0,1,2,...}. Next, let's define the transition matrix P, where P_{i,j} represents the probability of going from state i to state j. In our case, we have:

P_{i,i+1} = \left(\frac{i+1}{i} \right)^2 p_{i,i-1} = \left(\frac{i+1}{i} \right)^2 \left( 1 - p_{i,i+1} \right) = \left(\frac{i+1}{i} \right)^2 \left( 1 - \left(\frac{i+1}{i} \right)^2 p_{i,i-2} \right)

We can continue this pattern and write the transition matrix as:

P = \begin{pmatrix} 0 & 1 & 0 & 0 & \cdots \\ \left(\frac{1}{2} \right)^2 & 0 & \left(\frac{3}{2} \right)^2 & 0 & \cdots \\ 0 & \left(\frac{2}{3} \right)^2 & 0 & \left(\frac{4}{3} \right)^2 & \cdots \\ 0 & 0 & \left(\frac{3}{4} \right)^2 & 0 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \ddots \end{pmatrix}

Now, let's define the initial state distribution, where X_0 = 0. This means that the initial state distribution is:

\pi_0 = \begin{pmatrix} 1 & 0 & 0 & 0 & \cdots \end{pmatrix}

Using the master equation for discrete time, we can write the stationary distribution as:

\pi P = \pi

Substituting the values of \pi_0 and P, we get:

\begin{pmatrix} 1 & 0 & 0 & 0 & \cdots \end{pmatrix