Concept question on irreducibility (markov chain)

[hsinyihhsu]

Homework Statement

This is not really a homework question but just something that I'm confused about.

I'm having trouble with understanding the movement of Markov chain.
Say, if S = {0,1,2...} and 0<q<1. Let p(i,i+1)=q and p(i,i-1)=1-q. I can see how this Markov chain moves (back and forth, basically; going one step at a time from time t to time t+1 with probability of either q or 1-q). I can see that this Markov chain is irreducible, since if i<j, then p(i,j) = q^(j-i)>0 and p(j,i) = 1-q^(j-1)>0 (by the way, is this explanation right?)

However, what about for the Markov chain that looks like this:
S = {0,1,2...} and 0<q<1. Let p(i,i+1)=q and p(i,0)=1-q? How does it move? for p(i,i+1)=q it is going incrementally one step at a time, but how about p(i,0)=1-q? Does it skip all the way back to 0? How can I then tell whether if this Markov chain is irreducible or not?

Any help would be appreciated. Thanks in advance!

 

[Ray Vickson]

Homework Statement

This is not really a homework question but just something that I'm confused about.

I'm having trouble with understanding the movement of Markov chain.
Say, if S = {0,1,2...} and 0<q<1. Let p(i,i+1)=q and p(i,i-1)=1-q. I can see how this Markov chain moves (back and forth, basically; going one step at a time from time t to time t+1 with probability of either q or 1-q). I can see that this Markov chain is irreducible, since if i<j, then p(i,j) = q^(j-i)>0 and p(j,i) = 1-q^(j-1)>0 (by the way, is this explanation right?)

However, what about for the Markov chain that looks like this:
S = {0,1,2...} and 0<q<1. Let p(i,i+1)=q and p(i,0)=1-q? How does it move? for p(i,i+1)=q it is going incrementally one step at a time, but how about p(i,0)=1-q? Does it skip all the way back to 0? How can I then tell whether if this Markov chain is irreducible or not?

Any help would be appreciated. Thanks in advance!

Starting from state 0, is possible to reach state i > 1 for any given i (that is, is there a non-zero probability of reaching state i eventually)? Starting from state 0, is it possible to return to state 0 (at some finite time in the future)? If your answer is YES to both these questions, what would that tell you about the markov chain?

RGV