(adsbygoogle = window.adsbygoogle || []).push({}); 1. The problem statement, all variables and given/known data

This is not really a homework question but just something that I'm confused about.

I'm having trouble with understanding the movement of Markov chain.

Say, if S = {0,1,2....} and 0<q<1. Let p(i,i+1)=q and p(i,i-1)=1-q. I can see how this Markov chain moves (back and forth, basically; going one step at a time from time t to time t+1 with probability of either q or 1-q). I can see that this Markov chain is irreducible, since if i<j, then p(i,j) = q^(j-i)>0 and p(j,i) = 1-q^(j-1)>0 (by the way, is this explanation right?)

However, what about for the Markov chain that looks like this:

S = {0,1,2....} and 0<q<1. Let p(i,i+1)=q and p(i,0)=1-q? How does it move? for p(i,i+1)=q it is going incrementally one step at a time, but how about p(i,0)=1-q? Does it skip all the way back to 0? How can I then tell whether if this Markov chain is irreducible or not?

Any help would be appreciated. Thanks in advance!

**Physics Forums | Science Articles, Homework Help, Discussion**

Join Physics Forums Today!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

# Homework Help: Concept question on irreducibility (markov chain)

**Physics Forums | Science Articles, Homework Help, Discussion**