Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Concept question on irreducibility (markov chain)

  1. Feb 22, 2012 #1
    1. The problem statement, all variables and given/known data
    This is not really a homework question but just something that I'm confused about.

    I'm having trouble with understanding the movement of Markov chain.
    Say, if S = {0,1,2....} and 0<q<1. Let p(i,i+1)=q and p(i,i-1)=1-q. I can see how this Markov chain moves (back and forth, basically; going one step at a time from time t to time t+1 with probability of either q or 1-q). I can see that this Markov chain is irreducible, since if i<j, then p(i,j) = q^(j-i)>0 and p(j,i) = 1-q^(j-1)>0 (by the way, is this explanation right?)

    However, what about for the Markov chain that looks like this:
    S = {0,1,2....} and 0<q<1. Let p(i,i+1)=q and p(i,0)=1-q? How does it move? for p(i,i+1)=q it is going incrementally one step at a time, but how about p(i,0)=1-q? Does it skip all the way back to 0? How can I then tell whether if this Markov chain is irreducible or not?

    Any help would be appreciated. Thanks in advance!
  2. jcsd
  3. Feb 22, 2012 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Starting from state 0, is possible to reach state i > 1 for any given i (that is, is there a non-zero probability of reaching state i eventually)? Starting from state 0, is it possible to return to state 0 (at some finite time in the future)? If your answer is YES to both these questions, what would that tell you about the markov chain?

Share this great discussion with others via Reddit, Google+, Twitter, or Facebook