# Homework Help: Find all Invariant Probability Measures for P (Markov Chain)

1. Mar 23, 2012

### RoshanBBQ

1. The problem statement, all variables and given/known data
Find all Invariant Probability Measures for P (Markov Chain)
E = {1,2,3,4,5}

The screenshot below has P and my attempted solution. I am wondering if it acceptable to have infinitely many answers ("all" seems to indicate that is acceptable). Basically, I had too many unknowns and too few equations. Does my work look right?
http://i.minus.com/ioEfrJKWamvpR.JPG [Broken]

edit: I should add also that pi_1 >= 0 and pi_1 <= 1/2 to force the probabilities to be between 0 and 1. And after a bit of thought, I feel more comfortable with my answer. It seems to embody the fact that if I start at state 3, I stay there forever. Hence, pi_1 = 0 => steady-state probabilities are [0,0,1,0,0]. I can also be stuck in the recurrent class {1,5} in which case pi_1 = 1/2, and I never reach 3. I.e. pi = [1/2,0,0,0,1/2]. And there are many possibilities in between if I start in state 4 etc. where I have a finite probability of being sucked into the recurrent class or into the absorbing class. So pi = [nonzero, 0, nonzero, 0, nonzero] which the equation can handle.

edit2: Upon further thought, though, it seems there should only be 3 invariant probability measures. One if I start in state 3, one if I start in state one or five, one if I start in state 2 or 4 (since 2 a.s. goes to 4).