1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Find all Invariant Probability Measures for P (Markov Chain)

  1. Mar 23, 2012 #1
    1. The problem statement, all variables and given/known data
    Find all Invariant Probability Measures for P (Markov Chain)
    E = {1,2,3,4,5}

    The screenshot below has P and my attempted solution. I am wondering if it acceptable to have infinitely many answers ("all" seems to indicate that is acceptable). Basically, I had too many unknowns and too few equations. Does my work look right?
    http://i.minus.com/ioEfrJKWamvpR.JPG [Broken]

    edit: I should add also that pi_1 >= 0 and pi_1 <= 1/2 to force the probabilities to be between 0 and 1. And after a bit of thought, I feel more comfortable with my answer. It seems to embody the fact that if I start at state 3, I stay there forever. Hence, pi_1 = 0 => steady-state probabilities are [0,0,1,0,0]. I can also be stuck in the recurrent class {1,5} in which case pi_1 = 1/2, and I never reach 3. I.e. pi = [1/2,0,0,0,1/2]. And there are many possibilities in between if I start in state 4 etc. where I have a finite probability of being sucked into the recurrent class or into the absorbing class. So pi = [nonzero, 0, nonzero, 0, nonzero] which the equation can handle.

    edit2: Upon further thought, though, it seems there should only be 3 invariant probability measures. One if I start in state 3, one if I start in state one or five, one if I start in state 2 or 4 (since 2 a.s. goes to 4).

    I believe the answers are:
    start in 1 or 5: [1/2 0 0 0 1/2] (so pi_1 = 1/2)
    start in 3: [0 0 1 0 0] (so pi_1 = 0)
    start in 2 or 4: [1/4 0 1/2 0 1/4] (so pi_1 = 1/4)
    Last edited by a moderator: May 5, 2017
  2. jcsd
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Can you offer guidance or do you also need help?
Draft saved Draft deleted