Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Markov Chain - Rat w/4 Rooms

  1. Sep 14, 2008 #1
    I've attached the diagram of 4 rooms, which a rat must move through. Each period he changes his room (his state). As you can see if you click on the image, the rat cannot access room 2 from 3, vice versa.

    If I assume the rat begins in room 1, how do I calculate the probability he will be in room 4 after his second move? I have the transitional probability matrix correct, I'm just not sure how to move forward.

    rat markov chain.GIF

    Thank you,

    Jeff
     
  2. jcsd
  3. Sep 15, 2008 #2

    HallsofIvy

    User Avatar
    Staff Emeritus
    Science Advisor

    Set up the "transition matrix". According to your picture, if the rat is in room 1, the is a 1/3 chance he will move to each of 2, 3, 4. That means you want your first column to be [0 1/3 1/3 1/3]. That's because you will be multiplying the matrix by a column matrix with entries representing the probability the rat is in room 1, 2, 3, or 4 respectively. If you know for certain the rat is in room 1, then your column matrix is [1 0 0 0] and it is the first column of the matrix that multiplies that first 1.

    Similarly, if the rat is in room 2 the probability is 1/2 that it will move to room 1 and 1/2 that it will move to room 4: the second column in your matrix must be [1/2 0 0 1/2].

    If the rat is in room 3 the probability is also 1/2 that it will move to room 1 and 1/2 that it will move to room 4 so your third column is the same as the second column.

    Finally, if the rat is in room 3, the probability is 1/3 that it will move to room 1, 1/3 that it will move to room 2, and 1/3 that it will move to room 3 so the fourth column is [1/3 1/3 1/3 0].

    Your transition matrix is
    [tex]A= \left[\begin{array}{cccc} 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{2} & 0 & 0 \frac{1}{2} \\ \frac{1}{2} & 0 & 0 \frac{1}{2} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} & 0\end{array}\right][/tex]

    Is that what you have?

    To find "the probability he will be in room 4 after his second move If the rat begins in room 1" you need to calculate A2 times the column matrix [1 0 0 0].
     
  4. Sep 15, 2008 #3
    Thanks - that's what I thought, but I wasn't sure about the treatment of the initial probability vector.

    Jeff
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?