1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Markov Chain - Rat w/4 Rooms

  1. Sep 14, 2008 #1
    I've attached the diagram of 4 rooms, which a rat must move through. Each period he changes his room (his state). As you can see if you click on the image, the rat cannot access room 2 from 3, vice versa.

    If I assume the rat begins in room 1, how do I calculate the probability he will be in room 4 after his second move? I have the transitional probability matrix correct, I'm just not sure how to move forward.

    rat markov chain.GIF

    Thank you,

    Jeff
     
  2. jcsd
  3. Sep 15, 2008 #2

    HallsofIvy

    User Avatar
    Staff Emeritus
    Science Advisor

    Set up the "transition matrix". According to your picture, if the rat is in room 1, the is a 1/3 chance he will move to each of 2, 3, 4. That means you want your first column to be [0 1/3 1/3 1/3]. That's because you will be multiplying the matrix by a column matrix with entries representing the probability the rat is in room 1, 2, 3, or 4 respectively. If you know for certain the rat is in room 1, then your column matrix is [1 0 0 0] and it is the first column of the matrix that multiplies that first 1.

    Similarly, if the rat is in room 2 the probability is 1/2 that it will move to room 1 and 1/2 that it will move to room 4: the second column in your matrix must be [1/2 0 0 1/2].

    If the rat is in room 3 the probability is also 1/2 that it will move to room 1 and 1/2 that it will move to room 4 so your third column is the same as the second column.

    Finally, if the rat is in room 3, the probability is 1/3 that it will move to room 1, 1/3 that it will move to room 2, and 1/3 that it will move to room 3 so the fourth column is [1/3 1/3 1/3 0].

    Your transition matrix is
    [tex]A= \left[\begin{array}{cccc} 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{2} & 0 & 0 \frac{1}{2} \\ \frac{1}{2} & 0 & 0 \frac{1}{2} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} & 0\end{array}\right][/tex]

    Is that what you have?

    To find "the probability he will be in room 4 after his second move If the rat begins in room 1" you need to calculate A2 times the column matrix [1 0 0 0].
     
  4. Sep 15, 2008 #3
    Thanks - that's what I thought, but I wasn't sure about the treatment of the initial probability vector.

    Jeff
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?