1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Markov chain problem

  1. Nov 17, 2008 #1
    1. The problem statement, all variables and given/known data
    i have a scenario which i have to find the proportion of time spent in each area by a person using markov chains. i was given a word problem, which i have put into a matrix and the question asks what the proportion of time is spent in each area A, B and C.

    2. Relevant equations


    3. The attempt at a solution
    i have the final matrix as:
    0 0.34 0.67
    0 0 0.34
    1 0.34 0

    im stuck at how to determine how to do this, can someone help?
  2. jcsd
  3. Nov 18, 2008 #2
    Find the stationary distribution of the Markov chain (that is an eigen vector to the transition matrix)
  4. Nov 18, 2008 #3
    we havent been taught eigen vectors yet
  5. Nov 18, 2008 #4

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    That is not a valid transition matrix. Each row must sum to 1.
  6. Nov 18, 2008 #5
    sorry, i made a typo, its:
    0 0.67 0.67
    0 0 0.34
    1 0.33 0
  7. Nov 18, 2008 #6

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    That's still not valid. In fact, it's worse; now the first row is also invalid. The sum of each row must be identically one. What is the word problem that led to this matrix?
  8. Nov 18, 2008 #7
    heres the word problem:
    A fox hunts in three territories A, B and C. He
    never hunts in the same territory on two successive days.
    If he hunts in A, then he hunts in C the next day. If he
    hunts in B or C, he is twice as likely to hunt in A the next
    day as in the other territory.

    i only gave the scenerio, i dont need help with the actual questions, just how to get the initial transition mtrix. its just confusing for me reading the problem and transfering it to a mathematica matrix. i am going by the examples in the text book in terms of the rows adding up to 1, in my text book it has the columns adding to 1
  9. Nov 18, 2008 #8

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    Sorry for the misdirection. Your corrected matrix is fine.

    Suppose the probabilities that the fox at the nth time step tn is in state A, B, or C are PA(tn), PB(tn), and PC(tn). Let P(tn) be the column vector formed from these individual state probabilities. The state probabilities at the next time step are given by the transition matrix S: P(tn+1)=S×P(tn).

    The system is in steady-state if P(tn+1)=P(tn). Try to find this steady-state probability vector (the components must add to one).
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Similar Discussions: Markov chain problem
  1. Markov Chain (Replies: 1)

  2. Markov chains (Replies: 0)

  3. Markov Chain Problem (Replies: 1)

  4. Markov Chain Problem (Replies: 2)

  5. Markov Chain (Replies: 1)