Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Markov chain, sum of N dice rolls

  1. Jan 6, 2012 #1
    Question : Let Xn be the maximum score obtained after n throws of a fair dice

    a) Prove that Xn is a markov chain and write down the transition matrix

    Im having a problem starting the transition matrix

    im assuming the states are meant to be the sum. then do you write out the transition matrix for the first 2 throws and have this matrix to the power of n-1?
  2. jcsd
  3. Jan 6, 2012 #2

    Stephen Tashi

    User Avatar
    Science Advisor

    I suppose if you are studying markov chains with an infinite number of states, you could try interpreting "maximum score" to mean some sort of sum. However, it seems to me that the problem intends the state of the process on the nth roll to be [itex] max \{ R_1,R_2,...R_n\} [/itex] and not [itex] R_1 + R_2 + ... + R_n [/itex]. So if make 3 rolls and they are {3,5,4} the state of the process is [itex] X_3 = 5 [/itex]
  4. Jan 7, 2012 #3
    Thanks for your reply, that makes sense.

    So the transition matrix is an upper triangular matrix to the power of n-1 with the diagonal entries 1/6, 2/6, 3/6, 4/6, 5/6, 6/6 respectively?
  5. Jan 7, 2012 #4

    Stephen Tashi

    User Avatar
    Science Advisor

    That is incorrect terminology. To compute things about the state at step n in the process, one may raise the transition matrix to a power, but the transition matrix itself, in simple examples, is not a function of n.

  6. Jan 7, 2012 #5
    Thank you for your help
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook