1. PF Contest - Win "Conquering the Physics GRE" book! Click Here to Enter
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

[markov chain] reading expected value from the transition matrix

  1. Dec 29, 2009 #1
    Hello there,

    yet another trivial problem:
    I've attended the 'stochastic process' course some time ago but the only thing I remember is that this kind of problem is really easy to compute, there is some simple pattern for this I presume.

    thanks for your help,
  2. jcsd
  3. Dec 29, 2009 #2
    I don't think there's an easy answer to that. You can modify the matrix so, the chain will remain in state [itex] e_n [/itex] if it gets there, and compute [tex] \sum_{k=1}^\infty k (i M^k - i M^{k-1})[/tex]

    where i is the initial state of (1, 0, ..... , 0) and M the transition matrix. You need the last component of this of course.
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook