[markov chain] reading expected value from the transition matrix

[rahl___]

Hello there,

yet another trivial problem:

We have a transition matrix of some markov chain: \left[\begin{array}{ccc}e_{11}&...&e_{1n}\\...&...&...\\e_{n1}&...&e_{nn}\end{array}\right].
at the beginning our chain is in the state e_1. let T be the moment, when the chain reaches e_n for the first time. What is the expected value of T?

I've attended the 'stochastic process' course some time ago but the only thing I remember is that this kind of problem is really easy to compute, there is some simple pattern for this I presume.

thanks for your help,
rahl.

 

[willem2]

I don't think there's an easy answer to that. You can modify the matrix so, the chain will remain in state e_n if it gets there, and compute \sum_{k=1}^\infty k (i M^k - i M^{k-1})

where i is the initial state of (1, 0, ... , 0) and M the transition matrix. You need the last component of this of course.