[markov chain] reading expected value from the transition matrix

[rahl___]

Hello there,

yet another trivial problem:

We have a transition matrix of some markov chain: $$\left[\begin{array}{ccc}e_{11}&...&e_{1n}\\...&...&...\\e_{n1}&...&e_{nn}\end{array}\right]$$.
at the beginning our chain is in the state $$e_1$$. let T be the moment, when the chain reaches $$e_n$$ for the first time. What is the expected value of T?

I've attended the 'stochastic process' course some time ago but the only thing I remember is that this kind of problem is really easy to compute, there is some simple pattern for this I presume.

thanks for your help,
rahl.

 

[willem2]

I don't think there's an easy answer to that. You can modify the matrix so, the chain will remain in state $e_n$ if it gets there, and compute $$\sum_{k=1}^\infty k (i M^k - i M^{k-1})$$

where i is the initial state of (1, 0, ... , 0) and M the transition matrix. You need the last component of this of course.