Steady-state vector of infinite markov chain

[pokey909]

Today we encountered a problem with markov chains and are wondering if this can be solved analytically.

Suppose we have a banded transition probability matrix M of the following form:
M=
[
P P 0 0 0 ...
Q 0 P 0 0 ...
0 Q 0 P 0 ...
0 0 Q 0 P ...
0 0 0 Q 0 ...
. . . . .
. . . . . ]

So with the exception of column 1, all others are equal. The "P 0 Q" element is only shifted one down (or row-wise, "Q 0 P" is shifted to the right).
It is impossible to construct a stochastic matrix with finite size unless you set the bottom-right element to Q.
When this is done, a good approximation of the real solution can be obtained by calculating the eigenvectors of the matrix or just raising it ot high powers.

Generally I am only interested in the sum of the steady-state vector elements.
So I am wondering if the evolution of M^n with n goin from 1 to infinity can be expanded into a series for which limit values are known.

Does anyone know a way to exactly calculate the limit value of the sum of the steady-state vector?

 

[bpet]

The steady-state probabilities can be found by solving Mx=x. For your example this produces a recurrence relation whose solution is a geometric series.