Today we encountered a problem with markov chains and are wondering if this can be solved analytically.
Suppose we have a banded transition probability matrix M of the following form:
M=
[
P P 0 0 0 ...
Q 0 P 0 0 ...
0 Q 0 P 0 ...
0 0 Q 0 P ...
0 0 0 Q 0 ...
. . . . .
. . . . . ]...