Calculating Probability of Markov Chain State in Discrete Time

[Zaare]

Let $$\left( {X_n } \right)_{n \ge 0}$$ be a Markov chain (discrete time).

I have

$${\bf{P}} = \left[ {pij} \right]_{i,j} = \left[ {P\left( {X_1 = j|X_0 = i} \right)} \right]_{i,j}$$,

and the initial probability distribution $${\bf{p}}^{\left( 0 \right)}$$.

I need to calculate

$$P\left( {\mathop \cup \limits_{i = 0}^8 X_i = 3} \right)$$

I can use Matlab for the numerical calculations but I need to find an expression for this probability. The only expression I have been able to find consists of too many terms to be considered reasonable.
Any suggestions on how to find an expression, that at least in some way is repetitive so that the numerical calculations can be done by the computer, would be appreciated.

 

[Zaare]

Hehe, nevermind, I figuered it out. If the state 3 is turned into a absorbing state, then:

$$P\left( {\bigcup\limits_{i = 0}^8 {X_i = 3} } \right) = 1 - P\left( {\bigcup\limits_{i = 0}^8 {X_i \ne 3} } \right) = 1 - \left( {1 - P\left( {X_8 = 3} \right)} \right) = P\left( {X_8 = 3} \right)$$

 

[tacman]

To calculate the probability of the Markov chain being in state 3 at any point in time from 0 to 8, we can use the following formula:

P(X_n = 3) = \sum_{i = 0}^{8} P(X_n = 3|X_0 = i)P(X_0 = i)

This formula takes into account all possible initial states and the probability of transitioning to state 3 after n steps. This expression is repetitive in the sense that it considers all possible initial states and their corresponding probabilities.

Using this formula, we can easily calculate the probability of the Markov chain being in state 3 at any given time without having to consider each individual step. This can be easily implemented in Matlab for numerical calculations.

Another approach would be to use the Chapman-Kolmogorov equation, which states that:

P(X_n = j|X_0 = i) = \sum_{k = 0}^{n} P(X_n = j|X_k = i)P(X_k = i)

This formula allows us to calculate the probability of being in state j at time n, given an initial state i at time 0. This can also be implemented in Matlab for numerical calculations.

In conclusion, using these formulas, we can find an expression for the probability of the Markov chain being in state 3 at any given time. It may involve some calculations, but it is a more efficient and accurate approach compared to considering each individual step.