Markov chain

  • Thread starter Zaare
  • Start date
  • #1
54
0

Main Question or Discussion Point

Let [tex]\left( {X_n } \right)_{n \ge 0}[/tex] be a Markov chain (discrete time).

I have

[tex]{\bf{P}} = \left[ {pij} \right]_{i,j} = \left[ {P\left( {X_1 = j|X_0 = i} \right)} \right]_{i,j}[/tex],

and the initial probability distribution [tex]{\bf{p}}^{\left( 0 \right)}[/tex].

I need to calculate

[tex]P\left( {\mathop \cup \limits_{i = 0}^8 X_i = 3} \right)[/tex]

I can use Matlab for the numerical calculations but I need to find an expression for this probability. The only expression I have been able to find consists of too many terms to be considered reasonable.
Any suggestions on how to find an expression, that at least in some way is repetitive so that the numerical calculations can be done by the computer, would be appreciated.
 

Answers and Replies

  • #2
54
0
Hehe, nevermind, I figuered it out. If the state 3 is turned into a absorbing state, then:

[tex]P\left( {\bigcup\limits_{i = 0}^8 {X_i = 3} } \right) = 1 - P\left( {\bigcup\limits_{i = 0}^8 {X_i \ne 3} } \right) = 1 - \left( {1 - P\left( {X_8 = 3} \right)} \right) = P\left( {X_8 = 3} \right)[/tex]
 
Last edited:

Related Threads for: Markov chain

  • Last Post
Replies
6
Views
2K
  • Last Post
Replies
10
Views
3K
  • Last Post
Replies
17
Views
4K
  • Last Post
Replies
7
Views
1K
  • Last Post
Replies
1
Views
2K
  • Last Post
Replies
11
Views
9K
  • Last Post
Replies
1
Views
5K
  • Last Post
Replies
3
Views
1K
Top