Stationary probabilities.(Markov chain).

  • #1
MathematicalPhysicist
Gold Member
4,188
168

Main Question or Discussion Point

We are given two states 1,2 in an irreducible and positive recurrent Markov chain, and their stationary probabilities [tex]\pi_1[/tex] and [tex]\pi_2[/tex] respectively, try to characterise in general the probability (distribution) of the number of visits in state 2 after two consecutive visits in state 1.

Any hints?
 

Answers and Replies

  • #2
Write out the ways this can happen, then turn it into a formula:

1,1,2
2,1,1,2
2,2,1,1,2
1,2,1,1,2
...
 
  • #3
MathematicalPhysicist
Gold Member
4,188
168
Yes I thought in this direction but not sure how to get to the formula.
I mean for 1,1,2 the probability is: [tex]P_{1,1}\frac{\pi_1}{\pi_1+\pi_2}P_{1,2}[/tex]
2,1,1,2 [tex]\frac{\pi_2}{\pi_1+\pi_2}P_{1,1}P_{2,1}P_{1,2}[/tex]
1,2,1,1,2 [tex]\frac{\pi_1}{\pi_1+\pi_2}P^2_{1,2}P_{1,1}P_{2,1}[/tex]

So my hunch is that if we first get to 1 or 2, we should multiply by pi1/(pi1+pi2) or pi2/(pi1+pi2), and we should always multiply by P1,1, but other than this I don't see a general equation for all cases.
 
  • #4
I don't have an immediate answer, but you might find the approach in this paper to be helpful:

http://smu.edu/statistics/TechReports/TR211.pdf [Broken]

The authors derive the unconditional distribution of the number of successes (e.g., state 2) in n+1 trials. It seems to me that you need to derive a similar distribution, conditional on having obtained (exactly? or at least?) two consecutive failures (1,1).
 
Last edited by a moderator:

Related Threads for: Stationary probabilities.(Markov chain).

Replies
5
Views
13K
  • Last Post
Replies
1
Views
543
Replies
1
Views
2K
Replies
2
Views
1K
Replies
3
Views
211
Replies
1
Views
4K
  • Last Post
Replies
6
Views
2K
  • Last Post
Replies
17
Views
4K
Top