# Stationary distribution of Markov chain

1. Apr 8, 2008

### Pere Callahan

Hi all,

I'm given a Markov chain $Q_k$, k>0 with stationary transition probabilities. The state is space uncountable.
What I want to show is that the chain is asymptotically stationary, that is it converges in distribution to some random variable Q.

All I have at hand is an k-independent upper bound for $P(|Q_k|>x)$ for all x in the state space (and some indecomposability assumptions on the state space).

Is this enough to conclude convergence of the chain?

Thanks for any help.

-Pere

2. Apr 8, 2008

I don't have much experience with uncountable state-space Markov chains, but the irreducibility conditions will typically imply that any stationary distribution must be unique. Also, you'll need some ergodicity condition to ensure that the chain actually converges to the stationary distribution, even if it isn't necessary to show that the stationary distribution exists.

3. Apr 8, 2008

### Pere Callahan

Any more insights on the topic?
Thanks

4. Apr 8, 2008

Well, it seems that the basic result should be about checking the "return time" for a given state; in countable chains, there's a necessary and sufficient condition for the existance of a stationary distribution, where the expected return time for every state needs to be finite (and then the value of the stationary distribution for that state is the inverse of the return time). I'm not sure what the uncountable extension of that result would be: it seems that you can't ever expect to get back to *exactly* the same point in the state space (since it's a set of measure 0). Presumably some condition relating to the probability of returning within some $\delta$ of a given point in the state space?

5. Apr 8, 2008

### Pere Callahan

Thanks again.

So irreducibility ensures existence of a stationary distribution and ergodicity convergence to this stationary distribution?

Is ergodicity the same as non-periodicity....? I should read some more on the topic.

The theory of Harris chains seems to be a good starting point.

So i found two properties related to continous state space Markov chains

The first $\phi$ -irreducibility:

Here P_n are the n-step transition probabilities.

The second one is a-periodicity, that is there do not exist subsets X_1 , ... , X_d of the state space such that the chain jumps with probability one from X_1 to X_2, from X-2 to X_3 .. and from X_d back to X_1.

Under these two conditions, the Markov chain is (reportedly) asymptotically stationary.

So I have to think about it ..

Last edited: Apr 8, 2008
6. Apr 8, 2008