Autocorrelation Function of Markov Chain/Discrete Process

1. Apr 10, 2012

RoshanBBQ

1. The problem statement, all variables and given/known data
I'm working with a transition matrix that is
1-p, p
q, 1-q

And I need to find
$$E(X_n X_{n+\tau})$$
with E = {-1,1}. The first row is probabilities given currently in state -1.

And this needs to be a function of tau. I started by initiializing my M.C. with the steady-state probabilities, thinking that would ensure X_n was stationary (and so its autocorrelation would be a function of only tau).

I then thought there would be two cases:
$$\tau = 0$$
$$E(X_n^2)=\pi_1 + \pi_{-1} = 1$$

Then the other case
$$\tau \ne 0$$
$$E(x_n x_{n+\tau}) = (1)(1)\pi_1^2 + (-1)^2 \pi_{-1}^2 - 2 \pi_1 \pi_{-1} = \left ( \frac{p-q}{p+q} \right)^2$$

Problem:
I simulated the MC and computed the autocorrelation. Things I found:
1.) For tau = 0, I am right.
2.) My value for tau != 0 is the case when tau > about 6 or 7.
3.) For tau = 1,2,3 or so the values are consistently different from my solution. I need to know what is going on.
4.) If p = 1- q, P has all equal rows. In this situation, my solution works.

Confusions:
I don't understand how I could be wrong since I initialized the beginning state using the steady-state probabilities. So I thought the probability of going to any state in n steps from there was:
$$\pi P^n$$
evaluated at the state you want That gives you pi_state no matter what.

I can post my code if wanted.

Last edited: Apr 11, 2012