Two state markov chain model

[lemontree45]

I have a state transition probability matrix and a state probability vector

[0.9 0.1; 0.1 0.9] & [0.4 0.6] respectively.

Now, I want to generate the states of 1 and 0 according to this. say 100 state sequence.

Any sort of help would be appreciated.

Thanks.

 

[micromass]

Hi lemontree45!

What exactly do you want?? Do you want to know the probability of the states in the 100th step? That would be

$$\left(\begin{array}{cc}0.9 & 0.1\\ 0.1 & 0.9\end{array} \right)^{100}\left(\begin{array}{cc}0.4\\ 0.6\end{array} \right)$$

 

[lemontree45]

Hi,
Thanks for the response.

Actually, I want to generate a sequence of states either 1 or 0. like 1011111000010000...(100 states) using this state transition probability matrix.

Any idea?

 

[micromass]

You're going to have to give some more information about why you need this kind of thing.
Any sequence of 0's and 1's is possible from these transistion matrix, even 1111111...
or 0000000...

Or do you want to make a (Monte-Carlo) simulation of this Markov chain?? To do this:

• Take a number at random from 1-10, if you chose 1-4 you start in state 0 otherwise you start in state 1.
• Take a number at random from 1-10. If you chose 1 then you change your state, if you chose otherwise then you keep your state.
• Do the previous a 100 times.

More information from you would be very helpful!

 

[lemontree45]

This is what I wanted to find out.

given a state sequence with states '1' and '2'. and the #2 is what I am trying to solve.

1. Derive from this sequence the parameters for a first order markov chain: the state transition probability matrix and the state probability vector. The state frame length should be 1m.

2. Implement a state sequence generator based on the first order markov chain, using the parameters from 1. (it is a realization of a 1st order markov chain using the state transition probability matrix)

3. Use the generator and realize a markov chain with 1000000 states.

4. Generate the state duration statistic of the measured state sequence and the resimulated state sequence for the "good" and "bad" state. (x-axis: state duration in meter; y-axis: probability of the state duration)

5. Generate the theoretical state duration probability function of the 1st-order Markov chain for comparison.

 

[micromass]

That's exactly the Monte-Carlo simulation I described above. Use a computer program to simulate it.

 

[lemontree45]

Thanks :)