Simulating a discrete time markov chain

[bradyj7]

Hi,

I'm trying to simulate a discrete time time markov chain in matlab. Unfortunately I am neither a markov chain expert or a good MATLAB coder.

My problem is simple, I have a transition probability matrix with 100 states (100x100) and I want to simulate a 1000 steps beginning from state 1.

This is the code I have. I'd be grateful if somebody had time to look at the comments in the code to see if the code is correct for simulating a markov chain.

Kind Regards

Use a for-loop to loop n times for length you want. 

transC = cumsum(trans,2); %cumulative sum of rows, we will use this to decide on the next step.
n = 1000;
states = zeros(1,n); %storage of states
states(1) = 1; %start at state 1 (or whatever)
for ii = 2:n
   %Generate a random number and figure out where it falls in the cumulative sum of that state's trasition matrix row
   [~,states(ii)] = histc(rand,transC(states(ii-1),:));
end

 

[kmwest]

I don't know Matlab but I made a discrete markov function in R, here is the general outline:

Define transition matrix (let be named t_mat)

Define "result" matrix to be used during matrix algebra, initially set this res_mat= t_mat

Let n = number of transitions

Let i=1
For i=1 to n:
{
int_mat = res_mat* t_mat (using matrix mltiplication operator)
i = i +1
}

Return res_mat at the end, it will be the result of the transition matrix multiplied by itself n times.

If you want to do it in R let me know and I can post the code (it's on a different computer).

 

[bpet]

...
This is the code I have. I'd be grateful if somebody had time to look at the comments in the code to see if the code is correct for simulating a markov chain.
...

Whether or not the code is "correct", it's good programming practice to test thoroughly. For example:

1. Do the results make sense for small transition matrices such as eye(1) or [0.5,0.5;0,1]?
2. Do the results change when using a different function for sampling from arbitrary discrete distributions, such as datasample in the stats toolbox or randraw on the file exchange?
3. Is the program fast enough for the intended purposes?

 

[bradyj7]

Thanks for your comments.

 

[chiro]

Hey bradyj7.

Do you want to simulate a tonne of paths or some particular constrained ones?