# Simulating a discrete time markov chain

In summary, the code is for simulating a discrete time markov chain, but is not correct for that purpose. The program is fast enough for the intended purposes, but may not be correct for small transition matrices.
Hi,

I'm trying to simulate a discrete time time markov chain in matlab. Unfortunately I am neither a markov chain expert or a good MATLAB coder.

My problem is simple, I have a transition probability matrix with 100 states (100x100) and I want to simulate a 1000 steps beginning from state 1.

This is the code I have. I'd be grateful if somebody had time to look at the comments in the code to see if the code is correct for simulating a markov chain.

Kind Regards

Code:
Use a for-loop to loop n times for length you want.

transC = cumsum(trans,2); %cumulative sum of rows, we will use this to decide on the next step.
n = 1000;
states = zeros(1,n); %storage of states
states(1) = 1; %start at state 1 (or whatever)
for ii = 2:n
%Generate a random number and figure out where it falls in the cumulative sum of that state's trasition matrix row
[~,states(ii)] = histc(rand,transC(states(ii-1),:));
end

I don't know Matlab but I made a discrete markov function in R, here is the general outline:

Define transition matrix (let be named t_mat)

Define "result" matrix to be used during matrix algebra, initially set this res_mat= t_mat

Let n = number of transitions

Let i=1
For i=1 to n:
{
int_mat = res_mat* t_mat (using matrix mltiplication operator)
i = i +1
}

Return res_mat at the end, it will be the result of the transition matrix multiplied by itself n times.

If you want to do it in R let me know and I can post the code (it's on a different computer).

...
This is the code I have. I'd be grateful if somebody had time to look at the comments in the code to see if the code is correct for simulating a markov chain.
...

Whether or not the code is "correct", it's good programming practice to test thoroughly. For example:
1. Do the results make sense for small transition matrices such as eye(1) or [0.5,0.5;0,1]?
2. Do the results change when using a different function for sampling from arbitrary discrete distributions, such as datasample in the stats toolbox or randraw on the file exchange?
3. Is the program fast enough for the intended purposes?

Do you want to simulate a tonne of paths or some particular constrained ones?

## 1. What is a discrete time Markov chain?

A discrete time Markov chain is a mathematical model used to describe the behavior of a system that changes over time in a sequence of discrete steps. It is based on the concept of transition probabilities, where the future state of the system depends only on its current state and not on any previous states.

## 2. How is a discrete time Markov chain simulated?

To simulate a discrete time Markov chain, we need to define the initial state of the system and the transition probabilities for each possible state. Then, we use a random number generator to determine the next state based on the transition probabilities. This process is repeated for a desired number of steps or until a specific condition is met.

## 3. What are the applications of simulating a discrete time Markov chain?

Discrete time Markov chains are widely used in various fields such as biology, finance, and engineering to model and analyze complex systems. They are particularly useful in predicting the behavior of systems that involve randomness and uncertainty.

## 4. What are the limitations of simulating a discrete time Markov chain?

One of the main limitations of simulating a discrete time Markov chain is that it assumes the system is memoryless, meaning that the future state only depends on the current state and not on any previous states. This may not accurately reflect real-world systems, where the past can influence the future. Additionally, the accuracy of the simulation depends on the accuracy of the transition probabilities, which may be difficult to estimate in some cases.

## 5. Can a discrete time Markov chain be used to predict the long-term behavior of a system?

Yes, a discrete time Markov chain can be used to predict the long-term behavior of a system. As the number of steps in the simulation increases, the simulated results approach the long-term behavior of the system. However, it is important to note that the accuracy of the predictions may be affected by the limitations mentioned above.

Replies
3
Views
1K
Replies
2
Views
2K
Replies
3
Views
2K
Replies
1
Views
1K
Replies
4
Views
2K
Replies
13
Views
2K
Replies
2
Views
2K
Replies
4
Views
7K
Replies
14
Views
2K
Replies
1
Views
2K