Markov chain on state {1, 2, 3, 4, 5, 6 , 7}

Member warned that some effort must be shown

Homework Statement:

Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.

Relevant Equations:

In the long run, what fraction of the time does the chain spend in state 3?
I need this for a programming project. Could you help?

Last edited by a moderator:

Related Precalculus Mathematics Homework Help News on Phys.org
member 587159
What have you tried? What do you mean formally with "in the long run"?

What have you tried? What do you mean formally with "in the long run"?
In the long run (n→∞):

Mark44
Mentor
Homework Statement:: Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
Relevant Equations:: In the long run, what fraction of the time does the chain spend in state 3?

I need this for a programming project. Could you help?View attachment 260957
The diagram can be represented by a transition matrix. For this problem it is a 6 x 6 sparse matrix; i.e., most of the entries are 0 since many transitions aren't defined. To find the long-term behavior, you look at ##\lim_{n \to \infty}A^n##, where A is the transition matrix.

Janji said:
The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
It's been many years since I've done problems like this -- I don't know how this information fits into the problem.

haruspex