Markov chain on state {1, 2, 3, 4, 5, 6 , 7}

[Janji]

Homework Statement

Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.

Relevant Equations

In the long run, what fraction of the time does the chain spend in state 3?

I need this for a programming project. Could you help?

 

[member 587159]

What have you tried? What do you mean formally with "in the long run"?

 

[Janji]

What have you tried? What do you mean formally with "in the long run"?

In the long run (n→∞):

 

[Mark44]

Homework Statement:: Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
Relevant Equations:: In the long run, what fraction of the time does the chain spend in state 3?

I need this for a programming project. Could you help?View attachment 260957

The diagram can be represented by a transition matrix. For this problem it is a 6 x 6 sparse matrix; i.e., most of the entries are 0 since many transitions aren't defined. To find the long-term behavior, you look at $\lim_{n \to \infty}A^n$, where A is the transition matrix.

The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.

It's been many years since I've done problems like this -- I don't know how this information fits into the problem.

 

[haruspex]

The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.

Is that given or something to be proved? If given it would seem redundant - all the info is in the initial state and the diagram.

In the long run, what fraction of the time does the chain spend in state 3?

You must show some attempt.
Can you see how simplify the state diagram in respect of this question?