Markov chain on state {1, 2, 3, 4, 5, 6 , 7}

  • #1
5
0
Member warned that some effort must be shown

Homework Statement:

Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.

Relevant Equations:

In the long run, what fraction of the time does the chain spend in state 3?
I need this for a programming project. Could you help?
7_reducible.png
 
Last edited by a moderator:

Answers and Replies

  • #2
member 587159
What have you tried? What do you mean formally with "in the long run"?
 
  • #3
5
0
What have you tried? What do you mean formally with "in the long run"?
In the long run (n→∞):
 
  • #4
34,553
6,269
Homework Statement:: Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
Relevant Equations:: In the long run, what fraction of the time does the chain spend in state 3?

I need this for a programming project. Could you help?View attachment 260957
The diagram can be represented by a transition matrix. For this problem it is a 6 x 6 sparse matrix; i.e., most of the entries are 0 since many transitions aren't defined. To find the long-term behavior, you look at ##\lim_{n \to \infty}A^n##, where A is the transition matrix.

Janji said:
The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
It's been many years since I've done problems like this -- I don't know how this information fits into the problem.
 
  • #5
haruspex
Science Advisor
Homework Helper
Insights Author
Gold Member
2020 Award
35,277
6,320
The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
Is that given or something to be proved? If given it would seem redundant - all the info is in the initial state and the diagram.
In the long run, what fraction of the time does the chain spend in state 3?
You must show some attempt.
Can you see how simplify the state diagram in respect of this question?
 

Related Threads on Markov chain on state {1, 2, 3, 4, 5, 6 , 7}

  • Last Post
Replies
7
Views
2K
Replies
3
Views
1K
Replies
1
Views
1K
Replies
1
Views
6K
Replies
5
Views
2K
  • Last Post
Replies
5
Views
2K
Replies
0
Views
1K
  • Last Post
Replies
3
Views
1K
Replies
1
Views
1K
Top