Markov chain on state {1, 2, 3, 4, 5, 6 , 7}

Click For Summary

Homework Help Overview

The discussion revolves around a Markov chain defined on the state space {1, 2, 3, 4, 5, 6, 7}. The original poster seeks assistance for a programming project related to the long-term behavior of the chain, particularly focusing on the fraction of time spent in state 3.

Discussion Character

  • Exploratory, Assumption checking, Problem interpretation

Approaches and Questions Raised

  • Participants inquire about the original poster's attempts and clarify the meaning of "in the long run." There is discussion about the geometric distribution of returns to state 1 and its implications for the problem. Questions are raised regarding the necessity of proving certain parameters and the relevance of the transition matrix.

Discussion Status

The discussion is ongoing, with participants exploring different interpretations of the problem and questioning the assumptions made by the original poster. Some guidance has been offered regarding the need for attempts to simplify the state diagram in relation to the question posed.

Contextual Notes

There is mention of a transition matrix that is sparse, indicating that many transitions are not defined. The original poster expresses uncertainty about how the provided information fits into the problem context.

Janji
Messages
5
Reaction score
0
Member warned that some effort must be shown
Homework Statement
Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
Relevant Equations
In the long run, what fraction of the time does the chain spend in state 3?
I need this for a programming project. Could you help?
7_reducible.png
 
Last edited by a moderator:
Physics news on Phys.org
What have you tried? What do you mean formally with "in the long run"?
 
Math_QED said:
What have you tried? What do you mean formally with "in the long run"?
In the long run (n→∞):
 
Janji said:
Homework Statement:: Let's suppose the chain starts at state 1. The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
Relevant Equations:: In the long run, what fraction of the time does the chain spend in state 3?

I need this for a programming project. Could you help?View attachment 260957
The diagram can be represented by a transition matrix. For this problem it is a 6 x 6 sparse matrix; i.e., most of the entries are 0 since many transitions aren't defined. To find the long-term behavior, you look at ##\lim_{n \to \infty}A^n##, where A is the transition matrix.

Janji said:
The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
It's been many years since I've done problems like this -- I don't know how this information fits into the problem.
 
Janji said:
The distribution of the number of times that the chain returns to state 1 is geometric and the parameter would be 1/4.
Is that given or something to be proved? If given it would seem redundant - all the info is in the initial state and the diagram.
Janji said:
In the long run, what fraction of the time does the chain spend in state 3?
You must show some attempt.
Can you see how simplify the state diagram in respect of this question?
 

Similar threads

Replies
24
Views
4K
  • · Replies 4 ·
Replies
4
Views
2K
Replies
12
Views
4K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 7 ·
Replies
7
Views
5K
  • · Replies 3 ·
Replies
3
Views
3K
Replies
3
Views
2K
Replies
11
Views
2K
Replies
6
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K