Understanding Markov Chains: Transition Matrix and State Space Explained

  • Context: MHB 
  • Thread starter Thread starter Poirot1
  • Start date Start date
Click For Summary

Discussion Overview

The discussion revolves around the transition matrix and state space for a simple random walk with absorbing barriers at specific states. Participants seek to clarify the concept of a simple random walk and how it relates to the transition matrix, particularly in the context of absorbing barriers.

Discussion Character

  • Technical explanation
  • Conceptual clarification
  • Debate/contested

Main Points Raised

  • One participant inquires about the transition matrix and state space for a simple random walk with absorbing barriers at states 1 and 5, expressing uncertainty about the concept of a simple random walk.
  • Another participant mentions that the random walk has equal probabilities of moving +1 or -1.
  • There are requests for clarification regarding the transition matrix, indicating a lack of understanding of the initial explanation.
  • A proposed transition matrix is presented, but it is unclear if all participants agree on its correctness or applicability.

Areas of Agreement / Disagreement

Participants appear to have differing levels of understanding regarding the simple random walk and the corresponding transition matrix. There is no consensus on the explanation provided, and the discussion remains unresolved.

Contextual Notes

Some participants express confusion about the definitions and implications of the absorbing barriers and the structure of the transition matrix. The discussion does not clarify all assumptions or mathematical steps involved in deriving the matrix.

Poirot1
Messages
243
Reaction score
0
What is the transition matrix and state space corresponding to a simple random random walk with absorbing barriers at 1 and 5? I know an absorbing barrier will correspong to a row of zeroes but I don't know what a simple random walk is.Thanks
 
Physics news on Phys.org
Poirot said:
What is the transition matrix and state space corresponding to a simple random random walk with absorbing barriers at 1 and 5? I know an absorbing barrier will correspong to a row of zeroes but I don't know what a simple random walk is.Thanks


Equal probability of +1, -1.

CB
 
Sorry I don't understand what you mean. Can you give me the matrix?
 
Poirot said:
Sorry I don't understand what you mean. Can you give me the matrix?

Something like:

\[A=\left[ \begin{array}{ccccc}1& 0 & 0 & 0 & 0 \\ 0.5 & 0 & 0.5 & 0 & 0 \\ 0 & 0.5 & 0 & 0.5 & 0
\\ 0 & 0 & 0.5 & 0 & 0.5 \\ 0 & 0 & 0 & 0 & 1 \end{array} \right] \]

CB
 

Similar threads

MHB Regarding mixing time of a Markov chain
  • · Replies 2 ·
Replies
2
Views
2K
MHB Optimizing Markov Chain Production System Throughput w/ Exponential Rate of 50
  • · Replies 1 ·
Replies
1
Views
2K
MHB How Can Uniform Random Variables Simulate a Markov Chain?
  • · Replies 3 ·
Replies
3
Views
2K
Undergrad Simulating a discrete time markov chain
  • · Replies 4 ·
Replies
4
Views
5K
The infinite limits of the probability transition matrix for Markov chain
  • · Replies 24 ·
Replies
24
Views
4K
MHB Markov Chain - Is state 2 periodic?
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Jump probability of a random walker
  • · Replies 11 ·
Replies
11
Views
2K
Graduate Markov Chain as a function of dimensions
  • · Replies 1 ·
Replies
1
Views
2K
MHB How Do You Calculate Long-Term Proportions in a Markov Chain?
  • · Replies 1 ·
Replies
1
Views
2K
MHB Solving Transition Matrix: Octopus Training
  • · Replies 20 ·
Replies
20
Views
5K