MHB Understanding Markov Chains: Transition Matrix and State Space Explained

  • Thread starter Thread starter Poirot1
  • Start date Start date
Click For Summary
The discussion focuses on the transition matrix and state space for a simple random walk with absorbing barriers at states 1 and 5. A simple random walk involves equal probabilities of moving to adjacent states, either +1 or -1. The transition matrix provided illustrates how the probabilities are structured, with absorbing barriers represented by rows of zeroes. Specifically, states 1 and 5 are absorbing, meaning once reached, the process cannot leave these states. Understanding this matrix is crucial for analyzing the behavior of the random walk.
Poirot1
Messages
243
Reaction score
0
What is the transition matrix and state space corresponding to a simple random random walk with absorbing barriers at 1 and 5? I know an absorbing barrier will correspong to a row of zeroes but I don't know what a simple random walk is.Thanks
 
Physics news on Phys.org
Poirot said:
What is the transition matrix and state space corresponding to a simple random random walk with absorbing barriers at 1 and 5? I know an absorbing barrier will correspong to a row of zeroes but I don't know what a simple random walk is.Thanks


Equal probability of +1, -1.

CB
 
Sorry I don't understand what you mean. Can you give me the matrix?
 
Poirot said:
Sorry I don't understand what you mean. Can you give me the matrix?

Something like:

\[A=\left[ \begin{array}{ccccc}1& 0 & 0 & 0 & 0 \\ 0.5 & 0 & 0.5 & 0 & 0 \\ 0 & 0.5 & 0 & 0.5 & 0
\\ 0 & 0 & 0.5 & 0 & 0.5 \\ 0 & 0 & 0 & 0 & 1 \end{array} \right] \]

CB
 

Thread 'A variant of the Monty Hall problem'

There is a nice little variation of the problem. The host says, after you have chosen the door, that you can change your guess, but to sweeten the deal, he says you can choose the two other doors, if you wish. This proposition is a no brainer, however before you are quick enough to accept it, the host opens one of the two doors and it is empty. In this version you really want to change your pick, but at the same time ask yourself is the host impartial and does that change anything. The host...
View full post »

Similar threads

MHB Regarding mixing time of a Markov chain
  • · Replies 2 ·
Replies
2
Views
2K
MHB Optimizing Markov Chain Production System Throughput w/ Exponential Rate of 50
  • · Replies 1 ·
Replies
1
Views
2K
MHB How Can Uniform Random Variables Simulate a Markov Chain?
  • · Replies 3 ·
Replies
3
Views
2K
Simulating a discrete time markov chain
  • · Replies 4 ·
Replies
4
Views
5K
The infinite limits of the probability transition matrix for Markov chain
  • · Replies 24 ·
Replies
24
Views
4K
MHB Markov Chain - Is state 2 periodic?
  • · Replies 4 ·
Replies
4
Views
2K
I Jump probability of a random walker
  • · Replies 11 ·
Replies
11
Views
2K
A Markov Chain as a function of dimensions
  • · Replies 1 ·
Replies
1
Views
1K
MHB How Do You Calculate Long-Term Proportions in a Markov Chain?
  • · Replies 1 ·
Replies
1
Views
2K
MHB Solving Transition Matrix: Octopus Training
  • · Replies 20 ·
Replies
20
Views
5K