Transition Matrix for Finite State Random Walk

[Mark53]

Homework Statement

Define a simple random walk Yn on a finite state space S = {0, 1, 2, . . . , N} to be a random process that
• increases by 1, when possible, with probability p,
• decreases by 1, when possible, with probability 1 − p, and
• remains unchanged otherwise.
(a) Specify the transition matrix for Yn.

(b) Assume that N = 2 and initially, the process is evenly distributed across S. Calculate the probability the process is in state 0 after 2 steps.

The Attempt at a Solution

\begin{pmatrix}
1-p & p & 0 \\ 1-p & 0 & p \\ 0 & 1-p & p
\end{pmatrix}\quad

would this matrix be correct not sure about the first entry

b)

Just need to calculate P^2 and see what the probability is in state 0.
Need the correct matrix to do this first

 

[andrewkirk]

Yes that matrix looks like it implements the rules you wrote.

 

[andrewkirk]

The probabilities of increasing one = p and decreasing one = (1-p) total to 1. So there is no possible such thing as "otherwise".

The 'otherwise' is needed because in some cases one cannot do an increase, and in other cases one cannot decrease. The 'otherwise' pushes the unused probability into the diagonal entry in that row. That tells us what to do in rows 0 and 2.

 

[FactChecker]

I stand corrected. I missed the "if possible" and the "Assume N=2". Sorry. I agree that the matrix is correct for N=2.

The 'otherwise' is needed because in some cases one cannot do an increase, and in other cases one cannot decrease. The 'otherwise' pushes the unused probability into the diagonal entry in that row. That tells us what to do in rows 0 and 2.

You are correct. I deleted my wrong post. Sorry.