Show all other states are transient in Markov chain

Click For Summary

Homework Help Overview

The discussion revolves around a Markov chain with an absorbing state and the goal of demonstrating that all other states are transient. The original poster expresses an intuitive understanding but seeks guidance on how to initiate the proof.

Discussion Character

  • Exploratory, Mathematical reasoning, Assumption checking

Approaches and Questions Raised

  • Participants discuss constructing a transition matrix and analyzing its behavior as it is raised to higher powers. There is a focus on the limit of transition probabilities and the implications for transient states.

Discussion Status

Some participants have provided insights into the structure of the transition matrix and the interpretation of its powers. There is an ongoing exploration of the properties of the transition probabilities, particularly in relation to the absorbing state.

Contextual Notes

There is an emphasis on understanding the behavior of the Markov chain as the number of transitions increases, with specific attention to the probabilities associated with states other than the absorbing state.

xentity1x
Messages
9
Reaction score
0
Let X be a Markov chain with a state s that is absorbing, i.e. pss(1) = 1. All other states
communicate with s i.e. i → s for all states i ∈ S. Show that all states in S except s are
transient.

I understand this intuitively, but I'm not really sure how to start the proof.
 
Physics news on Phys.org
write down a transition matrix (which can be reasonably arbitrary, make the absorbing state the first for simplicity) and consider the applying it again and again, and think what happens in the limit...
 
Ok so I wrote down the transition matrix with s=1.
\begin{bmatrix}
\begin{array}{cc}
1 & 0 & ... & 0 \\
p_{21} & p_{22} & ... & p_{2N} \\
\vdots & \vdots & \ddots & \vdots \\
p_{N1} & p_{N2} & ... & p_{NN} \\
\end{array}
\end{bmatrix}

I then raised it to the nth power
\begin{bmatrix}
\begin{array}{cc}
1 & 0 & ... & 0 \\
p_{21}(n) & p_{22}(n) & ... & p_{2N}(n) \\
\vdots & \vdots & \ddots & \vdots \\
p_{N1}(n) & p_{N2}(n) & ... & p_{NN}(n) \\
\end{array}
\end{bmatrix}

So I guess I have to show [tex]\lim_{n\rightarrow \infty} p_{ij}(n)=0 \hspace{3mm} \forall i,j>1[/tex]

I'm not really to sure where to go from there.
 
not quite [tex]p_{ij}^{(n)}[/tex]will represent the probability of transitioning from state i at time 0, to state j at time n. So you would expect
[tex]p_{ij}^{(n)} = 1 / / i=1[/tex]
[tex]p_{ij}^{(n)} = 0 / / i \neq 1[/tex]I would start by looking at P^2, P^3 to get a feeling for the form and what happens and try and generalise from there

keep in mind each row of P, P^2 and so forth must sum to one...
 

Similar threads

Replies
6
Views
2K
  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 3 ·
Replies
3
Views
7K
Replies
9
Views
3K
Replies
16
Views
2K
Replies
0
Views
1K
  • · Replies 5 ·
Replies
5
Views
3K