1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Show all other states are transient in Markov chain

  1. Jun 26, 2011 #1
    Let X be a Markov chain with a state s that is absorbing, i.e. pss(1) = 1. All other states
    communicate with s i.e. i → s for all states i ∈ S. Show that all states in S except s are
    transient.

    I understand this intuitively, but I'm not really sure how to start the proof.
     
  2. jcsd
  3. Jun 26, 2011 #2

    lanedance

    User Avatar
    Homework Helper

    write down a transition matrix (which can be reasonably arbitrary, make the absorbing state the first for simplicity) and consider the applying it again and again, and think what happens in the limit...
     
  4. Jun 27, 2011 #3
    Ok so I wrote down the transition matrix with s=1.
    \begin{bmatrix}
    \begin{array}{cc}
    1 & 0 & ... & 0 \\
    p_{21} & p_{22} & ... & p_{2N} \\
    \vdots & \vdots & \ddots & \vdots \\
    p_{N1} & p_{N2} & ... & p_{NN} \\
    \end{array}
    \end{bmatrix}

    I then raised it to the nth power
    \begin{bmatrix}
    \begin{array}{cc}
    1 & 0 & ... & 0 \\
    p_{21}(n) & p_{22}(n) & ... & p_{2N}(n) \\
    \vdots & \vdots & \ddots & \vdots \\
    p_{N1}(n) & p_{N2}(n) & ... & p_{NN}(n) \\
    \end{array}
    \end{bmatrix}

    So I guess I have to show [tex] \lim_{n\rightarrow \infty} p_{ij}(n)=0 \hspace{3mm} \forall i,j>1[/tex]

    I'm not really to sure where to go from there.
     
  5. Jun 27, 2011 #4

    lanedance

    User Avatar
    Homework Helper

    not quite [tex] p_{ij}^{(n)} [/tex]will represent the probability of transitioning from state i at time 0, to state j at time n. So you would expect
    [tex] p_{ij}^{(n)} = 1 / / i=1[/tex]
    [tex] p_{ij}^{(n)} = 0 / / i \neq 1[/tex]


    I would start by looking at P^2, P^3 to get a feeling for the form and what happens and try and generalise from there

    keep in mind each row of P, P^2 and so forth must sum to one...
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Show all other states are transient in Markov chain
  1. Markov chains (Replies: 0)

  2. Markov Chain (Replies: 1)

Loading...