1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: [markov chain] prove that probability equals 6/pi²

  1. Jan 20, 2012 #1
    1. The problem statement, all variables and given/known data

    2. Relevant equations

    3. The attempt at a solution
    I'll shortly explain what my reasoning is so far, but please ignore it if it comes across too jumbled:
    Let P denote the markov matrix associated with this problem, then I think I was able to argue that the probability that is asked for is equal to [itex]1- \sum_{n=1}^{+\infty} P^n(0,0)[/itex] where [itex]P^n(0,0)[/itex] denotes the element in the first row of the first column of the n-th power of the Markov matrix.

    And I then wanted to calculate [itex]P^n(0,0)[/itex] for every n by trying to find a pattern in [itex]P^1(0,0)[/itex], [itex]P^2(0,0)[/itex], [itex]P^3(0,0)[/itex], etc. I think I found one: define [itex]t_n = \left(t_{n-1} + \left( \frac{n+1}{n} \right)^2 \prod_{k=1}^{n+1} x_k \right) x_n[/itex] (with [itex]t_0 = x_1[/itex]) with [itex]x_n = p_{n,n-1}[/itex], then I think [itex]\sum_{n=1}^{+\infty} P^n(0,0) = \sum_{n=0}^{+\infty} t_n[/itex]. But it seems near impossible to prove that 1 minus this expression equals [itex]\frac{6}{\pi^2}[/itex] so I'm probably way off track...

    Any better suggestions? How would you approach this problem instead?

    Attached Files:

  2. jcsd
  3. Jan 20, 2012 #2
    Just an idea. In some circumstances (if [itex]\|P\|\leq 1[/itex]) we have


    So all you need to do know is to invert I-P...
  4. Jan 20, 2012 #3
    *tries to invert infinite-dimensional matrix*
  5. Jan 21, 2012 #4

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    I found the notation confusing at first, but now I see that what you want to prove is that the chain is transient, with Pr{return to 0} = 1-6/pi^2.

    The first order of business is to establish that the chain is transient, with P{return to zero} < 1. Since you have a discrete-time birth-death process, quite a lot is known or knowable about your system. For example, look at the 1995 paper by Van Doorn, freely downloadable from
    (Google search on "birth-death process+discrete time", and look at the 4th entry "Geometric Ergodicity ... "). In particular, Theorem 2.1 may be a starting point. As for the problem of actually computing the return probability, all I can suggest is that you look at the first-passage-probability equations (for end-state 0) and try to solve them, perhaps using z-transform techniques, or something similar.

    Good luck.

    Last edited: Jan 21, 2012
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook