Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Markov Chains and Limit question : Where have I gone wrong?

  1. Jan 3, 2012 #1
    I am doing question 2 here:


    My solution so far is this:


    Where have I gone wrong? I'm assuming I'm taking the wrong limit but I'm really stumped here and no idea what else it should be..

    I hope my writing is ok, I tried to be as neat as possible!

  2. jcsd
  3. Jan 4, 2012 #2
    Your indexes k and n are not independent, but it looks like you have "independent" limits and terms in the sums, although I believe you have taken the limit with k = n. If I understood correctly, you should have k = n-3. If you plug this in you should get something else.
  4. Jan 4, 2012 #3
    I did initially have k = n-3, but I swapped to k because I thought the n's on the LHS and the RHS were different (and independant), and it was confusing me.

    Putting k = n-3 does it really change anything? I can't see what it changes..

  5. Jan 4, 2012 #4
    I'm sorry, it doesn't change a thing. But now I noticed that the sum isn't calculated correctly, you should change k back to n-3 so you can see the flaw. The sum is actually
    [itex] \sum_{n=3}^{\infty}\left(\frac{n}{n+1}\right)^{a(n-3)}-\left(\frac{n}{n+1}\right)^{a(n-2)}[/itex] so the cancellations are not correct.
    Last edited: Jan 4, 2012
  6. Jan 4, 2012 #5
    starting at n=3 I get (1 - Na) : n=4 I get (Na - N2a)


    Isn't it the same cancellation?
  7. Jan 4, 2012 #6
    Aren't consecutive terms at indexes n, n+1 like so:

    [itex]a_n= \left(\frac{n}{n+1}\right)^{a(n-3)}-\left(\frac{n}{n+1}\right)^{a(n-2)}[/itex]
    [itex]a_{n+1}= \left(\frac{n+1}{n+2}\right)^{a(n-2)}-\left(\frac{n+1}{n+2}\right)^{a(n-1)}[/itex]?
  8. Jan 4, 2012 #7
    Ah of course, woops!

    where do I go from here, I assume I can't cancel any of them, right?

    Not sure how to sum that series either..
  9. Jan 4, 2012 #8
    Yes, it looks like a very tough series to me. I don't know where to go from there. I don't know anything about Markov chains so I don't know how you calculate the [itex]f_n^i[/itex] 's. Did you calculate them by [itex]T^{(n)}=T(1)\cdots T(n)?[/itex]
    cause it looks to me like you should have [itex]f_3^3 = 1/2 \cdot 1 \cdot (1-\frac{3}{3+1})^a[/itex]
    [itex]f_3^4 = 1/2 \cdot 1 \cdot (\frac{3}{3+1})^a \cdot (1-\frac{4}{4+1})^a[/itex]
    [itex]f_3^5 = 1/2 \cdot 1 \cdot (\frac{3}{3+1})^a \cdot (\frac{4}{4+1})^a \cdot (1-\frac{5}{5+1})^a[/itex]
    and so forth. Mayby I'm not interpreting the probabilities correctly, as I said, I'm not familiar with these...
  10. Jan 4, 2012 #9

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    If the system is in state 5 at time n, the probability that it remains in state 5 at times n+1, n+2,...,n+k is
    [tex] \left(\frac{n}{n+1}\right)^a \left(\frac{n+1}{n+2}\right)^a \cdots \left(\frac{n+k-1}{n+k}\right)^a = \left( \frac{n}{n+k}\right)^a, [/tex]
    which --> 0 as k --> infinity. You can fix up this argument to show that [itex] \Pr \{ X_{n+k}=5 \; \mbox{ i.o. } |X_n = 5 \} = 0, [/itex], so state 5 is left with probability 1 at a finite time. When that happens, the system returns to state 3, implying that state 3 is recurrent.

  11. Jan 4, 2012 #10
    If it is calculated like I suspect it is, then you will get a satisfying answer.
    So first we want to find out what [itex]f_3^n[/itex] is when [itex]n\geq 3:[/itex]

    [itex]f_3^n = \displaystyle \frac{1}{2}\cdot1\cdot\left(\prod_{k=3}^{n-1}\left(\frac{k}{k+1}\right)^a\right)\cdot\left(1-\left(\frac{n}{n+1}\right)^a\right) = \frac{1}{2}\cdot\left(\frac{(n-1)!}{2}\cdot\frac{2\cdot3}{n!}\right)^a\cdot\left(1-\left(\frac{n}{n+1}\right)^a\right)[/itex]
    [itex] \displaystyle = \frac{1}{2} \cdot \left( \frac{3}{n} \right)^a \cdot \left(1-\left( \frac{n}{n+1} \right)^a \right) = \frac{3^a}{2} \cdot \left(\frac{1}{n^a}- \frac{1}{(n+1)^a}\right)[/itex]
    So now the sum of the probabilities is easy to calculate, and I believe that you will finally get the correct answer.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook