Markov Chains and Limit question : Where have I gone wrong?

  • Thread starter Firepanda
  • Start date
  • #1
430
0
I am doing question 2 here:

2qtzdx4.png


My solution so far is this:

5eg5dw.jpg


Where have I gone wrong? I'm assuming I'm taking the wrong limit but I'm really stumped here and no idea what else it should be..

I hope my writing is ok, I tried to be as neat as possible!

Thanks
 

Answers and Replies

  • #2
Your indexes k and n are not independent, but it looks like you have "independent" limits and terms in the sums, although I believe you have taken the limit with k = n. If I understood correctly, you should have k = n-3. If you plug this in you should get something else.
 
  • #3
430
0
I did initially have k = n-3, but I swapped to k because I thought the n's on the LHS and the RHS were different (and independant), and it was confusing me.

Putting k = n-3 does it really change anything? I can't see what it changes..

Thanks
 
  • #4
I'm sorry, it doesn't change a thing. But now I noticed that the sum isn't calculated correctly, you should change k back to n-3 so you can see the flaw. The sum is actually
[itex] \sum_{n=3}^{\infty}\left(\frac{n}{n+1}\right)^{a(n-3)}-\left(\frac{n}{n+1}\right)^{a(n-2)}[/itex] so the cancellations are not correct.
 
Last edited:
  • #5
430
0
I'm sorry, it doesn't change a thing. But now I noticed that the sum isn't calculated correctly, you should change k back to n-3 so you can see the flaw. The sum is actually
[itex]\sum_{n=3}^\infty{\left(\frac{n}{n+1}\right)^{a(n-3)}-\left(\frac{n}{n+1}\right)^{a(n-2)}[/itex] so the cancellations are not correct.

starting at n=3 I get (1 - Na) : n=4 I get (Na - N2a)

etc

Isn't it the same cancellation?
 
  • #6
Aren't consecutive terms at indexes n, n+1 like so:

[itex]a_n= \left(\frac{n}{n+1}\right)^{a(n-3)}-\left(\frac{n}{n+1}\right)^{a(n-2)}[/itex]
[itex]a_{n+1}= \left(\frac{n+1}{n+2}\right)^{a(n-2)}-\left(\frac{n+1}{n+2}\right)^{a(n-1)}[/itex]?
 
  • #7
430
0
Aren't consecutive terms at indexes n, n+1 like so:

[itex]a_n= \left(\frac{n}{n+1}\right)^{a(n-3)}-\left(\frac{n}{n+1}\right)^{a(n-2)}[/itex]
[itex]a_{n+1}= \left(\frac{n+1}{n+2}\right)^{a(n-2)}-\left(\frac{n+1}{n+2}\right)^{a(n-1)}[/itex]?

Ah of course, woops!

where do I go from here, I assume I can't cancel any of them, right?

Not sure how to sum that series either..
 
  • #8
Yes, it looks like a very tough series to me. I don't know where to go from there. I don't know anything about Markov chains so I don't know how you calculate the [itex]f_n^i[/itex] 's. Did you calculate them by [itex]T^{(n)}=T(1)\cdots T(n)?[/itex]
cause it looks to me like you should have [itex]f_3^3 = 1/2 \cdot 1 \cdot (1-\frac{3}{3+1})^a[/itex]
[itex]f_3^4 = 1/2 \cdot 1 \cdot (\frac{3}{3+1})^a \cdot (1-\frac{4}{4+1})^a[/itex]
[itex]f_3^5 = 1/2 \cdot 1 \cdot (\frac{3}{3+1})^a \cdot (\frac{4}{4+1})^a \cdot (1-\frac{5}{5+1})^a[/itex]
and so forth. Mayby I'm not interpreting the probabilities correctly, as I said, I'm not familiar with these...
 
  • #9
Ray Vickson
Science Advisor
Homework Helper
Dearly Missed
10,706
1,722
If the system is in state 5 at time n, the probability that it remains in state 5 at times n+1, n+2,...,n+k is
[tex] \left(\frac{n}{n+1}\right)^a \left(\frac{n+1}{n+2}\right)^a \cdots \left(\frac{n+k-1}{n+k}\right)^a = \left( \frac{n}{n+k}\right)^a, [/tex]
which --> 0 as k --> infinity. You can fix up this argument to show that [itex] \Pr \{ X_{n+k}=5 \; \mbox{ i.o. } |X_n = 5 \} = 0, [/itex], so state 5 is left with probability 1 at a finite time. When that happens, the system returns to state 3, implying that state 3 is recurrent.

RGV
 
  • #10
If it is calculated like I suspect it is, then you will get a satisfying answer.
So first we want to find out what [itex]f_3^n[/itex] is when [itex]n\geq 3:[/itex]

[itex]f_3^n = \displaystyle \frac{1}{2}\cdot1\cdot\left(\prod_{k=3}^{n-1}\left(\frac{k}{k+1}\right)^a\right)\cdot\left(1-\left(\frac{n}{n+1}\right)^a\right) = \frac{1}{2}\cdot\left(\frac{(n-1)!}{2}\cdot\frac{2\cdot3}{n!}\right)^a\cdot\left(1-\left(\frac{n}{n+1}\right)^a\right)[/itex]
[itex] \displaystyle = \frac{1}{2} \cdot \left( \frac{3}{n} \right)^a \cdot \left(1-\left( \frac{n}{n+1} \right)^a \right) = \frac{3^a}{2} \cdot \left(\frac{1}{n^a}- \frac{1}{(n+1)^a}\right)[/itex]
So now the sum of the probabilities is easy to calculate, and I believe that you will finally get the correct answer.
 

Related Threads on Markov Chains and Limit question : Where have I gone wrong?

Replies
2
Views
2K
Replies
2
Views
1K
  • Last Post
Replies
1
Views
1K
Replies
2
Views
1K
Replies
5
Views
1K
  • Last Post
Replies
0
Views
2K
  • Last Post
Replies
0
Views
1K
  • Last Post
Replies
9
Views
4K
  • Last Post
Replies
4
Views
3K
  • Last Post
Replies
0
Views
2K
Top