Proof that a stochastic process isn't a Markov Process
1 Attachment(s)
I've been trying to solve this problem for a week now, but haven't been able to. Basically I need to prove that a certain process satisfies ChapmanKolmogorov equations, yet it isn't a Markov Process (it doesn't satisfy the Markovian Property).
I attached the problem as a .doc below. Please, I really need a little help here. 
hi gesteves!
I read your question, and I think it is readily seen to be not markov (because it is easily seen that [tex]P(X_{3(m1)+3}=1X_{3(m1)+2}=1 \mbox{ and }X_{3(m1)+1}=1)[/tex] does not equal [tex]P(X_{3(m1)+3}=1X_{3(m1)+2}=1)[/tex]). In other words, since [tex]X_{3(m1)+3}, X_{3(m1)+2}, X_{3(m1)+1}[/tex] are giving information about the *same draw* from the mth box, most probably these variables are not independent and the proof should take into account of this. Also note that [tex]X_{3(n1)+i}, X_{3(m1)+j}, 1\leq i, j\leq 3[/tex] are independent when m and n are different, as they correspond to different draws. Since [tex]X_{3(n1)+i}, X_{3(m1)+j}, 1\leq i, j\leq 3[/tex] are independent when m and n are different, then [tex]P(X_{3(n1)+i}=lX_{3(m1)+j}=k) = P(X_{3(n1)+i}=l) = \frac{1}{2}, 0\leq l,k\leq 1[/tex] for different m and n. As to the case when m and n are the same, it is necessary to calculate the probability explicity. But amazingly you will find that [tex]P(X_{3(m1)+i}=lX_{3(m1)+j}=k)=\frac{1}{2}, 1\leq i,j\leq 3, 0\leq l,k \leq 1[/tex]. For example, [tex]P(X_{3(11)+2}=1X_{3(m1)+1}=1)=P(\mbox{1 or 2 in the first draw}\mbox{1 or 4 in the first draw}) = \frac{1}{2}[/tex]. Since all conditional probabilities are essentially 1/2, I think the assertion thus holds. 
Hi Wong,
Thanks for your quick reply! If I understood correctly, all I need to prove that it isn't a Markov Process is a counterexample that shows that [tex]P(X_{3(m1)+3}=1X_{3(m1)+2}=1 \mbox{ and }X_{3(m1)+1}=1)[/tex] doesn't equal [tex]P(X_{3(m1)+3}=1X_{3(m1)+2}=1)[/tex]. For m = 1, [tex]P(X_{3}=1X_{2}=1 \mbox{ and }X_{1}=1) = 0[/tex] and [tex]P(X_{3}=1X_{2}=1)=1/2[/tex]. Therefore it isn't a Markov Process. But how can I prove that it satisfies ChapmanKolmogorov? I'll try to prove it on my own, but I could use some pointers. Thanks in advance. 
Yes, gesteves, you got the nonmarkov part.
As for the ChapmanKolmogorov part, you may first think of the form of the equation. If I am not mistaken, the ChapmanKolmogorov equation says that [tex]P(X_{m+n+l}=iX_{l}=j) = \sum_{k}P(X_{m+n+l}=iX_{m+l}=k)P(X_{m+l}=kX_{l}=j)[/tex]. In my first post, I already gave you the various conditional probabilities for the equation. You may just "plug in" and see whether the LHS agrees with the RHS. 
I finally finished it. Thanks for all your help.

All times are GMT 5. The time now is 10:38 AM. 
Powered by vBulletin Copyright ©2000  2014, Jelsoft Enterprises Ltd.
© 2014 Physics Forums