Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Proof that a stochastic process isn't a Markov Process

  1. Sep 17, 2004 #1
    I've been trying to solve this problem for a week now, but haven't been able to. Basically I need to prove that a certain process satisfies Chapman-Kolmogorov equations, yet it isn't a Markov Process (it doesn't satisfy the Markovian Property).

    I attached the problem as a .doc below.

    Please, I really need a little help here.

    Attached Files:

  2. jcsd
  3. Sep 18, 2004 #2
    hi gesteves!

    I read your question, and I think it is readily seen to be not markov (because it is easily seen that [tex]P(X_{3(m-1)+3}=1|X_{3(m-1)+2}=1 \mbox{ and }X_{3(m-1)+1}=1)[/tex] does not equal [tex]P(X_{3(m-1)+3}=1|X_{3(m-1)+2}=1)[/tex]). In other words, since [tex]X_{3(m-1)+3}, X_{3(m-1)+2}, X_{3(m-1)+1}[/tex] are giving information about the *same draw* from the mth box, most probably these variables are not independent and the proof should take into account of this.

    Also note that [tex]X_{3(n-1)+i}, X_{3(m-1)+j}, 1\leq i, j\leq 3[/tex] are independent when m and n are different, as they correspond to different draws.

    Since [tex]X_{3(n-1)+i}, X_{3(m-1)+j}, 1\leq i, j\leq 3[/tex] are independent when m and n are different, then [tex]P(X_{3(n-1)+i}=l|X_{3(m-1)+j}=k) = P(X_{3(n-1)+i}=l) = \frac{1}{2}, 0\leq l,k\leq 1[/tex] for different m and n.

    As to the case when m and n are the same, it is necessary to calculate the probability explicity. But amazingly you will find that [tex]P(X_{3(m-1)+i}=l|X_{3(m-1)+j}=k)=\frac{1}{2}, 1\leq i,j\leq 3, 0\leq l,k \leq 1[/tex]. For example, [tex]P(X_{3(1-1)+2}=1|X_{3(m-1)+1}=1)=P(\mbox{1 or 2 in the first draw}|\mbox{1 or 4 in the first draw}) = \frac{1}{2}[/tex].

    Since all conditional probabilities are essentially 1/2, I think the assertion thus holds.
    Last edited: Sep 18, 2004
  4. Sep 18, 2004 #3
    Hi Wong,

    Thanks for your quick reply! If I understood correctly, all I need to prove that it isn't a Markov Process is a counterexample that shows that [tex]P(X_{3(m-1)+3}=1|X_{3(m-1)+2}=1 \mbox{ and }X_{3(m-1)+1}=1)[/tex] doesn't equal [tex]P(X_{3(m-1)+3}=1|X_{3(m-1)+2}=1)[/tex]. For m = 1, [tex]P(X_{3}=1|X_{2}=1 \mbox{ and }X_{1}=1) = 0[/tex] and [tex]P(X_{3}=1|X_{2}=1)=1/2[/tex]. Therefore it isn't a Markov Process.

    But how can I prove that it satisfies Chapman-Kolmogorov? I'll try to prove it on my own, but I could use some pointers.

    Thanks in advance.
  5. Sep 18, 2004 #4
    Yes, gesteves, you got the non-markov part.

    As for the Chapman-Kolmogorov part, you may first think of the form of the equation. If I am not mistaken, the Chapman-Kolmogorov equation says that [tex]P(X_{m+n+l}=i|X_{l}=j) = \sum_{k}P(X_{m+n+l}=i|X_{m+l}=k)P(X_{m+l}=k|X_{l}=j)[/tex]. In my first post, I already gave you the various conditional probabilities for the equation. You may just "plug in" and see whether the LHS agrees with the RHS.
  6. Sep 19, 2004 #5
    I finally finished it. Thanks for all your help.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook