1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Markov Chain - Time Reversibility proof

  1. Dec 3, 2015 #1
    1. The problem statement, all variables and given/known data

    Let X = {Xn : n ≥ 0} be an irreducible, aperiodic Markov chain with finite state space S, transition matrix P, and stationary distribution π. For x,y ∈ R|S|, define the inner product ⟨x,y⟩ = ∑i∈S xiyiπi, and let L2(π) = {x ∈ R|S| : ⟨x,x⟩ < ∞}. Show that X is time-reversible if and only if ⟨x, Py⟩ = ⟨Px, y⟩ for all x, y ∈ L2(π).

    2. Relevant equations

    X is reversible if and only if

    qij = pij
    where qij is the transition probability from state i to state j in the reverse chain.
    pij = pjiπj / πi
    πipij = πjpji.

    3. The attempt at a solution

    I tried equating ⟨x, Py⟩ = ⟨Px, y⟩ and subbing in the definition of the inner product as defined in the question. However, that doesn't seem to lead to anywhere.

    Thank you for all those who can help.
  2. jcsd
  3. Dec 3, 2015 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    You cannot receive help until you show your work on the problem. What have you actually done? Don't just describe it in words; write down formulas, equations, etc. Perhaps you wrote incorrect formulas; we cannot know if you don't show us.
  4. Dec 3, 2015 #3
    The attempt at solution:

    x, Py⟩ = ⟨Px, y
    ixi(Pyi = ∑i(Px)iyiπi
    i xi [Py]i πi = ∑i [Px]i yi πi

    However I don't know what is the form of the P matrix and thus the form of the matrix product Px and Py.
  5. Dec 3, 2015 #4

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    You don't know the formula for the components of ##\mathbf{Px}##? Then how can you have studied anything about Markov chains, where those formulas are used extensively?
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted