Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

When are Markov processes non-reversible?

  1. Feb 23, 2014 #1
    I had to solve a problem in which I had a Markov process described by a transition matrix with elements [itex]a _{ij} = P(X_t =x_j | X_{t-1} = x_i)[/itex], where [itex]X_t[/itex] is the state at time t and [itex]x_n[/itex] are the possible states of the system.

    I was asked to find, given the state of the system at a time t, the probability of the system having been in a certain state at time t-1. Using the Bayes Rule I found how to write a matrix [itex]B[/itex] that represents this reversed process:
    [itex]b_{ij} = P(X_{t-1} = x_i| X_t =x_j) =\frac{P(X_t =x_j | X_{t-1} = x_i) P(X_{t-1} = x_i)}{(P(X_t =x_j )} = \frac{a_{ji} \pi _j}{\pi _i}[/itex], where [itex]\vec{\pi}[/itex] is the stationary distribution vector.

    Trying to check my result I searched on google for "reversed markov processes" and I found that [itex]a_{ij}\pi_{i} = a_{ji} \pi _j[/itex] it the condition for a Markov process being reversible, and agrees with my solution when A = B. Should I interpret this as that the mean that a Markov process is reversible only if the matrix describing it is the same that describes the reversed process?
  2. jcsd
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Can you offer guidance or do you also need help?
Draft saved Draft deleted

Similar Threads - Markov processes reversible Date
I A seemingly simple problem about probability Jan 29, 2018
I Probability of a Stochastic Markov process Dec 27, 2017
A Probability amplitude May 7, 2017
Gauss-Markov process covariance function Oct 29, 2015
Random process derived from Markov process Mar 17, 2013