# When are Markov processes non-reversible?

• carllacan
In summary, understanding the concept of reversibility is crucial when dealing with Markov processes, and the condition you found is just one aspect of it.
carllacan
I had to solve a problem in which I had a Markov process described by a transition matrix with elements $a _{ij} = P(X_t =x_j | X_{t-1} = x_i)$, where $X_t$ is the state at time t and $x_n$ are the possible states of the system.

I was asked to find, given the state of the system at a time t, the probability of the system having been in a certain state at time t-1. Using the Bayes Rule I found how to write a matrix $B$ that represents this reversed process:
$b_{ij} = P(X_{t-1} = x_i| X_t =x_j) =\frac{P(X_t =x_j | X_{t-1} = x_i) P(X_{t-1} = x_i)}{(P(X_t =x_j )} = \frac{a_{ji} \pi _j}{\pi _i}$, where $\vec{\pi}$ is the stationary distribution vector.

Trying to check my result I searched on google for "reversed markov processes" and I found that $a_{ij}\pi_{i} = a_{ji} \pi _j$ it the condition for a Markov process being reversible, and agrees with my solution when A = B. Should I interpret this as that the mean that a Markov process is reversible only if the matrix describing it is the same that describes the reversed process?

it is important to be critical and analyze the information you find, especially when it comes to interpreting mathematical concepts like Markov processes. In this case, it is important to understand the concept of reversibility in relation to Markov processes.

First, let's define a reversible Markov process. A Markov process is considered reversible if the probability of transitioning from state i to state j is equal to the probability of transitioning from state j to state i. In other words, the process is reversible if the transition matrix is symmetrical.

Now, let's look at your solution and the condition you found on Google. Your solution shows that for a Markov process to be reversible, the matrix A (describing the forward process) must be equal to the matrix B (describing the reversed process). This is because the condition for reversibility is that a_{ij}\pi_{i} = a_{ji} \pi _j, which is exactly what your solution shows.

However, this does not mean that a Markov process is only reversible if the transition matrix is the same for both the forward and reversed processes. It is possible for a Markov process to be reversible even if the transition matrix is different for the forward and reversed processes. In fact, there are many examples of reversible Markov processes where the transition matrix is not symmetrical.

So, to answer your question, you should not interpret this as the only condition for a Markov process to be reversible. Reversibility is a concept that applies to the transition matrix of a Markov process, and it can have various conditions and implications. It is important to thoroughly understand the concept and its applications before making any interpretations.

## 1. What is a Markov process?

A Markov process is a stochastic process in which the future state of the system depends only on the present state and not on the sequence of events that preceded it.

## 2. What does it mean for a Markov process to be reversible?

A Markov process is reversible if the process can be run backwards in time and still maintain the same probabilities for transitioning between states.

## 3. When are Markov processes non-reversible?

Markov processes are non-reversible when there is an asymmetry in the transition probabilities between states, such as when certain transition probabilities are zero or when there are more possible transitions from one state than from another.

## 4. How is reversibility related to equilibrium in a Markov process?

In a reversible Markov process, the system eventually reaches a state of equilibrium where the probabilities of being in each state remain constant over time. This equilibrium state is not dependent on the initial state of the system.

## 5. Can non-reversible Markov processes still reach equilibrium?

Yes, non-reversible Markov processes can still reach equilibrium, but the equilibrium state will depend on the initial state of the system and may not be the same as in a reversible process.

Replies
1
Views
1K
Replies
1
Views
244
Replies
3
Views
617
Replies
4
Views
2K
Replies
5
Views
1K
Replies
1
Views
1K
Replies
4
Views
2K
Replies
13
Views
3K
Replies
13
Views
2K
Replies
1
Views
2K