Solving a first order matrix differential equation

Click For Summary
SUMMARY

This discussion focuses on solving a first-order matrix differential equation related to a continuous-time Markov chain with two states, represented by the generator matrix Q. The stationary distribution is derived as π = (λ/(λ + μ), μ/(λ + μ)), where λ and μ are the transition rates. The transition matrix P(t) is expressed as P(t) = Ce^(Qt), and its eigendecomposition is utilized to compute e^(Qt). The limit of P(t) as t approaches infinity confirms that P_{ij}(t) converges to the stationary distribution π_j.

PREREQUISITES
  • Understanding of continuous-time Markov chains
  • Familiarity with matrix exponentiation and eigendecomposition
  • Knowledge of stationary distributions in stochastic processes
  • Proficiency in LaTeX for mathematical notation
NEXT STEPS
  • Study the properties of continuous-time Markov chains
  • Learn about matrix exponentiation techniques in linear algebra
  • Explore the derivation of stationary distributions in stochastic models
  • Investigate applications of Markov chains in real-world scenarios
USEFUL FOR

Mathematicians, statisticians, and data scientists interested in stochastic processes, as well as students studying Markov chains and differential equations.

docnet
Messages
796
Reaction score
486
Homework Statement
TL;DR: solving for ##P(t)=e^{Qt}## doesn't go as planned, ending up with an answer that is similar to the given answer, but with missing terms in each entry.
Relevant Equations
.
Let X be a continuous-time Markov chain that hops between two states ##\{1, 2\}## with rates ##\lambda, \mu>0##, so its generator is
$$Q = \begin{pmatrix}
-\mu & \mu\\
\lambda & -\lambda
\end{pmatrix}.$$
Solve ##\pi Q = 0## for the stationary distribution, and verify that ##P_{ij}(t)\longrightarrow \pi_j## as ##t\longrightarrow \infty.##
\\\\
By looking at ##Q, ## it is easy to see that ##\text{Row }1 \cdot \lambda +\text{Row }2\cdot \mu =0##. This implies ##\pi = (\lambda,\mu) C## for a constant ##C## such that ##\pi_1+\pi_2=1.## Solving for ##C,## we have
$$(\lambda + \mu)C = 1,$$
$$C = \frac{1}{\lambda + \mu}.$$
And so
$$\pi = \Big(\frac{\lambda}{\lambda + \mu}, \frac{\mu}{\lambda + \mu}\Big).$$
\\\\
We know from Wiki that the transition matrix ##P(t)## is
$$P(t)=Ce^{Qt}$$
where ##C## is a constant matrix that is determined by the initial condition ##P(0)=I.## In order to find ##e^{Qt},## we use the eigendecomposition of ##Qt.## The eigenvectors of ##Q## are
$$\begin{pmatrix}
\mu & -\lambda
\end{pmatrix}
\text{ and }
\begin{pmatrix}
1 & 1
\end{pmatrix}$$
with respective eigenvalues ##-(\mu+\lambda)## and ##0##. And so we have the eigendecomposition
$$
Qt =
\begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}
\begin{pmatrix}
-(\mu+\lambda)t & 0 \\
0 & 0
\end{pmatrix}
\begin{pmatrix}
\lambda & \mu\\
1 & 1
\end{pmatrix}^{-1}.$$
And so,
\begin{align*}
e^{Qt}&=
\begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}
\begin{pmatrix}
e^{-(\mu+\lambda)t} & 0 \\
0 & 0
\end{pmatrix}
\begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}^{-1}\\
&= \begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}
\begin{pmatrix}
\frac{e^{-(\mu+\lambda)t}}{\mu+\lambda} & 0 \\
0 & 0
\end{pmatrix}
\begin{pmatrix}
1 & -1\\
\lambda & \mu
\end{pmatrix}\\
&= \begin{pmatrix}
\frac{\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}&\frac{-\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}\\
\frac{-\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}& \frac{\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}
\end{pmatrix}.
\end{align*}
However, this solution does not satisfy the initial condition ##P(0)=I##. My answer differs from the true solution given by wiki, which is given by Wiki as
$$P(t)=
\begin{pmatrix}
\frac{\lambda}{\mu+\lambda} + \frac{\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}&\frac{\mu}{\mu+\lambda} +\frac{-\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}\\
\frac{\lambda}{\mu+\lambda} +\frac{-\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}& \frac{\mu}{\mu+\lambda} + \frac{\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}
\end{pmatrix}.$$

My solution is lacking the terms
$$\begin{pmatrix}
\frac{\lambda}{\lambda + \mu} & \frac{\mu}{\lambda + \mu}\\
\frac{\lambda}{\lambda + \mu} & \frac{\mu}{\lambda + \mu}
\end{pmatrix}.$$
Which is $$\lim_{t \to \infty}P(t).$$

I've tried for a while and don't see where these terms could have come from, and where I made a mistake in computing ##e^{Qt}.##
 
Physics news on Phys.org
##e^0 = 1 \neq 0##
 
  • Like
Likes   Reactions: docnet
🤣 :doh: How embarrassing..
 
I am just winging it. :headbang:
 
Thank you for pointing out my error.
 
docnet said:
🤣 :doh: How embarrassing..
We all do those things from time to time.
 
  • Like
Likes   Reactions: docnet
$$P(t)=e^{Qt}$$
We use the eigendecomposition of ##Qt.## The eigenvectors of ##Q## are
$$\begin{pmatrix}
\mu & -\lambda
\end{pmatrix}
\text{ and }
\begin{pmatrix}
1 & 1
\end{pmatrix}$$
with respective eigenvalues ##-(\mu+\lambda)## and ##0##. And so we have the eigendecomposition
$$
Qt =
\begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}
\begin{pmatrix}
-(\mu+\lambda)t & 0 \\
0 & 0
\end{pmatrix}
\begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}^{-1}.$$
And so,
\begin{align*}
e^{Qt}&=
\begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}
\begin{pmatrix}
e^{-(\mu+\lambda)t} & 0 \\
0 & e^0
\end{pmatrix}
\begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}^{-1}\\
&=\frac{1}{\mu+\lambda} \begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}
\begin{pmatrix}
e^{-(\mu+\lambda)t} & 0 \\
0 & 1
\end{pmatrix}
\begin{pmatrix}
1 & -1\\
\lambda & \mu
\end{pmatrix}\\
&=\frac{1}{\mu+\lambda} \begin{pmatrix}
\mu & 1\\
-\lambda & 1
\end{pmatrix}
\begin{pmatrix}
e^{-(\mu+\lambda)t} & -e^{-(\mu+\lambda)t}\\
\lambda & \mu
\end{pmatrix}\\
P(t)&=
\begin{pmatrix}
\frac{\lambda}{\mu+\lambda} + \frac{\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}&\frac{\mu}{\mu+\lambda} +\frac{-\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}\\
\frac{\lambda}{\mu+\lambda} +\frac{-\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}& \frac{\mu}{\mu+\lambda} + \frac{\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}
\end{pmatrix}.
\end{align*}
We take the limit of ##P(t)## as ##t\longrightarrow \infty##.
\begin{align*}
\lim_{t \to \infty}P(t)&= \lim_{t \to \infty}\begin{pmatrix}
\frac{\lambda}{\mu+\lambda} + \frac{\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}&\frac{\mu}{\mu+\lambda} +\frac{-\mu e^{-(\mu+\lambda)t}}{\mu+\lambda}\\
\frac{\lambda}{\mu+\lambda} +\frac{-\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}& \frac{\mu}{\mu+\lambda} + \frac{\lambda e^{-(\mu+\lambda)t}}{\mu+\lambda}
\end{pmatrix}\\
&=
\begin{pmatrix}
\frac{\lambda}{\lambda + \mu} & \frac{\mu}{\lambda + \mu}\\
\frac{\lambda}{\lambda + \mu} & \frac{\mu}{\lambda + \mu}
\end{pmatrix}.
\end{align*}
And so ##P_{ij}(t)\longrightarrow \pi_j## as ##t\longrightarrow \infty##.


Edit: ##LATEX## Embedding.
 
Last edited:

Similar threads

Replies
6
Views
2K
  • · Replies 7 ·
Replies
7
Views
1K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 12 ·
Replies
12
Views
3K
  • · Replies 2 ·
Replies
2
Views
1K
Replies
7
Views
2K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 8 ·
Replies
8
Views
2K
Replies
20
Views
2K
  • · Replies 2 ·
Replies
2
Views
1K