# Random process derived from Markov process

1. Mar 17, 2013

### Mubeena

I have a query on a Random process derived from Markov process. I have stuck in this problem for more than 2 weeks.
Let $r(t)$ be a finite-state Markov jump process described by
\begin{alignat*}{1}
\lim_{dt\rightarrow 0}\frac{Pr\{r(t+dt)=j/r(t)=i\}}{dt} & =q_{ij}
\end{alignat*}
when $i \ne j$, and where $q_{ij}$ is the transition rate and represents the probability per time unit that $r(t)$ makes a transition from state $i$ to a
state $j$. Now, let $r(\rho(t))$ be a random process derived from $r(t)$ depending on a parameter $\rho(t)$, which is defined by
\begin{alignat*}{1}
\frac{d}{dt}\rho(t)=f(r(\rho(t))),\qquad\rho(0)=0
\end{alignat*}
Here $f(.)$ is a piecewise continuous function depending on $r(\rho(t))$
with range space as $\mathbb{R}$, a set of Real numbers. In this case can we describe the random process $r(\rho(t))$ as
\begin{alignat*}{1}
\lim_{dt\rightarrow 0}\frac{\mathrm{Pr}\{r(\rho(t+dt))=j/r(\rho(t))=i\}}{\rho(t+dt)-\rho(t)} =q_{ij},\qquad i\ne j\\
\end{alignat*}

2. Mar 18, 2013

### Stephen Tashi

I'll try to fix up the question a little:

Last edited: Mar 18, 2013
3. Mar 18, 2013

### chiro

Hey Mubeena and welcome to the forums.

For this proposition, I have a gut feeling it is true but only if you have specific conditions on the monotonic behavior of p(t).

If you have something that goes up and down then essentially you are screwing up with the ordering of the conditional statement since the Markovian aspect depends on time t with time t + dt and if p(t) starts decreasing then it screws up this forward attribute in time for the conditional distribution and things "reverse".

In short if p(t) is decreasing then p(t+dt) < p(t).

If my reasoning holds, then my best guess is that you can show that the Markovian condition fails because of the above.

4. Mar 19, 2013

### Mubeena

Hi Stephen and Chiro,
Thank you very much for your help.
By your arguments, If I assume ρ(t) to be monotonically increasing by assuming f(r(ρ(t)))>0, for all t, then the last equality (lim_{dt->0} Pr{}/ρ(t+dt)-ρ(t)=q_ij) holds right?

5. Mar 19, 2013

### chiro

If you want to prove it, you will need to show that the limit has the same form when the function is monotonic.

Once you formalize this in definitions you should be OK (I think).

Share this great discussion with others via Reddit, Google+, Twitter, or Facebook