Random process derived from Markov process

In summary, if the function ρ(t) is increasing then the limit lim_{dt->0} Pr{}/ρ(t+dt)-ρ(t) holds.
  • #1
Mubeena
2
0
I have a query on a Random process derived from Markov process. I have stuck in this problem for more than 2 weeks.
Let [itex]r(t)[/itex] be a finite-state Markov jump process described by
\begin{alignat*}{1}
\lim_{dt\rightarrow 0}\frac{Pr\{r(t+dt)=j/r(t)=i\}}{dt} & =q_{ij}
\end{alignat*}
when [itex]i \ne j[/itex], and where [itex]q_{ij}[/itex] is the transition rate and represents the probability per time unit that [itex]r(t)[/itex] makes a transition from state $i$ to a
state $j$. Now, let [itex]r(\rho(t))[/itex] be a random process derived from [itex]r(t)[/itex] depending on a parameter [itex]\rho(t)[/itex], which is defined by
\begin{alignat*}{1}
\frac{d}{dt}\rho(t)=f(r(\rho(t))),\qquad\rho(0)=0
\end{alignat*}
Here [itex]f(.)[/itex] is a piecewise continuous function depending on [itex]r(\rho(t))[/itex]
with range space as [itex]\mathbb{R}[/itex], a set of Real numbers. In this case can we describe the random process [itex]r(\rho(t))[/itex] as
\begin{alignat*}{1}
\lim_{dt\rightarrow 0}\frac{\mathrm{Pr}\{r(\rho(t+dt))=j/r(\rho(t))=i\}}{\rho(t+dt)-\rho(t)} =q_{ij},\qquad i\ne j\\
\end{alignat*}
 
Physics news on Phys.org
  • #2
I'll try to fix up the question a little:

Let [itex]r(t)[/itex] be a finite-state Markov jump process described by
\begin{alignat*}{1}
\lim_{dt\rightarrow 0}\frac{Pr\{r(t+dt)=j \ | \ r(t)=i\}}{dt} & =q_{ij}
\end{alignat*} when [itex]i \ne j[/itex], and where [itex]q_{ij}[/itex] is the transition rate and represents the probability per time unit that [itex]r(t)[/itex] makes a transition from state [itex]i[/itex] to a state [itex]j[/itex].

For a given real valued piecewise continuous function [itex] f() [/itex], define [itex]\rho(t)[/itex] by
\begin{alignat*}{1}
\frac{d}{dt}\rho(t)=f(r(\rho(t))),\qquad\rho(0)=0
\end{alignat*}

Does the random process [itex]r(\rho(t))[/itex] satisfy the following?:

\begin{alignat*}{1}
\lim_{dt\rightarrow 0}\frac{\mathrm{Pr}\{r(\rho(t+dt))=j \ | \ r(\rho(t))=i\}}{\rho(t+dt)-\rho(t)} =q_{ij},\qquad i\ne j\\
\end{alignat*}
 
Last edited:
  • #3
Hey Mubeena and welcome to the forums.

For this proposition, I have a gut feeling it is true but only if you have specific conditions on the monotonic behavior of p(t).

If you have something that goes up and down then essentially you are screwing up with the ordering of the conditional statement since the Markovian aspect depends on time t with time t + dt and if p(t) starts decreasing then it screws up this forward attribute in time for the conditional distribution and things "reverse".

In short if p(t) is decreasing then p(t+dt) < p(t).

If my reasoning holds, then my best guess is that you can show that the Markovian condition fails because of the above.
 
  • #4
Hi Stephen and Chiro,
Thank you very much for your help.
By your arguments, If I assume ρ(t) to be monotonically increasing by assuming f(r(ρ(t)))>0, for all t, then the last equality (lim_{dt->0} Pr{}/ρ(t+dt)-ρ(t)=q_ij) holds right?
 
  • #5
If you want to prove it, you will need to show that the limit has the same form when the function is monotonic.

Once you formalize this in definitions you should be OK (I think).
 

1. What is a Markov process?

A Markov process is a mathematical model that describes a sequence of random events, where the probability of the next event depends only on the current state and not on the previous states. It is also known as a memoryless process.

2. How is a random process derived from a Markov process?

A random process derived from a Markov process is created by taking a set of states and defining a probability transition matrix that describes the probability of transitioning from one state to another. This matrix is then used to generate a sequence of random events.

3. What is the difference between a Markov process and a random process derived from a Markov process?

A Markov process is a mathematical model that describes a sequence of random events, while a random process derived from a Markov process is a specific sequence of events generated by the probability transition matrix defined for the Markov process. In other words, a random process derived from a Markov process is an application of the Markov process model.

4. What are some real-world applications of random processes derived from Markov processes?

Random processes derived from Markov processes are commonly used in various fields such as finance, economics, biology, and computer science. They can be used to model stock prices, population dynamics, gene regulation, and network traffic, among others.

5. How are Markov processes and random processes derived from Markov processes evaluated or tested?

There are various statistical tests and measures that can be used to evaluate the performance and accuracy of a Markov process or a random process derived from a Markov process. These include measures such as the mean, variance, and autocorrelation of the generated sequences and statistical tests such as the chi-square test and the Kolmogorov-Smirnov test.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
9
Views
4K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • Precalculus Mathematics Homework Help
Replies
4
Views
718
Replies
3
Views
247
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
823
  • Advanced Physics Homework Help
Replies
0
Views
197
  • Advanced Physics Homework Help
Replies
4
Views
699
  • Set Theory, Logic, Probability, Statistics
Replies
10
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
11
Views
1K
Back
Top