Need assistance figuring out a model PDF

AI Thread Summary
The discussion revolves around modeling a system with three states (0, 1, and 2) transitioning over discrete time steps, where transitions between states are governed by Poisson processes. The main challenge is to determine the probability density function (PDF) of the time since the last state was 1, given that the system can only be measured at discrete intervals. The conversation highlights the need for a clearer formulation of the problem, emphasizing the importance of estimating transition probabilities in a Hidden Markov model. Suggested methods for estimation include the Baum-Welch algorithm and the use of transition matrices for deriving properties like hitting times. The overall goal is to refine the understanding of the transition rates as experiments are repeated.
omg!
Messages
50
Reaction score
0
please excuse any inaccuracies in the following formulation of my problem. here it goes:

1. there are n discrete time steps
2. between every time step, transitions 0 -> 1, 1-> 0 can happen between states denoted by 0 and 1. the two possible transition directions are poisson processes, i.e. number of time steps between transitions of the same kind are exponentially distributed.
3. additionally, there is another unidirectional transition from 1 to 2.

alternatively, you can think of the sequence X_1, X_2, \ldots, X_n of random variables, with X_i\in\left{0,1,2}\right} with the conditions P[X_{i+1}=1|X_{i}=0]=\text{const.}, P[X_{i+1}=0|X_{i}=1]=const., P[X_{i+1}=2|X_{i}=1]=const., P[X_{i+1}=2|X_{i}=0]=0 and P[X_{i+1}=0,1|X_{i}=2]=0.

now take the continuum limit n\rightarrow\infty

the problem is: what is the PDF of time where the state was last 1, i.e. X(t)=1,X(s)\neq 1,\quad\forall s\geq t
 
Physics news on Phys.org
omg! said:
please excuse any inaccuracies in the following formulation of my problem. here it goes:

1. there are n discrete time steps
2. between every time step, transitions 0 -> 1, 1-> 0 can happen between states denoted by 0 and 1. the two possible transition directions are poisson processes, i.e. number of time steps between transitions of the same kind are exponentially distributed.
3. additionally, there is another unidirectional transition from 1 to 2.

...

now take the continuum limit n\rightarrow\infty

the problem is: what is the PDF of time where the state was last 1, i.e. X(t)=1,X(s)\neq 1,\quad\forall s\geq t

The terms poisson, exponential, continuum limit, pdf all seem to be misapplied here but if I understand you right it would be a stationary discrete-time 3-state Markov process with a single absorbing state and you need to work out distribution of the last hitting time of a particular state.

This example is simplified by observing that the last hitting time of 1 always occurs just before it reaches 2 (if the relevant transition probabilities are non-zero), so P[T<t] = P[X(t)=2] and the latter probability can be calculated by standard methods such as matrix exponentiation.
 
Last edited:
thank you for your answer.

but the state need not necessarily end up in 2, e.g. a sequence like this

...11000000

where the last state is the nth time step would give rise to a sequence where the last observed 1 state is at the n-6th time step.

my current approach is like this

P[X_k=1, X_l\neq 1, \forall l>k]=P[X_{k+1}=2]+P[X_{l}=0, \forall l>k]
but if i write down the PDF, it doesn't integrate to 1.
 
Last edited:
omg! said:
thank you for your answer.

but the state need not necessarily end up in 2, e.g. a sequence like this

...11000000

where the last state is the nth time step would give rise to a sequence where the last observed 1 state is at the n-6th time step.

Sorry I thought you only wanted the limit as n\rightarrow\infty, in which case X_n\rightarrow2 with probability 1.

my current approach is like this

P[X_k=1, X_l\neq 1, \forall l>k]=P[X_{k+1}=2]+P[X_{l}=0, \forall l>k]
but if i write down the PDF, it doesn't integrate to 1.

Almost right, it should say P[X_k=1, X_l\neq 1, \forall l>k]=P[X_k=1, X_{k+1}=2]+P[X_k=1, X_{l}=0, \forall l>k]. The rhs can be expressed in terms of P[X_k=1] and the transition probabilities (p_{10},p_{00}, p_{12}).
 
ok, let me describe the problem I'm really facing. the version above is badly formulated and too confusing, please forget it.

the thing is, i measure the state of a system as a function of time. The state can be 0,1 or 2. the measuring apparatus samples the state in a finite time period [t_{0},t_{1}] at discrete, evenly spaced points in time and it can only distinguish whether the system is in state 1 or NOT in state 1. after repeating the experiment many times, what i can calculate is q_k:=P[X_k=1, X_l\neq 1, \forall l>k,l\leq n], or, if i ignore the discrete sampling, q(t):=P[X(t)=1, X(s)\neq 1, \forall s>t,s\leq t_1]. But what I'm really interested in is p_k:=P[X_k=1, X_{k+1}=2], or p(t):=P[X(t)=1, X(s)=2, \forall s\geq t]. I also know what kind of distribution p(t) should have (exponential) but the distribution parameter is unknown (p(t) is the PDF of the random variable T=time t until the state of the system becomes 2, right?) . Also, i can assume r_k:=P[X_k=1, X_l=0, \forall l>k]=P[X_k=1, X_{k+1}=0]\cdot P[X_l,X_{l+1},\ldots,X_m=0, m\geq n] (right?), where the distribution type of the second factor is known.
I hope this statement of the problem is clearer. if not, i apologize for wasting your time.
 
omg! said:
ok, let me describe the problem I'm really facing. the version above is badly formulated and too confusing, please forget it.

the thing is, i measure the state of a system as a function of time. The state can be 0,1 or 2. the measuring apparatus samples the state in a finite time period [t_{0},t_{1}] at discrete, evenly spaced points in time and it can only distinguish whether the system is in state 1 or NOT in state 1. after repeating the experiment many times, what i can calculate is q_k:=P[X_k=1, X_l\neq 1, \forall l>k,l\leq n], or, if i ignore the discrete sampling, q(t):=P[X(t)=1, X(s)\neq 1, \forall s>t,s\leq t_1]. But what I'm really interested in is p_k:=P[X_k=1, X_{k+1}=2], or p(t):=P[X(t)=1, X(s)=2, \forall s\geq t]. I also know what kind of distribution p(t) should have (exponential) but the distribution parameter is unknown (p(t) is the PDF of the random variable T=time t until the state of the system becomes 2, right?) . Also, i can assume r_k:=P[X_k=1, X_l=0, \forall l>k]=P[X_k=1, X_{k+1}=0]\cdot P[X_l,X_{l+1},\ldots,X_m=0, m\geq n] (right?), where the distribution type of the second factor is known.

I hope this statement of the problem is clearer. if not, i apologize for wasting your time.

Ah that makes it a lot clearer now, so the problem is to estimate a transition probability (or transition rate) in a Hidden Markov model. There are a few methods that could be useful like Baum-Welch for example, but I don't know enough about the area to say which is most appropriate for your version of the problem. Someone in your local statistics department should be able to point you in the right direction. In particular it should be possible to get a rough estimate of p_k after only the first experiment and then refine that estimate as you do more experiments.

Also it's worth writing the model in vector form using transition matrices, which will give neat ways to derive other properties such as hitting times etc.

HTH
 
Back
Top