Need assistance figuring out a model PDF

[omg!]

please excuse any inaccuracies in the following formulation of my problem. here it goes:

1. there are $$n$$ discrete time steps
2. between every time step, transitions 0 -> 1, 1-> 0 can happen between states denoted by 0 and 1. the two possible transition directions are poisson processes, i.e. number of time steps between transitions of the same kind are exponentially distributed.
3. additionally, there is another unidirectional transition from 1 to 2.

alternatively, you can think of the sequence $$X_1, X_2, \ldots, X_n$$ of random variables, with $$X_i\in\left{0,1,2}\right}$$ with the conditions $$P[X_{i+1}=1|X_{i}=0]=\text{const.}$$, $$P[X_{i+1}=0|X_{i}=1]=const.$$, $$P[X_{i+1}=2|X_{i}=1]=const.$$, $$P[X_{i+1}=2|X_{i}=0]=0$$ and $$P[X_{i+1}=0,1|X_{i}=2]=0$$.

now take the continuum limit $$n\rightarrow\infty$$

the problem is: what is the PDF of time where the state was last 1, i.e. $$X(t)=1,X(s)\neq 1,\quad\forall s\geq t$$

 

[bpet]

please excuse any inaccuracies in the following formulation of my problem. here it goes:

1. there are $$n$$ discrete time steps
2. between every time step, transitions 0 -> 1, 1-> 0 can happen between states denoted by 0 and 1. the two possible transition directions are poisson processes, i.e. number of time steps between transitions of the same kind are exponentially distributed.
3. additionally, there is another unidirectional transition from 1 to 2.

...

now take the continuum limit $$n\rightarrow\infty$$

the problem is: what is the PDF of time where the state was last 1, i.e. $$X(t)=1,X(s)\neq 1,\quad\forall s\geq t$$

The terms poisson, exponential, continuum limit, pdf all seem to be misapplied here but if I understand you right it would be a stationary discrete-time 3-state Markov process with a single absorbing state and you need to work out distribution of the last hitting time of a particular state.

This example is simplified by observing that the last hitting time of 1 always occurs just before it reaches 2 (if the relevant transition probabilities are non-zero), so $$P[T<t] = P[X(t)=2]$$ and the latter probability can be calculated by standard methods such as matrix exponentiation.

 

[omg!]

thank you for your answer.

but the state need not necessarily end up in 2, e.g. a sequence like this

...11000000

where the last state is the nth time step would give rise to a sequence where the last observed 1 state is at the n-6th time step.

my current approach is like this

$$P[X_k=1, X_l\neq 1, \forall l>k]=P[X_{k+1}=2]+P[X_{l}=0, \forall l>k]$$
but if i write down the PDF, it doesn't integrate to 1.

 

[bpet]

thank you for your answer.

but the state need not necessarily end up in 2, e.g. a sequence like this

...11000000

where the last state is the nth time step would give rise to a sequence where the last observed 1 state is at the n-6th time step.

Sorry I thought you only wanted the limit as $$n\rightarrow\infty$$, in which case $$X_n\rightarrow2$$ with probability 1.

my current approach is like this

$$P[X_k=1, X_l\neq 1, \forall l>k]=P[X_{k+1}=2]+P[X_{l}=0, \forall l>k]$$
but if i write down the PDF, it doesn't integrate to 1.

Almost right, it should say $$P[X_k=1, X_l\neq 1, \forall l>k]=P[X_k=1, X_{k+1}=2]+P[X_k=1, X_{l}=0, \forall l>k].$$ The rhs can be expressed in terms of $$P[X_k=1]$$ and the transition probabilities $$(p_{10},p_{00}, p_{12})$$.

 

[omg!]

ok, let me describe the problem I'm really facing. the version above is badly formulated and too confusing, please forget it.

the thing is, i measure the state of a system as a function of time. The state can be 0,1 or 2. the measuring apparatus samples the state in a finite time period $$[t_{0},t_{1}]$$ at discrete, evenly spaced points in time and it can only distinguish whether the system is in state 1 or NOT in state 1. after repeating the experiment many times, what i can calculate is $$q_k:=P[X_k=1, X_l\neq 1, \forall l>k,l\leq n]$$, or, if i ignore the discrete sampling, $$q(t):=P[X(t)=1, X(s)\neq 1, \forall s>t,s\leq t_1]$$. But what I'm really interested in is $$p_k:=P[X_k=1, X_{k+1}=2]$$, or $$p(t):=P[X(t)=1, X(s)=2, \forall s\geq t]$$. I also know what kind of distribution p(t) should have (exponential) but the distribution parameter is unknown (p(t) is the PDF of the random variable T=time t until the state of the system becomes 2, right?) . Also, i can assume $$r_k:=P[X_k=1, X_l=0, \forall l>k]=P[X_k=1, X_{k+1}=0]\cdot P[X_l,X_{l+1},\ldots,X_m=0, m\geq n]$$ (right?), where the distribution type of the second factor is known.
I hope this statement of the problem is clearer. if not, i apologize for wasting your time.

 

[bpet]

ok, let me describe the problem I'm really facing. the version above is badly formulated and too confusing, please forget it.

the thing is, i measure the state of a system as a function of time. The state can be 0,1 or 2. the measuring apparatus samples the state in a finite time period $$[t_{0},t_{1}]$$ at discrete, evenly spaced points in time and it can only distinguish whether the system is in state 1 or NOT in state 1. after repeating the experiment many times, what i can calculate is $$q_k:=P[X_k=1, X_l\neq 1, \forall l>k,l\leq n]$$, or, if i ignore the discrete sampling, $$q(t):=P[X(t)=1, X(s)\neq 1, \forall s>t,s\leq t_1]$$. But what I'm really interested in is $$p_k:=P[X_k=1, X_{k+1}=2]$$, or $$p(t):=P[X(t)=1, X(s)=2, \forall s\geq t]$$. I also know what kind of distribution p(t) should have (exponential) but the distribution parameter is unknown (p(t) is the PDF of the random variable T=time t until the state of the system becomes 2, right?) . Also, i can assume $$r_k:=P[X_k=1, X_l=0, \forall l>k]=P[X_k=1, X_{k+1}=0]\cdot P[X_l,X_{l+1},\ldots,X_m=0, m\geq n]$$ (right?), where the distribution type of the second factor is known.

I hope this statement of the problem is clearer. if not, i apologize for wasting your time.

Ah that makes it a lot clearer now, so the problem is to estimate a transition probability (or transition rate) in a Hidden Markov model. There are a few methods that could be useful like Baum-Welch for example, but I don't know enough about the area to say which is most appropriate for your version of the problem. Someone in your local statistics department should be able to point you in the right direction. In particular it should be possible to get a rough estimate of $$p_k$$ after only the first experiment and then refine that estimate as you do more experiments.

Also it's worth writing the model in vector form using transition matrices, which will give neat ways to derive other properties such as hitting times etc.

HTH