Long-run proportion in state ′A′

[iikii]

The question asks:

A physical device can be in three states: A,B,C. The device operates as follows (all time units are in hours):

The device spends an exponentially distributed amount of time in stateAA (with mean of 12minutes) and then with probability 0.6 goes to state B, and with prob.0.4goes to state C. When in state B, the device moves to state C after an Exp(3) amount of time. When in state C, the device goes to state A at rate 1/hour, and to state B at rate 2/hour. Let Xt represent the device state at time tt, and suppose X0=′A0′. What is the long-run proportion of time the device spends in state ′A′.

So I am thinking about maybe I should calculate the probability the device is in state A after 30 minutes and the probability the device is in state A after 30 minutes given that it was in state B after 5 minutes and in state C after 10 minutes. But what not sure how to do. Any help is appreciated!

 

[Ray Vickson]

The question asks:

A physical device can be in three states: A,B,C. The device operates as follows (all time units are in hours):

The device spends an exponentially distributed amount of time in stateAA (with mean of 12minutes) and then with probability 0.6 goes to state B, and with prob.0.4goes to state C. When in state B, the device moves to state C after an Exp(3) amount of time. When in state C, the device goes to state A at rate 1/hour, and to state B at rate 2/hour. Let Xt represent the device state at time tt, and suppose X0=′A0′. What is the long-run proportion of time the device spends in state ′A′.

So I am thinking about maybe I should calculate the probability the device is in state A after 30 minutes and the probability the device is in state A after 30 minutes given that it was in state B after 5 minutes and in state C after 10 minutes. But what not sure how to do. Any help is appreciated!

Does Exp(3) mean an exponential random variable with a rate of 3 (per hour), or does it mean an exponential with a mean of 3 (hours)?

After obtaining the transition-rate matrix, this problem is a simple example of using the standard equations to find the long-run limiting state probabilities. Finding state probabilities for finite times (such as t = 30 min = 0.5 hr) would require finding transient behavior, and finding that would require solving the 3x3 system of coupled linear differential equations. It is much, much easier to determine the limiting values. This material is covered in just about every textbook on the subject.

 

[iikii]

Does Exp(3) mean an exponential random variable with a rate of 3 (per hour), or does it mean an exponential with a mean of 3 (hours)?

After obtaining the transition-rate matrix, this problem is a simple example of using the standard equations to find the long-run limiting state probabilities. Finding state probabilities for finite times (such as t = 30 min = 0.5 hr) would require finding transient behavior, and finding that would require solving the 3x3 system of coupled linear differential equations. It is much, much easier to determine the limiting values. This material is covered in just about every textbook on the subject.

Thanks! So I got the probability going from A to B P(A,B)=0.6, P(A,C)=0.4, P(B,C)=1, but what are the probabilities of P(C,B) and P(C,A)?

 

[Ray Vickson]

Thanks! So I got the probability going from A to B P(A,B)=0.6, P(A,C)=0.4, P(B,C)=1, but what are the probabilities of P(C,B) and P(C,A)?

So, what is your answer to my question about the meaning of Exp(3)?

You need to construct a continuous-time Markov chain transition rate matrix.

 

[iikii]

So, what is your answer to my question about the meaning of Exp(3)?

You need to construct a continuous-time Markov chain transition rate matrix.

I think it is the rate parameter.