Rate from one state to another state in Markov

  • I
  • Thread starter mertcan
  • Start date
  • Tags
    Rate State
In summary, the conversation is discussing the use of units when calculating the rate of a process that occurs in steps. The question asks for the rate at which the production goes from up to down, and whether there should be a time unit included in the result. The solution only includes numbers without specifying a time unit. The speaker explains that the rate can be from one step to the next, and the units of the steps are not defined. The conversation then shifts to discussing the proportion of up time, and the use of the formula $$\frac {\pi_i*\mu_i} {\sum_{j=0}^n \pi_j*\mu_j}$$ to find it. The speaker believes that this formula makes sense as it includes the
  • #1
mertcan
340
6
upload_2016-11-27_13-23-43.png

Hi, initially I would like to express that if you look at the first question it asks the rate at which the production goes from up to down, and ıf there is a rate I think the result should include time unit, but when I look at the solution result only consists of numbers ( the multiplication of probabilities), there is no time unit. Could you explain why it is true? Although rate definition is number of times something happens, or the number of examples of something within a certain period, why don't we have any time period unit?
 
Physics news on Phys.org
  • #2
This is a process that is happening in steps. The rate would be from one step to the next. What units the steps are taken in is not defined. It could be from one hour to the next, from one second to the next, or it might not have anything to do with time. It could be from one position on a Z axis to the next or from one computer run to the next, etc. etc. It could even be going backward in time if you were trying to analyze how something happened and were analyzing it step-by-step backward in time. All the calculations can be done and make sense without specifying what the steps mean.
 
Last edited:
  • #3
FactChecker said:
This is a process that is happening in steps. The rate would be from one step to the next. What units the steps are taken in is not defined. It could be from one hour to the next, from one second to the next, or it might not have anything to do with time. It could be from one position on a Z axis to the next or from one computer run to the next, etc. etc. It could even be going backward in time if you were trying to analyze how something happened and were analyzing it step-by-step backward in time. All the calculations can be done and make sense without specifying what the steps mean.
upload_2016-11-27_15-11-36.png

Ok, thanks for return, but another question: also the question wants us to find proportion of up time (third part of question), but as far as I know in order to find proportion of time we use the following formula : $$\frac {\pi_i*\mu_i} {\sum_{j=0}^n \pi_j*\mu_j}$$, where $$\mu_i$$ is expected amount of time in state "i" before transition. But towards the solution expected amount of time is not used, why is not expected amount of time used?
 

Attachments

  • upload_2016-11-27_15-11-28.png
    upload_2016-11-27_15-11-28.png
    6.5 KB · Views: 370
  • #4
I don't know why you say that. πi is already the long-run proportion of time in state i. Why would you care where it transitioned to i from or how long it was in that prior state j?
 
  • #5
FactChecker said:
I don't know why you say that. πi is already the long-run proportion of time in state i. Why would you care where it transitioned to i from or how long it was in that prior state j?
Ok then, could you clarify for me what is the difference between $$\frac {\pi_i*\mu_i} {\sum_{j=0}^n \pi_j*\mu_j}$$ and $$\pi_i$$?I think I have some confusion here, because I know $$\frac {\pi_i*\mu_i} {\sum_{j=0}^n \pi_j*\mu_j}$$ as a proportion of time, and actually it makes sense because in this formula we have expected amount of time being in state i and total expected time, afterwards if we divide them we should obtain the proportion of time being in state i. Nevertheless, I know $$\pi_i$$ as Ni(m)/m where m is the number of transition and goes infinity, Ni(m) is the number of times being in state i in m transitions, I saw this definition in many books, and from here we can deduce that$$\pi_i$$ actually is not the proportion of time being in state i... What can you say about that ?
 
Last edited:
  • #6
I would interpret πi as already including any necessary calculations. It is already the long-run proportion of time in state i. What more do you want? It's not clear to me that your μi is any different from πi. In that case, it would be wrong to do your calculation using μi * πi because i does not always transition to i. Your equation might be saying something about the odds of staying in exactly the same state for one step (although I don't think that is right either because neither factor is a transition probability) Is that what you want?
 
  • #7
FactChecker said:
I would interpret πi as already including any necessary calculations. It is already the long-run proportion of time in state i. What more do you want? It's not clear to me that your μi is any different from πi. In that case, it would be wrong to do your calculation using μi * πi because i does not always transition to i. Your equation might be saying something about the odds of staying in exactly the same state for one step (although I don't think that is right either because neither factor is a transition probability) Is that what you want?
upload_2016-11-27_21-57-15.png

I cut this out of ross stochastic process book and I really wonder what you understand here ?, I understand Pi is the proportion of time in state i, and πi is different thing here...
 
  • #8
Hard to tell without all the definitions of the symbols, but the first line of the solution in post #1 says that πi are the "long-run proportions". So I would take that as a given. It is possible that the meanings of symbols are the same throughout, but I doubt it. In Theorem 4.8.3 it's not clear to me what the exact definitions of Pi, πi, and μi are.
CORRECTION: I meant to say "It is possible that the meanings of symbols are not the same throughout, but I doubt it."
 
Last edited:
  • #9
FactChecker said:
Hard to tell without all the definitions of the symbols, but the first line of the solution in post #1 says that πi are the "long-run proportions". So I would take that as a given. It is possible that the meanings of symbols are the same throughout, but I doubt it. In Theorem 4.8.3 it's not clear to me what the exact definitions of Pi, πi, and μi are.
upload_2016-11-28_12-34-48.png

By the way also I found the definition of πi @FactChecker in the same book I mentioned (ross stochastic process, I hope it can clarify something for you). And still I can not get the logic of why we write the proportion of up time in post#1 just in terms of πi whereas in post#7 we also use μi ??
 
  • #10
FactChecker said:
Hard to tell without all the definitions of the symbols, but the first line of the solution in post #1 says that πi are the "long-run proportions". So I would take that as a given. It is possible that the meanings of symbols are the same throughout, but I doubt it. In Theorem 4.8.3 it's not clear to me what the exact definitions of Pi, πi, and μi are.
CORRECTION: I meant to say "It is possible that the meanings of symbols are not the same throughout, but I doubt it."
 
  • #11
mertcan said:
View attachment 109570
By the way also I found the definition of πi @FactChecker in the same book I mentioned (ross stochastic process, I hope it can clarify something for you). And still I can not get the logic of why we write the proportion of up time in post#1 just in terms of πi whereas in post#7 we also use μi ??
I am still struggling with the definitions of the notation. I don't see any post that defines μi. I have several other questions about the parts you have posted and I don't think that I can help you. About the only thing that seems unambiguous to me is the statement in the first line of the solution in post #1 says that πi are the "long-run proportions". But that seems to be all that is needed for that problem.
 
Last edited:
  • #12
FactChecker said:
I am still struggling with the definitions of the notation. I don't see any post that defines μi. I have several other questions about the parts you have posted and I don't think that I can help you. About the only thing that seems unambiguous to me is the statement in the first line of the solution in post #1 says that πi are the "long-run proportions". But that seems to be all that is needed for that problem.
upload_2016-11-27_21-57-15-png.109532.png


upload_2016-11-28_20-13-3.png

@FactChecker I want to share one more attachment which may enable you to understand μi definition, also I would like to express that this attachment is the following part of post#7 (following part of proof). Besides in order to ease your comprehension I added both new attachment and the attachment in post#7 together. Therefore let me repeat my question why do we write the proportion of up time in post#1 just in terms of πi whereas in post#7 we also use μi in order to find proportion of time??
 
  • #13
I don't know. There are 2 statements of what the πi are:
1) In the Example 4.21 Solution they are described as the "long-run proportions in state i"
If you accept this meaning do you agree with the solution? I think it makes sense.

2) In Theorem 4.8.3, they are described as 1/E( number of transitions between visits to i )
Is this different from the "long-run proportions in state i"?
And then there is Pi, which again seems to me like it is the same as the "long-run proportions in state i", although there are a lot of definitions to wade through.
I will have to "tap out" and leave this to smarter people than me.
 
  • #14
mertcan said:
View attachment 109485
Hi, initially I would like to express that if you look at the first question it asks the rate at which the production goes from up to down, and ıf there is a rate I think the result should include time unit, but when I look at the solution result only consists of numbers ( the multiplication of probabilities), there is no time unit. Could you explain why it is true? Although rate definition is number of times something happens, or the number of examples of something within a certain period, why don't we have any time period unit?

If you have a discrete-time Markov chain, the times are t = 0,1,2,..., and ##P_{ij}= P\{X(t+1) = j | X(t) = i \}##. A transition takes place at every single unit of time, although it may be a transition from a state to itself (so not really a change of state at all). So, for example, if the time unit is 1 day, the rate of going (from the "up" states ##U## to the "down" states ##D##) is
$$R_{U \to D} = \sum_{j \in D} \sum_{i \in U} \pi_i P_{ij} $$
This will be a "dimensionless" fractional number < 1, so is a fraction of a day. For example, if ##R_{U \to D} = 1/20##, that means that, over the long run, there is about a 5% chance of a breakdown in an operating day.

If you have a continuous-time Markov chain, the times are real number t ≥ 0. Now the system remains in a state ##i## for an exponentially-distributed random amount of time ##T_i##, then jumps to another state ##j \neq i## with some probability ##q_{ij}##. If ##\mu_i = E(T_i)##, the transition rate from state ##i## to state ##j \neq i## is ##a_{ij} = q_{ij}/\mu_i## and the diagonal element of the transition matrix is the negative quantity ##a_{ii} = - 1/\mu_i##. In this case the long-run average rate of transition from up states ##U## to down states ##D## is
$$R_{U \to D} = \sum_{j \in D} \sum_{i \in U} \pi_i a_{ij} $$
This is not a dimensional number, now, but has dimensions of a ##\text{rate} = 1/\text{time}##.

As for proportions of time up or down, there is a difference between the discrete-time continuous-time cases. Look first at the discrete-time case. Over a large number N of discrete time periods 1,2,..., N, we might observe the system at each time and get a record of results something like
uuudduuuuuuddddduduudduuuuuddduuuuuuddddddduuuu,
where u = "up| and d = "down". If ##N_u## and ##N_d## are the number of u's and d's (with ##N_u + N_d = N##, then the observed fraction of time the system is up is ##f_u(N)= N_u/N##. Of course, ##N_u## and ##f_u(N)## are random for any fixed, finite ##N##, but in the limit ##N \to \infty## the fraction ##f_u(N)## converges to ##f_u##, a definite, non-random quantity. That is just the long-run proportion of time the system is in one of the states in ##U##, so is just ##f_u = \sum_{i \in U} \pi_i##.

In the continuous-time case the situation is different. Now, over a long time interval ##[0,T]## the fraction of time the system is up depends both on the probability it is up and the time spent in up states when it is up. That is why you would get ##f_u = \sum_{i \in U} \pi_i \mu_i## in the continuous-time case.
 

1. What is a Markov chain and how does it relate to rates between states?

A Markov chain is a mathematical model that describes the probabilities of moving from one state to another in a system. It is used to analyze processes that involve random transitions between different states, such as changes in weather patterns or stock market fluctuations. The rates between states in a Markov chain represent the likelihood of moving from one state to another within a certain time frame.

2. How is the rate between states calculated in a Markov chain?

The rate between states in a Markov chain is calculated by dividing the number of transitions from one state to another by the total number of transitions from that state. This is known as the transition rate or transition probability. The rate can also be expressed as a percentage or a decimal value.

3. What factors influence the rate between states in a Markov chain?

The rate between states in a Markov chain is influenced by the transition probabilities, which represent the likelihood of moving from one state to another. These probabilities can be affected by external factors such as time, population size, or environmental conditions. Additionally, the current state of the system can also impact the transition rates.

4. Can the rate between states in a Markov chain change over time?

Yes, the rate between states in a Markov chain can change over time. This is because the transition probabilities can vary depending on external factors or the current state of the system. In some cases, the transition rates may also be influenced by the previous state or states in the chain.

5. How is the rate between states useful in predicting future states in a Markov chain?

The rate between states is a key component in predicting future states in a Markov chain. By understanding the transition probabilities and the current state of the system, we can calculate the probability of moving from one state to another in the next time step. This information can be used to make more accurate predictions about the future states of the system.

Similar threads

  • Precalculus Mathematics Homework Help
Replies
4
Views
708
  • Calculus and Beyond Homework Help
Replies
17
Views
881
  • General Math
Replies
31
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • Special and General Relativity
Replies
4
Views
980
Replies
2
Views
600
  • Set Theory, Logic, Probability, Statistics
Replies
9
Views
4K
  • Quantum Physics
Replies
3
Views
703
Replies
1
Views
790
  • Advanced Physics Homework Help
Replies
6
Views
1K
Back
Top