Markov Chain of Stochastic Processes

Click For Summary
SUMMARY

The discussion centers on constructing a model using a Markov chain that incorporates different stochastic processes for each state. The term for this concept is a "time-heterogeneous" or "non-stationary" Markov chain, where the probability of transitioning from one state to another is dependent on the time index, t. A relevant resource is the paper by HS Chang titled "Multitime Scale Markov Decision Processes," published in 2003, which explores similar concepts.

PREREQUISITES
  • Understanding of Markov chains and their properties
  • Familiarity with stochastic processes
  • Knowledge of time-heterogeneous models
  • Basic grasp of Markov Decision Processes (MDPs)
NEXT STEPS
  • Research "time-heterogeneous Markov chains" for advanced modeling techniques
  • Study the paper "Multitime Scale Markov Decision Processes" by HS Chang
  • Explore applications of non-stationary Markov chains in real-world scenarios
  • Learn about Markov Decision Processes and their relation to stochastic modeling
USEFUL FOR

Researchers, data scientists, and statisticians interested in advanced modeling techniques involving Markov chains and stochastic processes.

stargazer_iq
Messages
6
Reaction score
0
I would like to construct a model using a markov chain that has different stochastic processes for each state in the chain. Is there a term for such a thing, or anything similar to it?
Thanks
 
Physics news on Phys.org
i found something similar to what I'm looking for in this paper:
HS Chang, "Multitime Scale Markov Decision Processes", 2003
 

Similar threads

Undergrad Question about ARMA process: Changing a Stochastic Process into a Transfer Function
  • · Replies 2 ·
Replies
2
Views
3K
Undergrad Probability of a Stochastic Markov process
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Looking for text on stochastic processes
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad How many tosses to flip a coin HHTH?
  • · Replies 41 ·
2
Replies
41
Views
9K
High School Can I replace ##X_n = i## with ##A## to type less? Rules of math.
  • · Replies 5 ·
Replies
5
Views
2K
MHB Optimizing Markov Chain Production System Throughput w/ Exponential Rate of 50
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad A seemingly simple problem about probability
  • · Replies 13 ·
Replies
13
Views
3K
Undergrad Is throwing dice a stochastic or a deterministic process?
  • · Replies 6 ·
Replies
6
Views
3K
MHB "Two step" Markov chain is also a Markov chain.
  • · Replies 4 ·
Replies
4
Views
2K
High School Markov model on a sequence of numbers
  • · Replies 4 ·
Replies
4
Views
2K