Dynamic transition probability matrix in markov chains

Click For Summary
SUMMARY

The discussion centers on the concept of dynamic transition probability matrices in Markov chains, specifically focusing on non-homogeneous Markov chains where transition probabilities vary over time. Participants express interest in understanding how factors such as population dynamics can influence these probabilities. The need for systematic studies and references on this topic is highlighted, indicating a gap in available literature for those exploring time-dependent Markov processes.

PREREQUISITES
  • Understanding of Markov chains and their fundamental principles.
  • Familiarity with probability theory and stochastic processes.
  • Knowledge of non-homogeneous Markov chains and their applications.
  • Basic statistical analysis skills to interpret dynamic models.
NEXT STEPS
  • Research "non-homogeneous Markov chains" for foundational knowledge.
  • Explore case studies on population migration and its impact on transition probabilities.
  • Investigate academic papers on time-dependent Markov processes.
  • Learn about tools for modeling dynamic systems, such as MATLAB or R.
USEFUL FOR

Researchers, data scientists, and statisticians interested in advanced Markov chain applications, particularly in fields like population studies and dynamic modeling.

pyrole
Messages
3
Reaction score
0
I am intrigued by the idea of changing/dynamic transition probability matrix instead of the assumption of constant transition probability between states.

For eg. Probability of population migrating between 2 states might not remain constant and can be a function of its population or some other factors. Is there a systematic study of this particular category of dynamic or changing transition probabilities in markov processes.
 
Physics news on Phys.org
You're interested in a topic called "non-homogeneous Markov chains" or "time non-homogeneous Markov chains".

I don't know a good reference that introduces this topic and I don't claim to be able to introduce it myself!
 

Similar threads

Graduate Question about a property of a matrix of transition probabilities
  • · Replies 6 ·
Replies
6
Views
2K
The infinite limits of the probability transition matrix for Markov chain
  • · Replies 24 ·
Replies
24
Views
4K
MHB Regarding mixing time of a Markov chain
  • · Replies 2 ·
Replies
2
Views
2K
MHB Solving Transition Matrix: Octopus Training
  • · Replies 20 ·
Replies
20
Views
5K
Changing the minimum value of a probability matrix
  • · Replies 4 ·
Replies
4
Views
994
Graduate Martingale, calculation of probability - Markov chain needed?
  • · Replies 3 ·
Replies
3
Views
2K
Transition matrix of a paint ball game
  • · Replies 6 ·
Replies
6
Views
2K
MHB Markov Chain - Is state 2 periodic?
  • · Replies 4 ·
Replies
4
Views
2K
Graduate Communicating Classes in Markov Chains
  • · Replies 2 ·
Replies
2
Views
1K
MHB How Can Uniform Random Variables Simulate a Markov Chain?
  • · Replies 3 ·
Replies
3
Views
2K