Understanding Markov Chains: Deriving and Solving Probabilities

  • Context: Graduate 
  • Thread starter Thread starter giglamesh
  • Start date Start date
  • Tags Tags
    deriving Probabilities
Click For Summary
SUMMARY

This discussion centers on understanding Markov chains, specifically focusing on deriving transition probabilities that rely on stationary probabilities. The user seeks clarification on the classification of these Markov chains and inquires about numerical solutions, particularly the applicability of the Power method. A reference to a relevant resource, a probability textbook chapter, is also provided for further reading.

PREREQUISITES
  • Understanding of Markov chains and their properties
  • Familiarity with stationary probability concepts
  • Knowledge of numerical methods for solving equations
  • Experience with the Power method for eigenvalue problems
NEXT STEPS
  • Research the classification of Markov chains, focusing on discrete-time and continuous-time types
  • Study numerical methods for solving Markov chains, emphasizing the Power method
  • Explore the implications of stationary probabilities in Markov processes
  • Read the referenced probability textbook chapter for in-depth understanding
USEFUL FOR

Mathematicians, data scientists, and anyone involved in probabilistic modeling or numerical analysis of Markov chains will benefit from this discussion.

giglamesh
Messages
14
Reaction score
0
Hello all
I have a question about Markov chain I've obtained in an application.
There is no need to mention the application or the details of markov chain because my question is simply:

The transition probabilities are derived with equations that depend on the stationary probability, I know it's something complicated ...

The question is:
1. do you know what is the class of these markov chains?
2. how to solve it numerically, does it depend on Power method?

If you have any paper or book it will be great
Thanks
 
Physics news on Phys.org
https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf

 

Similar threads

Graduate Martingale, calculation of probability - Markov chain needed?
  • · Replies 3 ·
Replies
3
Views
2K
Graduate Communicating Classes in Markov Chains
  • · Replies 2 ·
Replies
2
Views
1K
Graduate Deriving Probability Amplitude from Markov Density Function
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad A seemingly simple problem about probability
  • · Replies 13 ·
Replies
13
Views
3K
The infinite limits of the probability transition matrix for Markov chain
  • · Replies 24 ·
Replies
24
Views
4K
Undergrad Probability of a Stochastic Markov process
  • · Replies 1 ·
Replies
1
Views
2K
Graduate Markov chain, sum of N dice rolls
  • · Replies 4 ·
Replies
4
Views
8K
Graduate Is this a correct observation about Markov-chains and stoc.processes?
  • · Replies 14 ·
Replies
14
Views
2K
Graduate Area distribution of inscribed triangles in a circumference
  • · Replies 12 ·
Replies
12
Views
3K
Undergrad Bell vs Kolmogorov: Unravelling Probability Theory Limits
  • · Replies 93 ·
4
Replies
93
Views
8K