Understanding Markov Chains: Deriving and Solving Probabilities

Click For Summary
Markov chains can be classified based on their properties, such as irreducibility and aperiodicity, which affect their transition probabilities and stationary distributions. To solve these chains numerically, methods like the Power method are commonly used to find the stationary distribution, especially for large matrices. Understanding the relationship between transition probabilities and stationary probabilities is crucial for accurate computations. Resources such as academic papers or textbooks on probability theory can provide further insights into these concepts. Exploring these materials will enhance comprehension of Markov chains and their applications.
giglamesh
Messages
14
Reaction score
0
Hello all
I have a question about Markov chain I've obtained in an application.
There is no need to mention the application or the details of markov chain because my question is simply:

The transition probabilities are derived with equations that depend on the stationary probability, I know it's something complicated ...

The question is:
1. do you know what is the class of these markov chains?
2. how to solve it numerically, does it depend on Power method?

If you have any paper or book it will be great
Thanks
 
Mathematics news on Phys.org
https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf

 

Thread 'Area of Overlapping Squares'

Here is a little puzzle from the book 100 Geometric Games by Pierre Berloquin. The side of a small square is one meter long and the side of a larger square one and a half meters long. One vertex of the large square is at the center of the small square. The side of the large square cuts two sides of the small square into one- third parts and two-thirds parts. What is the area where the squares overlap?
View full post »

Similar threads

MHB Designing a Markov Model for Coke and Pepsi Purchases
  • · Replies 1 ·
Replies
1
Views
1K
The infinite limits of the probability transition matrix for Markov chain
  • · Replies 24 ·
Replies
24
Views
4K
Undergrad Bell vs Kolmogorov: Unravelling Probability Theory Limits
  • · Replies 93 ·
4
Replies
93
Views
7K
Graduate Martingale, calculation of probability - Markov chain needed?
  • · Replies 3 ·
Replies
3
Views
2K
Math Modeling with markov chains
  • · Replies 2 ·
Replies
2
Views
2K
Communicating Classes in Markov Chains
  • · Replies 2 ·
Replies
2
Views
1K
Comp Sci Markov model to find the probability of a Pepsi drinker buying a Coke?
  • · Replies 4 ·
Replies
4
Views
2K
Prob/Stats Finding a Probability Book for Physics Students for a Complex Course
  • · Replies 4 ·
Replies
4
Views
2K
What is the proportion of time the runner runs barefooted using a Markov Chain?
  • · Replies 9 ·
Replies
9
Views
2K
Undergrad A seemingly simple problem about probability
  • · Replies 13 ·
Replies
13
Views
3K