Understanding Markov Chains: Deriving and Solving Probabilities

AI Thread Summary
Markov chains can be classified based on their properties, such as irreducibility and aperiodicity, which affect their transition probabilities and stationary distributions. To solve these chains numerically, methods like the Power method are commonly used to find the stationary distribution, especially for large matrices. Understanding the relationship between transition probabilities and stationary probabilities is crucial for accurate computations. Resources such as academic papers or textbooks on probability theory can provide further insights into these concepts. Exploring these materials will enhance comprehension of Markov chains and their applications.
giglamesh
Messages
14
Reaction score
0
Hello all
I have a question about Markov chain I've obtained in an application.
There is no need to mention the application or the details of markov chain because my question is simply:

The transition probabilities are derived with equations that depend on the stationary probability, I know it's something complicated ...

The question is:
1. do you know what is the class of these markov chains?
2. how to solve it numerically, does it depend on Power method?

If you have any paper or book it will be great
Thanks
 
Mathematics news on Phys.org
https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf

 

Thread 'Trigonometry problem of interest'

Seemingly by some mathematical coincidence, a hexagon of sides 2,2,7,7, 11, and 11 can be inscribed in a circle of radius 7. The other day I saw a math problem on line, which they said came from a Polish Olympiad, where you compute the length x of the 3rd side which is the same as the radius, so that the sides of length 2,x, and 11 are inscribed on the arc of a semi-circle. The law of cosines applied twice gives the answer for x of exactly 7, but the arithmetic is so complex that the...
View full post »
Thread 'Imaginary Pythagoras'

Thread 'Imaginary Pythagoras'

I posted this in the Lame Math thread, but it's got me thinking. Is there any validity to this? Or is it really just a mathematical trick? Naively, I see that i2 + plus 12 does equal zero2. But does this have a meaning? I know one can treat the imaginary number line as just another axis like the reals, but does that mean this does represent a triangle in the complex plane with a hypotenuse of length zero? Ibix offered a rendering of the diagram using what I assume is matrix* notation...
View full post »

Thread 'Fermat's Last Theorem'

Fermat's Last Theorem has long been one of the most famous mathematical problems, and is now one of the most famous theorems. It simply states that the equation $$ a^n+b^n=c^n $$ has no solutions with positive integers if ##n>2.## It was named after Pierre de Fermat (1607-1665). The problem itself stems from the book Arithmetica by Diophantus of Alexandria. It gained popularity because Fermat noted in his copy "Cubum autem in duos cubos, aut quadratoquadratum in duos quadratoquadratos, et...
View full post »

Similar threads

MHB Designing a Markov Model for Coke and Pepsi Purchases
Replies
1
Views
1K
The infinite limits of the probability transition matrix for Markov chain
Replies
24
Views
4K
I Bell vs Kolmogorov: Unravelling Probability Theory Limits
2
Replies
93
Views
7K
A Martingale, calculation of probability - Markov chain needed?
Replies
3
Views
2K
Math Modeling with markov chains
Replies
2
Views
2K
Communicating Classes in Markov Chains
Replies
2
Views
1K
Comp Sci Markov model to find the probability of a Pepsi drinker buying a Coke?
Replies
4
Views
2K
Prob/Stats Finding a Probability Book for Physics Students for a Complex Course
Replies
4
Views
2K
What is the proportion of time the runner runs barefooted using a Markov Chain?
Replies
9
Views
2K
I A seemingly simple problem about probability
Replies
13
Views
3K

Hot Threads

Recent Insights

Back
Top