Understanding Markov Chains: Deriving and Solving Probabilities

In summary, the conversation is about someone asking for information on Markov chains and their transition probabilities. They mention that the equations used to derive the probabilities are complicated, and they have two questions: 1. What is the class of these Markov chains? 2. How can they be solved numerically, and does it involve the Power method? They also ask for any recommended resources on the topic.
  • #1
giglamesh
14
0
Hello all
I have a question about Markov chain I've obtained in an application.
There is no need to mention the application or the details of markov chain because my question is simply:

The transition probabilities are derived with equations that depend on the stationary probability, I know it's something complicated ...

The question is:
1. do you know what is the class of these markov chains?
2. how to solve it numerically, does it depend on Power method?

If you have any paper or book it will be great
Thanks
 
Mathematics news on Phys.org
  • #2
https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf

 

1. What is a Markov chain?

A Markov chain is a mathematical model used to describe the probabilistic behavior of a system over time. It consists of a set of states and transition probabilities between these states.

2. How are Markov chains used in scientific research?

Markov chains are used in a variety of scientific fields, including biology, economics, and computer science. They can be used to model and predict the behavior of complex systems, such as the spread of diseases, financial markets, and machine learning algorithms.

3. What is the process for deriving probabilities in a Markov chain?

The process for deriving probabilities in a Markov chain involves defining the states and transition probabilities, setting up a transition matrix, and using matrix multiplication to calculate the probabilities of transitioning between states over multiple time steps.

4. How can Markov chains be solved?

Markov chains can be solved using a variety of methods, including the stationary distribution method, the power method, and the Monte Carlo simulation method. Each method has its own advantages and may be more suitable for certain types of problems.

5. What are some real-world applications of Markov chains?

Markov chains have numerous real-world applications, such as predicting stock prices, modeling gene sequences, analyzing customer behavior, and optimizing search engine algorithms. They are also commonly used in natural language processing and speech recognition systems.

Similar threads

MHB Designing a Markov Model for Coke and Pepsi Purchases
    • General Math
Replies
1
Views
794
The infinite limits of the probability transition matrix for Markov chain
    • Precalculus Mathematics Homework Help
Replies
24
Views
2K
I Bell vs Kolmogorov: Unravelling Probability Theory Limits
    • Quantum Physics
3
Replies
93
Views
5K
A Martingale, calculation of probability - Markov chain needed?
    • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K
Comp Sci Markov model to find the probability of a Pepsi drinker buying a Coke?
    • Engineering and Comp Sci Homework Help
Replies
4
Views
2K
Math Modeling with markov chains
    • General Math
Replies
2
Views
2K
Communicating Classes in Markov Chains
    • Set Theory, Logic, Probability, Statistics
Replies
2
Views
964
Prob/Stats Finding a Probability Book for Physics Students for a Complex Course
    • Science and Math Textbooks
Replies
4
Views
1K
I A seemingly simple problem about probability
    • Set Theory, Logic, Probability, Statistics
Replies
13
Views
1K
General first-order markov chain of 2 states
    • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K

Hot Threads

Recent Insights

Back
Top