Markov chains and steady state vectors

1. Jun 1, 2005

Physics is Phun

In my data management class we are studying matricies, we have broken the whole unit up into sections and then in small groups we have to teach the rest of the class a section or the matrix unit. Well, our section is markov chains. I understand it myself but I don't know if I'll be able to explain/teach it to others very well.
Could someone go over the key points and try and put it in simple terms, so I can best explain it to the rest of my class?

Thanks

2. Jun 1, 2005

robphy

Last edited by a moderator: Apr 21, 2017
3. Jun 1, 2005

neurocomp2003

www.mathworld.com

markov chains are Pn = P0^N where P0 is a probability matrix right? and N is the number of generations?

4. Jun 2, 2005

TenaliRaman

Markov chains are a sequence of random variables X_1,....,X_n, where probability that a system is in state x_n at time t_n is exclusively dependent on the probability that the system is in state x_(n-1) at time t_(n-1).

In a very informal way we can say, that this is a record of n occurrences and each occurrence is such that its completely dependent only on the occurrence that occurred just before it and no other previous occurrence. (If you a sytem control person, you can think it of as first order linear/nonlinear system)

A more convenient way to think of this is to draw a state space diagram with every link from one state to other state denoting the probability of transition.

These probabilities of transition can be recorded in the form of a matrix (say P). If Q_0 denotes the initial probability (the probability with which a system is in some initial state), then the further developments of the system can be predicted as,
Q_1 = Q_0*P
Q_2 = Q_1*P
and so on.....

Note that this is just a mathematical model and there are many mathematical works made on this model. What this means is, if you ever find a real world problem which follows this model, then you can directly apply all the results that you see for markov model to your real world problem.

Interestingly, this simple model is highly useful in many places. Many of the real world engineering problems are first order linear/non-linear type and if there is a probabilistic control associated with it, then it becomes the perfect candidate for a markov model.

-- AI

I Proper notation in chain rule Jun 18, 2016
Continuous-Time Markov Chain populations Jan 12, 2015
Math Modeling with markov chains Oct 22, 2012
Markov chain Feb 18, 2012
What's a Markov Chain? Mar 31, 2007