Markov Chain - Find the Stationary Distribution

[jgbradley1]

Ok, I had a homework problem that I cannot for the life of me, figure out. I've tried to google for different sources that would show me how to find the stationary distribution of a markov chain, but I can't seem to find one that makes sense to me.

The transition matrix of a markov chain is given by

R1 [0.25 0.5 0]
R2 [0.5 0 0.25]
R3 [0.25 0.5 0.75]


and the instructions say to find the stationary distribution if it exist. If someone could walk through the process of how to find a stationary distribution of this, I would appreciate it. I tried to follow the directions from a .pdf file I found online that explained the whole process and I got

Let T = the above transition matrix

Tv = v

*after some calculations
v = [7/30 1/3 13/30]

Now, when I plugged this answer back into the equation Tv = v, it didn't work out, so I'm not to sure what I did wrong. I'm not looking for the answer explicitly...I would like a layman's explanation on how to find the stationary distribution and maybe an example if that's not asking too much?

 

[Mark44]

You made an error in your calculation of v. The exact values are [2/13, 3/13, 8/13], assuming these are probabilities that have to sum to 1. Your first two coordinates are close, but the third is off by quite a bit.

Any vector that is a multiple of [2, 3, 8] will work, but if I recall, the input vectors should be scaled so that their coordinates add to 1, which is the reason for the 13 in the denominators. Or if the input vector needs to be a unit vector, then each fraction should have a denominator of sqrt(77).

As for the layman's explanation, in a Markov process, you start with some initial vector v₀, and multiply it by the matrix to get an output vector v₁. At the next step, you use the old output vector as the new input vector, and multiply it again, to get a new output vector. You keep repeating the process over and over again. If the process is stable, eventually you will get to a vector that is unchanged by multiplication by your matrix.

At this point Av = v. I.e., for that input vector v, the output is exactly the same. Another way to say this is that (A - I)v = 0. Here I is the 3 x 3 identity matrix.

What I did was solve the matrix equation above for v. One solution I got was [2, 3, 8]. All solutions are multiples of this vector.

If you multiply your matrix times that vector, you'll get exactly the same vector out, so that is a stationary distribution (or rather, the 1/13 multiple of it is, if the coordinates need to add to 1).

 

[statdad]

From the setup of your matrix (coumns sum to one) you need to multiply as

$$v A$$

so see $$vA = v$$