Finding long term Markov behaviors

[Lolsauce]

Homework Statement

Find the long-term behavior of the Markov system described by the following matrix with initial conditions below: (i.e. find T^infinity)

Find x_infinity

Homework Equations

x_0 = [.3 .3 .4]^TT =

[.5 .4 .1]
[.4 .3 .3]
[.1 .3 .6]

The Attempt at a Solution

So in my attempt to find x_infinity I know that x_n = ( T^n)(x_0)

This means:

[.5^n .4^n .1^n] * [.3 .3 .4]^T
[.4^n .3^n .3^n]
[.1^n .3^n .6^n]

Which I solved to be
[ .3*.5^n .3*.4^n .4*.1^n]
[ .3*.4^n .3*.3^n .4*.3^n]
[ .3*.1^n .3*.3^n .4*.6^n]

But as n approaches infinity all the terms get smaller and all go to zero. Does this seem correct so far?

That would mean:

x_infinity = ( 0 0 0 )^T ? I tried this on my webassign, but it is not correct. Any tips or help in pointing out my mistakes would be great, thanks!

 

[Ray Vickson]

Your expression for T^n is wrong. T^n = nth power of T in the sense of matrix multiplication. For example, T^2 = T*T =[[.42,.35,.23],[.35,.35,.31],[.23,.31,.46]], etc. You need to perform vector-matrix multiplications, or else solve the "equilibrium equations" to find the long-run limiting probabilities.

RGV

 

[Lolsauce]

Your expression for T^n is wrong. T^n = nth power of T in the sense of matrix multiplication. For example, T^2 = T*T =[[.42,.35,.23],[.35,.35,.31],[.23,.31,.46]], etc. You need to perform vector-matrix multiplications, or else solve the "equilibrium equations" to find the long-run limiting probabilities.

RGV

Oh I see how I was doing the expression of T^n was wrong. What do you mean by vector-matrix multiplcation or "equilibrium equations"? Do you mean keep finding T^1,T^2,T^3, and see if it approaches a limit?

 

[Ray Vickson]

The state-probability row vector after n steps is x_n = x_0 * T^n, which can be written as x_{n-1}*T (a vector times a matrix). If the x_n approach a limiting distribution y, we must have y = y*T, and sum y(j) = 1. This is a simple system of linear equations, called the equilibrium or balance equations. I cannot believe that this material is not all in your textbook, but if not, Google "Markov chain" to see various relevant articles.

RGV