Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Finding long term Markov behaviors

  1. Aug 7, 2011 #1
    1. The problem statement, all variables and given/known data

    Find the long-term behavior of the Markov system described by the following matrix with initial conditions below: (i.e. find T^infinity)

    Find x_infinity

    2. Relevant equations

    x_0 = [.3 .3 .4]^T

    T =

    [.5 .4 .1]
    [.4 .3 .3]
    [.1 .3 .6]

    3. The attempt at a solution

    So in my attempt to find x_infinity I know that x_n = ( T^n)(x_0)

    This means:

    [.5^n .4^n .1^n] * [.3 .3 .4]^T
    [.4^n .3^n .3^n]
    [.1^n .3^n .6^n]

    Which I solved to be
    [ .3*.5^n .3*.4^n .4*.1^n]
    [ .3*.4^n .3*.3^n .4*.3^n]
    [ .3*.1^n .3*.3^n .4*.6^n]

    But as n approaches infinity all the terms get smaller and all go to zero. Does this seem correct so far?

    That would mean:

    x_infinity = ( 0 0 0 )^T ??? I tried this on my webassign, but it is not correct. Any tips or help in pointing out my mistakes would be great, thanks!
    Last edited: Aug 7, 2011
  2. jcsd
  3. Aug 7, 2011 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Your expression for T^n is wrong. T^n = nth power of T in the sense of matrix multiplication. For example, T^2 = T*T =[[.42,.35,.23],[.35,.35,.31],[.23,.31,.46]], etc. You need to perform vector-matrix multiplications, or else solve the "equilibrium equations" to find the long-run limiting probabilities.

  4. Aug 7, 2011 #3
    Oh I see how I was doing the expression of T^n was wrong. What do you mean by vector-matrix multiplcation or "equilibrium equations"? Do you mean keep finding T^1,T^2,T^3, and see if it approaches a limit?
  5. Aug 7, 2011 #4

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    The state-probability row vector after n steps is x_n = x_0 * T^n, which can be written as x_{n-1}*T (a vector times a matrix). If the x_n approach a limiting distribution y, we must have y = y*T, and sum y(j) = 1. This is a simple system of linear equations, called the equilibrium or balance equations. I cannot believe that this material is not all in your textbook, but if not, Google "Markov chain" to see various relevant articles.

Share this great discussion with others via Reddit, Google+, Twitter, or Facebook