1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Markov Chain Steady State (?were am i going wrong?)

  1. Apr 9, 2013 #1
    1. The problem statement, all variables and given/known data

    Subpart of the question requires me to find the steady state of the transition matrix:
    [itex]P=\begin{bmatrix}
    0.1 & 0.7 & 0.2 \\
    0.1 & 0.8 & 0.1\\
    0.3 & 0.1 & 0.6
    \end{bmatrix}[/itex]

    2. Relevant equations
    We thus need to find vector [itex]\boldsymbol{v}[/itex] in the equation [itex]P\boldsymbol{v}=\boldsymbol{v}[/itex] under the constraint [itex]sum(\boldsymbol{v})=1[/itex].

    3. The attempt at a solution
    Basically a system of linear equations with the added constraint:
    [itex]\begin{matrix}
    -0.9x_1 &+& 0.7x_2 &+& 0.2x_3 & = 0 \\
    0.1x_1 &+& -0.2x_2 &+& 0.1x_3 & = 0 \\
    0.3x_1 &+& 0.1x_2 &+& -0.4x_3 & = 0 \\
    x_1 &+& x_2 &+& x_3 & = 1
    \end{matrix}[/itex]

    I put this in matrix form:
    [itex]\begin{bmatrix}
    -0.9 & 0.7 & 0.2 & | 0\\
    0.1 & -0.2 & 0.1 & | 0\\
    0.3 & 0.1 & -0.4 & | 0\\
    1 & 1 & 1 & | 1
    \end{bmatrix}[/itex]

    and solve:
    [itex]\begin{bmatrix}
    1 & 0 & 0 & | \frac{1}{3}\\
    0 & 1 & 0 & | \frac{1}{3}\\
    0 & 0 & 0 & | 0\\
    0 & 0 & 1 & | \frac{1}{3}
    \end{bmatrix}[/itex]

    However this is wrong as the answer states:
    [itex]\boldsymbol{v}=\begin{bmatrix}
    0.1+0.2\frac{3}{16} \\
    0.9-1.2\frac{3}{16} \\
    \frac{3}{16} \\
    \end{bmatrix}[/itex]


    Can any one tell me were i am going wrong? Not entirely sure how to find the steady state of a markov chain. The only examples i can find are examples of steady states in 2x2 transition matrices.
     
  2. jcsd
  3. Apr 9, 2013 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    You need to solve πP = π for a row vector π, not Pv = v for a column vector v. Also, you should leave out one of the three equations in π = πP and replace it by sum(π) = 1.

    Note that *some* sources use transition matrices where the columns sum to 1 instead of the rows; that is, their transition matrices are transposes of yours. In those cases it is, indeed, true that you would need to solve Tv = v, where T is the transition matrix (with column-sums = 1). Every textbook or research monograph I have ever seen about probability and its applications, or about Operations Research, etc., uses the convention that rows sum to 1. For some mysterious reason, many web pages use the other convention, and that can be dangerous if you do not check before using their results. Of course, there are hundreds of books I have not seen, and some of them may use the column-sum = 1 convention; I think it is more common in Asia.
     
    Last edited: Apr 9, 2013
  4. Apr 10, 2013 #3
    Thnx for the reply!! I wasn't aware of this row vector convention, seems rather strange to me. Now i just transpose the transition matrix and use the 'normal method'.
     
  5. Apr 10, 2013 #4

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    What I wrote is not a 'vector convention'. The steady-state distribution obeys some equations, and the issue is: how do you remember those equations? The easiest way is to recognize that they *can be written as* π = πP for a row vector π; but that is not the original source of the equations, it is just a memory aid.

    The real issue is to always check with a reference as to whether their 'transition' matrices have row-sums or column-sums equal to 1.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Markov Chain Steady State (?were am i going wrong?)
  1. Where am I going wrong (Replies: 8)

Loading...