1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Finding the rate of convergence for a markov chain

Tags:
  1. Oct 25, 2015 #1
    1. The problem statement, all variables and given/known data
    For the following Markov chain, find the rate of convergence to the stationary distribution:
    [itex] \begin{bmatrix} 0.4 & 0.6 \\ 1 & 0 \end{bmatrix} [/itex]


    2. Relevant equations

    none

    3. The attempt at a solution

    I found the eigenvalues which were [itex] \lambda_1=-.6 [/itex] or [itex] \lambda_2=1 [/itex]. The corresponding eigenvectors I found were [itex] \vec{v_1}=\begin{bmatrix} -0.6 & 1 \end{bmatrix}[/itex] and [itex] \vec{v_2}=\begin{bmatrix} -1 & 1 \end{bmatrix} [/itex]. The stationary distribution which I found that satisfies [itex] p=pA [/itex](A is the transition matrix) and [itex]p_1+p_2=1[/itex] is [itex] \vec{p}=\begin{bmatrix} .625 & .375 \end{bmatrix} [/itex]. From here I do not know how to get the rate of convergence. I think it has something to do with the eigenvalues or eigenvectors. Any help would be great thanks.
     
    Last edited: Oct 25, 2015
  2. jcsd
  3. Oct 25, 2015 #2

    andrewkirk

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

  4. Oct 25, 2015 #3
    I checked the eigenvectors. I had them switched. So do I use the formula right below where it "since the eigenvectors are orthonormal..."?
     
  5. Oct 26, 2015 #4

    andrewkirk

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    That depends on what you think it does. It's best to only use formulas that one understands. Without going through every step of the maths, do you understand broadly why that formula has that form?
     
  6. Oct 26, 2015 #5
    Actually I don't. We talked about limiting distributions and stationary distributions in class but did not cover rate of convergence. In essence I only understand the very basics. So somehow I have to show the rate of convergence with what I have so far.
     
  7. Oct 26, 2015 #6

    andrewkirk

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    Well, the matrix eigenvectors will be orthogonal and hence form a basis for the vector space. So any vector ##\mathbf{x}## can be expressed as a linear sum ##\sum_{i=1}^na_i\mathbf{u}_i## of eigenvectors, as shown in the wiki article. If only one of the eigenvalues, say ##\lambda_1##, corresponding to eigenvector ##\mathbf{u}_1##, has absolute value equal to 1, and all others have absolute values less than 1, what will happen to ##\mathbf{x}=\sum_{i=1}^na_i\mathbf{u}_i## as you repeatedly right-multiply it by the matrix? Think about the impact of each term in the sum separately.
     
  8. Oct 26, 2015 #7

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Please tun off the bold font, reserving it for emphasis on individual words, etc, like what I do below.

    Anyway, do you know the definition of rate of convergence? Is this discussed in your textbook or course notes? Even if it is in a section of your text that you have not reached yet, I would bet you would be allowed to read it ahead of time and use the material as needed. If such material is not in your text or course notes, please tell us so we can point you in the appropriate direction.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Finding the rate of convergence for a markov chain
  1. Markov Chain (Replies: 1)

Loading...