Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Question on Discrete Parameter Markov Chains

  1. Apr 23, 2006 #1

    AD

    User Avatar

    I am required to find a formula expressing the probability of return to some state in a Markov chain at time n in terms of the probability of return to that state at time n - k and the probability of first return at time k. I cannot find this in my notes, and I have tried looking at several online resources. Can anyone help me?
     
  2. jcsd
  3. Apr 23, 2006 #2

    AD

    User Avatar

    It is no longer necessary for you to answer this question as I have just discovered the answer elsewhere.
     
  4. Apr 23, 2006 #3

    Hurkyl

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    You could just derive it. How many ways can you return at time n?

    One way is to have time n be the first time you return.
    A second way is to return at time 1, and then have n be the next time you return.
    Yet another way is to return at time 2, and then have n be the next time you return...
     
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook