First step analysis hitting time

[rickywaldron]

I'm having a little trouble understanding the mean hitting time of a random walk, with p(i,j) = p if j=i+1, q if j=i-1 and 0 otherwise. 0 is an absorbing state and no upper absorbing state ie. dealer has unlimited amount of money.

Need to work out the mean hitting times k(i)(0) for i=0,1,2 etc.

I believe that i is the initial starting point however wouldn't the mean hitting time be different for each i? The way the question is worded sounds like same answer for all i.

Need to calculate for the cases where p<q, p=q and p>q and I have the theorem k(i)(0) = sum (over j does not equal 0) of p(i,j)k(j,0)
I don't get what this theorem means?

Thanks

 

[Valkarie]

The mean hitting time k(i)(0) is the average number of steps it takes to reach state 0, starting from state i. The theorem you have states that the mean hitting time k(i)(0) can be calculated by summing up all the probabilities p(i,j) multiplied by the mean hitting time k(j,0) for each non-zero state j. Intuitively, this makes sense because the probability p(i,j) tells us how likely it is to move from state i to state j, and then the mean hitting time k(j,0) tells us how long it will take to reach state 0 from state j. Note that this theorem only applies when there is no upper absorbing state. So, in order to calculate the mean hitting time k(i)(0) for a given initial state i, you need to compute the probabilities p(i,j) and mean hitting times k(j,0) for each non-zero state j, and then multiply each probability with the corresponding mean hitting time, and sum up all the results. The values of p(i,j) will depend on whether p<q, p=q or p>q.