# Homework Help: Markov Chains

1. Dec 4, 2007

### Firepanda

Basically I have to write a 2 page report briefly on some applications of markov chains, other than Population dynamics and gambling scenarios.

It would be great if someone could give me some ideas on relevant applications of these, fairly simple ofc :P The more indepth i read into markov chains, the less i understand :)

I've tried googling some stuff, I saw an application of markov chains in a squash game. I can see relevance as the shot your oponent makes next is based upon your shot now in the present. But surely the shot in the present is dependant on the shot your opponent played in the past, which to me isn't the definition of a markov chain. So i don't see how this can work.

But any other ideas I could use? :) Thanks

2. Dec 5, 2007

### Firepanda

Noone has any examples? :(

3. Dec 5, 2007

### PowerIso

Last edited by a moderator: May 3, 2017
4. Dec 5, 2007

### Firepanda

Awesome thanks :D

5. Dec 6, 2007

### Firepanda

It turns out im having trouble understandng transition matrices, especially from the link:

http://faculty.darden.virginia.edu/pfeiferp/Homepage/ModelingCustomersasMarkovChains.pdf [Broken]

What does each row/column tell me in transition matrices? Using the example in the link, on page 9 would be helpful :) Also when moving up transition step matrices in the example, I see that the matrix P(1) already has its first column explained in the question, but for the 2nd, 3rd and 4th step transtion matrices I don't see where the 1st column in those come from.

thanks!

Last edited by a moderator: May 3, 2017
6. Dec 7, 2007

### Firepanda

Ok now I know what the rows and columns stand for. But I'm still unsure what the transition steps mean. In the example are the transition steps different periods in time, for set periods, and there are 5 states within each transition matrix for all the steps that happen within a period?

I also still can't figure out where how i acheive the P^2, p^3 etc. transiotion matrices, I THINK I have to right multiply it by a column vector, but how do I get the vector?

Thanks

7. Dec 7, 2007

### Firepanda

/facepalm

Sorry I was being stupid, I see the P^2 matrices were just where P^2=PP.

Still though, I don't know what the matrices mean :P In terms of what the probabilities mean inside the matrix, and what the differnet steps show us.

8. Dec 7, 2007

### ozymandias

You're forgetting perhaps the most important application of Markov chains - they power the search engine you've just used ... (Google) :). Google uses a Markov chain for its so called "link analysis". I recommend reading the book "Google's pagerank and beyond", or just, well, Googling it ;).

--------
Assaf
http://www.physicallyincorrect.com/" [Broken]

Last edited by a moderator: May 3, 2017
9. Dec 7, 2007

### Firepanda

ooh nice, sounds very interesting, it seems its a book though, and I can't seem to find any free info of the math involved :P Could you briefly descibe the markoviness behind it? ;)

Last edited by a moderator: May 3, 2017
10. Dec 7, 2007