Exploring Applications of Markov Chains

[Firepanda]

Basically I have to write a 2 page report briefly on some applications of markov chains, other than Population dynamics and gambling scenarios.

It would be great if someone could give me some ideas on relevant applications of these, fairly simple ofc :P The more indepth i read into markov chains, the less i understand :)

I've tried googling some stuff, I saw an application of markov chains in a squash game. I can see relevance as the shot your oponent makes next is based upon your shot now in the present. But surely the shot in the present is dependant on the shot your opponent played in the past, which to me isn't the definition of a markov chain. So i don't see how this can work.

But any other ideas I could use? :) Thanks

 

[Firepanda]

Noone has any examples? :(

 

[PowerIso]

Hm...when have I used markov chains, Queuing theory, statistical mechanics

geology
http://www.stieltjes.org/archief/rep9899/node19.html

economics
http://faculty.darden.virginia.edu/pfeiferp/Homepage/ModelingCustomersasMarkovChains.pdf
http://www.stat.fi/isi99/proceedings/arkisto/varasto/pell0333.pdf

 

[Firepanda]

Awesome thanks :D

 

[Firepanda]

It turns out I am having trouble understandng transition matrices, especially from the link:

http://faculty.darden.virginia.edu/pfeiferp/Homepage/ModelingCustomersasMarkovChains.pdf

What does each row/column tell me in transition matrices? Using the example in the link, on page 9 would be helpful :) Also when moving up transition step matrices in the example, I see that the matrix P(1) already has its first column explained in the question, but for the 2nd, 3rd and 4th step transtion matrices I don't see where the 1st column in those come from.

thanks!

 

[Firepanda]

Ok now I know what the rows and columns stand for. But I'm still unsure what the transition steps mean. In the example are the transition steps different periods in time, for set periods, and there are 5 states within each transition matrix for all the steps that happen within a period?

I also still can't figure out where how i achieve the P^2, p^3 etc. transiotion matrices, I THINK I have to right multiply it by a column vector, but how do I get the vector?

Thanks

 

[Firepanda]

/facepalm

Sorry I was being stupid, I see the P^2 matrices were just where P^2=PP.

Still though, I don't know what the matrices mean :P In terms of what the probabilities mean inside the matrix, and what the differnet steps show us.

 

[ozymandias]

You're forgetting perhaps the most important application of Markov chains - they power the search engine you've just used ... (Google) :). Google uses a Markov chain for its so called "link analysis". I recommend reading the book "Google's pagerank and beyond", or just, well, Googling it ;).

--------
Assaf
http://www.physicallyincorrect.com/"

 

[Firepanda]

You're forgetting perhaps the most important application of Markov chains - they power the search engine you've just used ... (Google) :). Google uses a Markov chain for its so called "link analysis". I recommend reading the book "Google's pagerank and beyond", or just, well, Googling it ;).

--------
Assaf
http://www.physicallyincorrect.com/"

ooh nice, sounds very interesting, it seems its a book though, and I can't seem to find any free info of the math involved :P Could you briefly descibe the markoviness behind it? ;)

 

[Firepanda]

Scrub everything I asked above, now I would like to know stuff about google =P

I found this link here:
http://www.mathworks.com/company/newsletters/news_notes/clevescorner/oct02_cleve.html

It seems everywhere I look you need a few more years of education that what I have to understand, especially since the google owners devised the algorithm at the end of their degree =)

Has anyone here read the book 'Google's pagrerank and beyond'? Or knows anything to do with it and its relevance to Markov chains? Just briefly explained would be awesome! I'm sure there's a way to break down all of the hard notation into simpler english :).

thanks