Markov Chains

  • Thread starter srose
  • Start date
  • #1
4
0

Main Question or Discussion Point

Hi
Can anyone help me with the following question from a past paper I am working through?
http://imageshack.us/photo/my-images/607/capturect.jpg/

I'm not quite sure what I do. Do I square the martrix?

Any help would be great
Thanks
 

Answers and Replies

  • #2
22,097
3,278
You need to calculate [tex]\pi^{(0)}P^2[/tex]. Do you see why?
 
  • #3
4
0
not completely sure.

What exactly is pi in all this. Is it the probability that it will stay in state 0 (down) and therefore has (1/4) chance of staying in down, 1/2 chance of going to usable and 1/4 chance of going to overloaded?
 
  • #4
22,097
3,278
[tex]\pi^{(0)}[/tex] is the initial probability. Thus we have 1/4 chance the the computer is initially down, we have 1/2 chance that it is initially usable and 1/4 chance that the computer is overloaded.

Then our Markov chain goes to the next state, and then we have [tex]\pi^{(1)}[/tex]. The first coordinate is the chance that the computer is down in the first step, the second coordinate is the chance that the computer is usable in the first step and the third coordinate is the chance that the computer is overloaded in the first step.

Then our Markov chain goes to the next state to obtain [tex]\pi^{(2)}[/tex]. The coordinates are the chance that the computer is down/usable/overloaded in the second step.
 
  • #5
4
0
ok, thanks very much, I think I understand it now
 
  • #6
As far as part b) goes my notes don't seem to mention what makes a DTMC regular? Is that just the fact that P and initial distribution completely characterize the chain?
 
  • #7
22,097
3,278
A regular Markov chain is a chain such that a power of the transition matrix P only has strict positive elements. This is trivial in your case since P already has strict positive elements. So the Markov chain in question is regular.

I think there has to be another (equivalent) definition of regular that you need to check, but I don't know what it is of course.

For more information on regular, see http://www.google.be/url?sa=t&source=web&cd=3&ved=0CC0QFjAC&url=http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf&rct=j&q=regular markov matrix&ei=fIPNTdL6F46YOufbpYAN&usg=AFQjCNEogEyES31QtziNw6NF5ftruRMuMg&cad=rja"
 
Last edited by a moderator:

Related Threads for: Markov Chains

  • Last Post
Replies
17
Views
4K
  • Last Post
Replies
10
Views
3K
  • Last Post
Replies
1
Views
2K
  • Last Post
Replies
1
Views
2K
  • Last Post
Replies
7
Views
1K
  • Last Post
Replies
11
Views
9K
  • Last Post
Replies
0
Views
6K
  • Last Post
Replies
1
Views
5K
Top