Help Needed: Squaring a Matrix - Any Advice Appreciated!

[srose]

Hi
Can anyone help me with the following question from a past paper I am working through?
http://imageshack.us/photo/my-images/607/capturect.jpg/

I'm not quite sure what I do. Do I square the martrix?

Any help would be great
Thanks

 

[micromass]

You need to calculate \pi^{(0)}P^2. Do you see why?

 

[srose]

not completely sure.

What exactly is pi in all this. Is it the probability that it will stay in state 0 (down) and therefore has (1/4) chance of staying in down, 1/2 chance of going to usable and 1/4 chance of going to overloaded?

 

[micromass]

\pi^{(0)} is the initial probability. Thus we have 1/4 chance the the computer is initially down, we have 1/2 chance that it is initially usable and 1/4 chance that the computer is overloaded.

Then our Markov chain goes to the next state, and then we have \pi^{(1)}. The first coordinate is the chance that the computer is down in the first step, the second coordinate is the chance that the computer is usable in the first step and the third coordinate is the chance that the computer is overloaded in the first step.

Then our Markov chain goes to the next state to obtain \pi^{(2)}. The coordinates are the chance that the computer is down/usable/overloaded in the second step.

 

[srose]

ok, thanks very much, I think I understand it now

 

[QuendeltonPG]

As far as part b) goes my notes don't seem to mention what makes a DTMC regular? Is that just the fact that P and initial distribution completely characterize the chain?

 

[micromass]

A regular Markov chain is a chain such that a power of the transition matrix P only has strict positive elements. This is trivial in your case since P already has strict positive elements. So the Markov chain in question is regular.

I think there has to be another (equivalent) definition of regular that you need to check, but I don't know what it is of course.

For more information on regular, see http://www.google.be/url?sa=t&sourc...g=AFQjCNEogEyES31QtziNw6NF5ftruRMuMg&cad=rja"