# Markov chain problem

1. Nov 17, 2008

### subopolois

1. The problem statement, all variables and given/known data
i have a scenario which i have to find the proportion of time spent in each area by a person using markov chains. i was given a word problem, which i have put into a matrix and the question asks what the proportion of time is spent in each area A, B and C.

2. Relevant equations

na

3. The attempt at a solution
i have the final matrix as:
0 0.34 0.67
0 0 0.34
1 0.34 0

im stuck at how to determine how to do this, can someone help?

2. Nov 18, 2008

### Pere Callahan

Find the stationary distribution of the Markov chain (that is an eigen vector to the transition matrix)

3. Nov 18, 2008

### subopolois

we havent been taught eigen vectors yet

4. Nov 18, 2008

### D H

Staff Emeritus
That is not a valid transition matrix. Each row must sum to 1.

5. Nov 18, 2008

### subopolois

sorry, i made a typo, its:
0 0.67 0.67
0 0 0.34
1 0.33 0

6. Nov 18, 2008

### D H

Staff Emeritus
That's still not valid. In fact, it's worse; now the first row is also invalid. The sum of each row must be identically one. What is the word problem that led to this matrix?

7. Nov 18, 2008

### subopolois

heres the word problem:
A fox hunts in three territories A, B and C. He
never hunts in the same territory on two successive days.
If he hunts in A, then he hunts in C the next day. If he
hunts in B or C, he is twice as likely to hunt in A the next
day as in the other territory.

i only gave the scenerio, i dont need help with the actual questions, just how to get the initial transition mtrix. its just confusing for me reading the problem and transfering it to a mathematica matrix. i am going by the examples in the text book in terms of the rows adding up to 1, in my text book it has the columns adding to 1

8. Nov 18, 2008

### D H

Staff Emeritus
Sorry for the misdirection. Your corrected matrix is fine.

Suppose the probabilities that the fox at the nth time step tn is in state A, B, or C are PA(tn), PB(tn), and PC(tn). Let P(tn) be the column vector formed from these individual state probabilities. The state probabilities at the next time step are given by the transition matrix S: P(tn+1)=S×P(tn).

The system is in steady-state if P(tn+1)=P(tn). Try to find this steady-state probability vector (the components must add to one).