Do Markov Chain Transition Matrices Sum by Row or Column?

[Jamin2112]

Homework Statement

Homework Equations

The Attempt at a Solution

A few things.

First of all, the homework problem notes that "all the columns should sum to 1," whereas Wikipedia says ∑P_ij = 1 when we sum all along the the row i.

Second of all, I don't know where to go after I've constructed my transition matrix. A hint would be much appreciated.

 

[chiro]

Are you having problems with part b?

Basically you are interested in player 10 having a turn on turns 1-5 which means that you are interested in p_(x,5)^(n) where x is the initial state and n is the number of iterations. Based on that, what do you think you need to do (Remember in the homework forum, we can't do your work for you, but give you hints).

With regard to the sum of all elements along the rows being 1 that is correct. An easy way to think about this is that all probabilities in one row are those of disjoint events.

For example the probability p_(0,0) and p(0,1) are disjoint and all the probabilities of p(0,x) where x is any valid state must equal 1 because all possible probabilities starting from 0 and going to something else are considered and there can't be anymore.

Part c is more algorithmic and I'm sure you have the algorithm in your notes. Part d asks you to interpret your results from part c.

 

[Ray Vickson]

Homework Statement

Homework Equations

The Attempt at a Solution

A few things.

First of all, the homework problem notes that "all the columns should sum to 1," whereas Wikipedia says ∑P_ij = 1 when we sum all along the the row i.

Second of all, I don't know where to go after I've constructed my transition matrix. A hint would be much appreciated.

The vast majority of books and papers use the standard convention in which rows sum to 1. However, I have seen a few papers (mostly from Asian sources) that take the other convention of having columns summing to 1. Basically, one matrix is just the transpose of the other. You should stick to whatever convention your instructor uses, at least when writing up the final solution.

RGV