PDA

View Full Version : Markov Chain of Stochastic Processes


stargazer_iq
Oct7-09, 10:17 AM
I would like to construct a model using a markov chain that has different stochastic processes for each state in the chain. Is there a term for such a thing, or anything similar to it?
Thanks

EnumaElish
Oct7-09, 02:37 PM
Do you mean: "the probability of going from state r to state s is a function of the time index, t"? That concept is a "time-heterogenous" (or a "non-stationary") Markov chain:

http://en.wikipedia.org/wiki/Markov_chain#Variations

stargazer_iq
Oct8-09, 07:40 AM
i found something similar to what I'm looking for in this paper:
HS Chang, "Multitime Scale Markov Decision Processes", 2003