- #1
Edwinkumar
- 23
- 0
Can someone prove that an irreducible markov chain on a finite state space {0,1,...,m} is not a Martingale?
quadraphonics said:Consider the case where [itex]S_n = 0[/itex]. Then the Martingale condition would be [itex]E(S_{n+1}|S_n=0) = 0[/itex], which would require that [itex]P(S_{n+1}=0|S_n=0)=1[/itex], which violates the assumption of irreducibility.
quadraphonics said:Since there is no "edge" to the state space, it's easy to construct non-trivial conditional distributions with the required expected values, which then gives an irreducible chain. Can you think of an example?
Boxcar Billy said:If [itex]P(S_{n+1} = -1 | S_{n} = 0) = 0.5[/itex] and [itex]P(S_{n+1} = 1 | S_{n} = 0) = 0.5[/itex] then the Martingale condition still holds because the expected value is still 0. Is this right or am I missing something?
Edwinkumar said:1) How can you assume that [itex]S_n=0[/itex]?
Edwinkumar said:2) How can you condition [itex]S_n[/itex] instead of [itex]\mathcal{F}_n[/itex]?
Edwinkumar said:3) Moreover, can you define some stopping time [itex]\tau[/itex] so that the stopped process is a Martingale?
With respect to the finite state irreducible markov chain.quadraphonics said:A stopping time with respect to what stochastic process? A finite-state Markov Chain? Or a martingale?
quadraphonics said:Notice that this is not the case for Markov Chains with infinite state spaces. Since there is no "edge" to the state space, it's easy to construct non-trivial conditional distributions with the required expected values, which then gives an irreducible chain. Can you think of an example?
Edwinkumar said:With respect to the finite state irreducible markov chain.
Edwinkumar said:I don't understand why is it not working in case of a an irreducible Markov chain with infinite state space. Can you please explain to me?
No! I want a stopping time(an integer valued random variable) [itex]\tau[/itex] for my finite state irreducible markov chain [itex]S_n[/itex] such that the stopped process [itex]S_{\tau \wedge n}[/itex] is a Martingale.quadraphonics said:Maybe [itex]\sum_{i=0}^{n}S_i/n[/itex] would work?[/itex].
Edwinkumar said:Do you mean either [itex]\tau=0[/itex] or [itex]\tau=m[/itex]?
Edwinkumar said:Moreover, can you give an example of a Martingale which is not a Markov chain?
quadraphonics said:No, [itex]\tau[/itex] will be whatever time step [itex]S_n[/itex] first equals either 0 or m.
Edwinkumar said:Do you mean [itex]\tau(\omega)=min\{n: S_n(\omega)=0 or S_n(\omega)=m\}[/itex]?
Edwinkumar said:Thank you very much quadraphonics.
One final question.
Can you prove that the stopped process [itex]Y_n=X_{\tau \wedge n}[/itex], where [itex]\tau(\omega)=min\{n: S_n(\omega)=0 or S_n(\omega)=m\}[/itex] is a martingale w.r.to the natural filtration [itex]\mathcal{F}_n=\sigma(X_0, X_1,..., X_n)[/itex]
An Irreducible Markov Chain is a type of stochastic process in which the future state of the system depends only on its current state and not on any of its previous states. It is also known as a homogeneous Markov chain because the transition probabilities between states remain constant over time.
To prove that a Markov Chain is irreducible, we need to show that there is a non-zero probability of transitioning from any state to any other state in the chain. This can be done by constructing a transition matrix and showing that it is possible to reach any state from any other state by following a sequence of transitions.
A Martingale is a type of stochastic process in which the expected value of the next state in the chain is equal to the current state, regardless of the previous states. In other words, the future state of the system is independent of the past states.
To prove that an Irreducible Markov Chain is not a Martingale, we need to show that the expected value of the next state does not equal the current state. This can be done by constructing a counterexample, where the transition probabilities between states do not satisfy the Martingale property.
The applications of proving Irreducible Markov Chains not Martingales include understanding the behavior of complex systems, predicting future states in stochastic processes, and developing efficient algorithms for various problems in mathematics, physics, and computer science.