Next: State Probabilities
A Markov chain is a discrete state space process in which the
next state depends only on the present state.
For a discrete time system,
if is the state of the system at time ,
is a Markov chain if:
i.e., the state of the system at time depends only on
the state of the system at time , and does not depend on
any other state before time .