Markov Chains
Discrete Time Markov Chains
State Probabilities
Transition Probabilities
Long-Run Behavior of Markov Chains
Continuous Time Markov Chains