   Next: Long-Run Behavior of Markov Up: Discrete Time Markov Chains Previous: State Probabilities

## Transition Probabilities

The one-step transition probability is the probability of transitioning from one state to another in a single step. The Markov chain is said to be time homogeneous if the transition probabilities from one state to another are independent of time index . The transition probability matrix, , is the matrix consisting of the one-step transition probabilities, .

The -step transition probability is the probability of transitioning from state to state in steps. The -step transition matrix whose elements are the -step transition probabilities is denoted as .

The -step transition probabilities can be found from the single-step transition probabilities as follows.

To transition from to in steps, the process can first transition from to in steps, and then transition from to in steps, where . In matrix form, this becomes: Setting yields: From this equation we can see that: Substituting this back into the previous equation yields: Continuing these substitutions, eventually we have: Therefore, the -step transition probability matrix can be found by multiplying the single-step probability matrix by itself times.

The state vector at time can also be found in terms of the transition probability matrix and the intial state vector . We first observe that: In vector and matrix form, this becomes: We also find that, through substitution: or, Continuing the substitution yields: where is the vector containing the intial probabilities of being in each state at time 0.