In a continuous time Markov chain, the state transitions may occur at any time, and the time between transitions is exponentially distributed. Since the exponential distribution is memoryless, the future outcome of the process depends only on the present state and does not depend on when the last transition occurred or what any of the previous states were.
We denote the state of the system at time as .
The state probability at time is the probability that the system
is in state at time , and is denoted as:
The steady-state or limiting probability of being in state is:
For a continuous time Markov chain, we can define its intensity matrix or rate matrix, . The elements of indicate the rate of transitions from state to state for . In other words, the time to make a transition to state given that the process is in state is exponentially distributed with rate parameter . For , .
The steady-state probabilities can be found from using: