Markov renewal process

Relation to other stochastic processes

1. Let ${\displaystyle X_n}$ and ${\displaystyle T_n}$ be as defined in the previous statement. Defining a new stochastic process ${\displaystyle Y_t:=X_n}$ for ${\displaystyle t\in [T_n,T_{n+1})}$, then the process ${\displaystyle Y_t}$ is called a semi-Markov process as it happens in a continuous-time Markov chain. The process is Markovian only at the specified jump instants, justifying the name semi-Markov. (See also: hidden semi-Markov model.)
2. A semi-Markov process (defined in the above bullet point) in which all the holding times are exponentially distributed is called a continuous-time Markov chain. In other words, if the inter-arrival times are exponentially distributed and if the waiting time in a state and the next state reached are independent, we have a continuous-time Markov chain.

   ${\displaystyle \begin{array}{cc}& Pr({\tau }_{n+1}\le t,X_{n+1}=j\mid (X_0,T_0),(X_1,T_1),\dots ,(X_n=i,T_n))\\ =& Pr({\tau }_{n+1}\le t,X_{n+1}=j\mid X_n=i)\\ =& Pr(X_{n+1}=j\mid X_n=i)(1-e^{-{\lambda }_{i}t}),\text{ for all }n\ge 1,t\ge 0,i,j\in \mathrm{S},i\ne j\end{array}}$

3. The sequence ${\displaystyle X_n}$ in the Markov renewal process is a discrete-time Markov chain. In other words, if the time variables are ignored in the Markov renewal process equation, we end up with a discrete-time Markov chain.

   ${\displaystyle Pr(X_{n+1}=j\mid X_0,X_1,\dots ,X_n=i)=Pr(X_{n+1}=j\mid X_n=i)\mathrm{\forall }n\ge 1,i,j\in \mathrm{S}}$

4. If the sequence of ${\displaystyle \tau }$s is independent and identically distributed, and if their distribution does not depend on the state ${\displaystyle X_n}$, then the process is a renewal. So, if the states are ignored and we have a chain of iid times, then we have a renewal process.

   ${\displaystyle Pr({\tau }_{n+1}\le t\mid T_0,T_1,\dots ,T_n)=Pr({\tau }_{n+1}\le t)\mathrm{\forall }n\ge 1,\mathrm{\forall }t\ge 0}$