This article is cited in
3 papers
On Derived and Nonstationary Markov Chains
J. W. Cohen Mathematical Institute,Technological University, Delft
Abstract:
Given a stationary Markov chain
${}_1M$, with a countable set of states
$\mathscr{E}$, two new, nonstationary Markov chains
${}_2M$ and
${}_3M$ are formed from
$\mathscr{E}$, according to the following rules (henceforth,
${}_i P_h,i=1,2,3$, denotes the transition-probability matrix at the
$m$-th step of the chain
${}_i M$, and
${}_1P_h = {}_1 P$):
$${}_2 P_h=\sum\limits_{n=0}^\infty {a_{nh_1}P^n},$$
where
$$0\leq a_{nh}\leq 1,\quad\sum\limits_{n=1}^\infty{a_{nh}=1},\quad\mathop{\sup}\limits_h a_{0h}<1$$
and
${}_2M$ is called the derived chain, while
$${}_3P_h={}_2P_h+R_h,$$
where
$$\sum\limits_{h=1}^\infty{\left\|{R_h} \right\|}<\infty$$
and
${}_3M$ is called the perturbated chain. We study the problem of how various characteristics of the same state (e.g., return properties, periodicity, ergodicity), as well as certain other qualitative and quantitative indices of the chains, are interrelated in the chains
${}_1M$,
${}_2M$ and
${}_3M$. The results obtained can be generalized to the case of Markov chains with a continuous set of states, and similar constructions can be carried out for the case off continuous time.
Received: 28.06.1960
Language: English