Abstract
Full Text
MATHEMATICS
Academician of the Academy of Sciences of the Uzbek SSR T. A. SARYMSAKOV
ON INHOMOGENEOUS MARKOV CHAINS
1. In the study of inhomogeneous Markov chains governed by a sequence of transition probabilities ({P_n(x,\mathfrak S)}), various restrictions are imposed on the functions (P_n(x,\mathfrak S)) (we shall henceforth call them stochastic functions).* The task is to choose these restrictions so that they are not too strong. At the same time, in almost all works on inhomogeneous Markov chains the restrictions on the stochastic functions (P_n(x,\mathfrak S)) are such that the following requirement is satisfied for them:
((\mathrm R)). Let (P(x,\mathfrak S)) be a stochastic function defined on the domain ((X,F_Y)), and let (A) be an arbitrary set belonging to the system (F_Y). There exists a number (\lambda_P>0), independent of the choice of (A), such that at least one of the two inequalities holds:
[
\text{or}\qquad \sup_{x\in X} P(x,A)\leq 1-\lambda_P,
\tag{1}
]
[
\text{or}\qquad \sup_{x\in X} P(x,Y\setminus A)\leq 1-\lambda_P.
\tag{2}
]
The well-known proof of the ergodic theorem ((^1)) for inhomogeneous Markov chains with a finite number of states (in this case stochastic matrices ((^5)) are considered as the sequence of stochastic functions ({P_n(x,\mathfrak S)})) requires that, in the sequence of stochastic matrices governing the Markov chain, each matrix contain at least one column consisting entirely of positive elements bounded below by its own number. For such matrices condition ((\mathrm R)) is satisfied in an obvious way.
In the works of E. B. Dynkin ((^2)) and R. L. Dobrushin ((^3)) stochastic functions (P) satisfying the condition
[
(\mathrm D)\qquad
\sup_{\substack{x\in X,\ y\in X\ A\in F_Y}}
\left|P(x,A)-P(y,A)\right|=1-\lambda_P
\qquad (\lambda_P>0);
]
are considered; the number (\lambda_P) is called the ergodicity coefficient of the stochastic function (P).
In this case the following propositions are valid:
Lemma 1. If condition ((\mathrm D)) is satisfied for the stochastic function (P), then for any (A\;(\in F_Y)) one of the inequalities holds:
[
\text{or}\qquad \sup_{x\in X} P(x,A)\leq 1-\frac{\lambda_P}{2},
\tag{3}
]
[
\text{or}\qquad \sup_{x\in X} P(x,Y-A)\leq 1-\frac{\lambda_P}{2},
\tag{4}
]
i.e. condition ((\mathrm R)) is satisfied.
* The notation and definitions used in the present article are close to those in the work of A. N. Kolmogorov ((^4)). Only for greater definiteness shall we agree to write the domain of definition of the stochastic function (P(x,\mathfrak S)) in the form ((X,F_Y)), preserving the general conditions ((^4)) which it must satisfy.
Lemma 2. The fulfillment of one of the inequalities (1) or (2) entails the fulfillment of the relation
[
(\mathrm{D}') \qquad
\sup_{\substack{x\in X,\ y\in X\ A\in F_Y}}
\left|P(x,A)-P(y,A)\right|\leq 1-\lambda_P .
]
Definition 1. We shall say that the elements (x\;(\in X)) and (y\;(\in X)) intersect with respect to the stochastic function (P), defined in the domain ((X,F_Y)), on a set of probability measure (\alpha\;(>0)), if
[
\sup_{A\in F_Y}\left|P(x,A)-P(y,A)\right|=1-\alpha .
]
At the same time, if (\alpha=0), we shall say that (x) and (y) do not intersect.
The following proposition is of interest.
Lemma 3. If, with respect to the stochastic function (P), defined in the domain ((X,F_Y)), two elements (x\;(\in X)) and (y\;(\in X)) intersect on a set of probability measure (\alpha\;(>0)), then with respect to the composition (P*Q=R), where (Q) is any stochastic function defined in the domain ((Y,F_Z)), the same elements (x) and (y) intersect on a set of probability measure not less than (\alpha); we note here that the stochastic function (R) will be defined in the domain ((X,F_Z)).
Definition 2. One says that a Markov chain uniformly obeys the ergodic principle if, for every (m), uniformly with respect to (x\;(\in X_m)), (y\;(\in X_m)), and (A\;(\in F_{X_{m+n}})), the limiting equality
[
\lim_{n\to\infty}\left[P_{m,m+n}(x,A)-P_{m,m+n}(y,A)\right]=0
]
holds.
Theorem 1. In order that a Markov chain governed by a sequence of stochastic functions ({P_n(x,\mathfrak{S})}), defined respectively in the domains ((X_n,F_{X_{n+1}})), uniformly obey the ergodic principle, it is necessary and sufficient that there exist a sequence of positive integers
[
n_0=0<n_1<n_2<\cdots,
]
for which the inequalities
[
\sup_{\substack{x\in X_{n_{k-1}+1},\ y\in X_{n_{k-1}+1}\ A\in F_{X_{n_k}}}}
\left|P_{n_{k-1}+1,n_k}(x,A)-P_{n_{k-1}+1,n_k}(y,A)\right|
<1-\lambda_{n_k}
\tag{5}
]
hold, and
[
\text{the series }\sum_{k=1}^{\infty}\lambda_{n_k}\text{ diverges;}
\tag{6}
]
moreover,
[
P_{n_{k-1}+1,n_k}
=
P_{n_{k-1}+2}P_{n_{k-1}+3}\cdots*P_{n_k}.
]
Remark. It is clear that if in condition (5) the (\sup) is considered only over (A), while condition (6) is retained, then the ergodic principle will hold uniformly only with respect to (A). In this case the choice of the sequence ({n_k}) obviously depends on (x) and (y).
An illustration of this case may be the example considered by S. N. Bernstein ((1), p. 132).
2. We shall now consider inhomogeneous Markov chains with a finite number of states. By (\mathfrak{M}^{(s)}) we denote the collection of stochastic matrices (P), square or rectangular, with the number of rows and the number of columns not exceeding the number (s\;(\geq 2)), and not representable in the form
[
P=\begin{pmatrix}
T & 0\
L_1 & L_2
\end{pmatrix},
]
where (L_1, L_2), and (T) are nonnegative fields (i.e., submatrices), and the field (T) represents a (T)-matrix (5); moreover, for the field (T) the inequality (t \leqslant r) must also hold, where (t) is the number of columns and (r) the number of rows of the field (T). In this case a stochastic matrix (P=|p_{ij}|) will be called a (T)-matrix if there exists a partition of all rows into two nonempty parts (A) and (\bar A), and a partition of all columns into two nonempty parts (B) and (\bar B), such that: (1) (p_{ij}>0) when (i\in A) and (j\in B), or (i\in\bar A) and (j\in\bar B); (2) (p_{ij}\equiv 0) in all other cases.
Lemma 4. Let (s-1) stochastic matrices (P_1, P_2,\ldots,P_{s-1}) be given, belonging to the aggregate (\mathfrak N^{(s)}) ((s\geqslant 2)); moreover, the number of columns of the matrix (P_i) is equal to the number of rows of the matrix (P_{i+1}) ((i=1,2,\ldots,s-2)). Let (\alpha) and (\beta) be any pair of rows in the matrix (P_1) which do not intersect one another (see Definition 1). Then in the matrix
[
P_1 * P_2 * \cdots * P_{s-1}=P
]
these same rows (\alpha) and (\beta) will intersect on a set of positive probabilistic measure. In other words, the matrix (P) will satisfy condition (D). In this case the number (s-1) cannot, generally speaking, be decreased.
If a stochastic matrix (P) belongs to the class (\mathfrak N^{(s)}), but does not satisfy condition (D), then the coefficient of ergodicity for it in the sense indicated above is equal to zero. Examples of stochastic matrices belonging to the class (\mathfrak N^{(s)}), but not satisfying condition (D), are given in article (5). Therefore in this case one should introduce another definition for the coefficient of ergodicity.
For the case considered by us we shall define it as follows. By (p_{ij}^{+}) denote the positive elements of the matrix (P) ((\in \mathfrak N^{(s)})). The number
[
\lambda=\min_{i,j}(p_{ij}^{+})
]
we shall call the coefficient of ergodicity of the matrix (P).
Theorem 2. Let a sequence of stochastic matrices be given,
[
P_1,\ P_2,\ldots,P_n,\ldots,
]
regulating a Markov chain with a finite number of states, and let (P_h\in\mathfrak N^{(s)}) ((h=1,2,\ldots)) and have coefficient of ergodicity (\lambda_h). Then this chain is uniformly subject to the ergodic principle if the series (\sum_{k=1}^{\infty}\bar\lambda_k) converges, where
[
\bar\lambda_k=\lambda_{(k-1)s-k+2}\lambda_{(k-1)s-k+3}\cdots\lambda_{k(s-1)}
\qquad (k=1,2,\ldots).
]
The application of the conception set forth in the present article to the central limit theorem will be given by us in another article; here we shall note only that Theorem 2 makes it possible easily to obtain assumptions generalizing the results of the author’s article (6).
Central Asian State University
named after V. I. Lenin
Received
21 I 1958
CITED LITERATURE
- S. N. Bernstein, Theory of Probability, 1946.
- E. B. Dynkin, Ukr. Mat. Zh., 6, No. 1, 21 (1954).
- R. L. Dobrushin, Theory of Probability and Its Applications, 1, Nos. 1 and 4 (1956).
- A. N. Kolmogorov, Uspekhi Mat. Nauk, issue 5, 52 (1938).
- T. A. Sarymsakov, Kh. A. Mustafin, “On the ergodic theorem for nonhomogeneous Markov chains,” Tashkent, 1957 (Tr. Sredneaziatsk. Gos. Univ., new series, issue 74 (1957)).
- T. A. Sarymsakov, Dokl. AN UzSSR, No. 8, 3 (1956).