MATHEMATICS

M. ROSENBLATT-ROT

ON THE STRONG LAW OF LARGE NUMBERS FOR NONHOMOGENEOUS MARKOV CHAINS

(Presented by Academician A. N. Kolmogorov on 26 VII 1961)

Let \(\alpha_i\) be the ergodicity coefficient of the \(i\)-th probabilistic transition function of a certain, in general nonstationary, Markov chain (see \((^1,^2)\)), and denote
\[ \eta_n=\max_{1\le i\le n-1}(1-\alpha_i),\qquad 1-\eta_n=O(n^{-\beta})\quad (0\le \beta<1). \]

Theorem 1. If a sequence of random variables \(\xi_i\) \((i\in I)\)*, connected in a nonhomogeneous Markov chain with nonzero ergodicity coefficients \(\alpha_i>0\) \((i\in I)\), satisfies the condition
\[ \sum_{n=1}^{\infty}\frac{\mathbf D\xi_n}{n^{2-\beta}}<+\infty, \]
then it obeys the strong law of large numbers.

If \(\alpha_i>\rho>0\) \((i\in I)\), this condition takes the form
\[ \sum_{n=1}^{\infty}\frac{\mathbf D\xi_n}{n^2}<+\infty, \]
and it is best possible in the sense that if, for some sequence of nonnegative constants \(b_n\), the series
\[ \sum_{n=1}^{\infty}\frac{b_n}{n^2} \]
diverges, then one can construct a sequence of random variables \(\xi_n\) \((n\in I)\), connected in a Markov chain (not expressible as a sequence of independent random variables), with \(\mathbf D\xi_n=b_n\), \(\beta=0\), which does not obey the strong law of large numbers.

Theorem 2. If, under the conditions of Theorem 1, \(\mathbf D\xi_i\le C<\infty\) \((i\in I)\), then the sequence \(\xi_i\) \((i\in I)\) obeys the strong law of large numbers.

Theorem 3. If in a discrete Markov chain with \(\alpha_i>0\) \((i\in I)\) the probability of the occurrence of event \(i\) in the \(k\)-th trial is equal to \(p_k^{(i)}\), and \(\mu^{(i)}\) denotes the number of occurrences of event \(i\) in the first \(n\) trials, then
\[ \mathbf P\left\{\lim_{n\to\infty}\left(\frac{\mu^{(i)}}{n}-\frac{p_1^{(i)}+p_2^{(i)}+\cdots+p_n^{(i)}}{n}\right)=0\right\}=1. \]

Let \(R\) be the real line and let \(\Omega=\{\omega_k\}\) be some finite or countable system of disjoint Borel sets on it, such that
\[ \bigcup_k \omega_k=R. \]
The totality of all real random variables \(\xi\)

* \(I\) is the totality of all natural numbers, \(\mathbf M\) is mathematical expectation, \(\mathbf D\) is variance.

we shall partition into nonintersecting classes \(\Lambda=\Lambda(\Omega)\), so that
\(\mathbf P\{|\xi|\in\omega_k\}=\varphi_\Lambda(k)\) depends only on \(\Lambda\), and not on \(\xi\in\Lambda\), for all \(k\). All \(\xi\) contained in one and the same \(\Lambda\) will be called \(\Omega\)-identically distributed. Obviously, if all random variables of some collection are identically distributed, then they are \(\Omega\)-identically distributed for any system \(\Omega\). Let

\[ \omega_k=\{k\le x<k+1\},\qquad \Omega=\{\omega_k\}\quad (k\in I). \]

Theorem 4. For a sequence of \(\Omega\)-identically distributed random variables \(\xi_n\in\Lambda\) \((n\in I)\), connected into a nonstationary Markov chain with \(\alpha_i>0\) \((i\in I)\), the existence of some random variable \(\xi\in\Lambda\) possessing a finite moment of order \(1+\beta\) is sufficient for the applicability of the strengthened law of large numbers.

If \(\alpha_i>\rho>0\) \((i\in I)\) (for example, in the case of independent \(\xi_n\)), for this it is sufficient that there exist some random variable \(\xi\in\Lambda\) possessing a finite mathematical expectation.

Theorem 5. The existence of the mathematical expectation is a necessary and sufficient condition for the applicability of the strengthened law of large numbers to a sequence of random variables that are identically distributed and connected into a homogeneous Markov chain with ergodicity coefficient \(\alpha>0\).

Theorem 6. Let \(\mu_i\) be the number of occurrences of event \(i\) in \(n\) consecutive trials according to the law of a homogeneous Markov chain with \(\alpha>0\), and let \(p_i\) be the probability of occurrence of event \(i\) in each of the trials; then

\[ \mathbf P\left\{\lim_{n\to\infty}\frac{\mu_i}{n}=p_i\right\}=1. \]

In the proof of Theorem 1 the following results are used.

Lemma 1. If the random variables \(\xi_i\) \((i\in I)\) are connected into a nonhomogeneous Markov chain with nonzero ergodicity coefficients \(\alpha_i>0\) \((i\in I)\) and have finite variances, then for any \(\varepsilon>0\) the inequality

\[ \mathbf P\left\{\max_{1\le \Delta\le n}\sum_{k=1}^{s}\left|(\xi_k-\mathbf M\xi_k)\right|>\varepsilon\right\} \le \frac{n^\beta}{\varepsilon_1^2}\sum_{i=1}^{n}\mathbf D\xi_i, \]

holds, where \(\varepsilon_1\) is a certain constant (depending on \(\varepsilon\) and on the chain).

If \(\alpha_i>\rho>0\) \((i\in I)\), this condition becomes

\[ \mathbf P\left\{\max_{1\le s\le n}\left|\sum_{k=1}^{s}(\xi_k-\mathbf M\xi_k)\right|>\varepsilon\right\} \le \frac{1}{\varepsilon_1^2}\sum_{i=1}^{n}\mathbf D\xi_i. \]

Lemma 2. In order that a sequence of random variables \(\xi_i\) \((i\in I)\), connected into a Markov chain, satisfy the strengthened law of large numbers, it is necessary that for every \(i>0\) the condition

\[ \sum_{n=1}^{\infty} \mathbf P\left\{ |\xi_n-\mathbf M\xi_n|> \frac{\varepsilon n}{|\xi_{n-1}-\mathbf M\xi_{n-1}|} \le \varepsilon(n-1) \right\}<+\infty \]

be fulfilled.

Whatever \(\varphi(n)=o(n)\) may be, this condition ceases to be necessary if in it \(\varepsilon n\) is replaced by \(\varphi(n)\).

Theorems 1–5 and Lemma 1 generalize the classical results of A. N. Kolmogorov (3–6), while Theorem 6 generalizes the classical Borel–Cantelli theorem; they are obtained from our results when \(\rho=1\), i.e. \(\alpha_i=1\) \((i\in I)\),

i.e., when the Markov chain degenerates into a sequence of independent random variables ((1), p. 78).

The class of chains satisfying Theorem 6 intersects with the classes of chains satisfying the conditions of V. I. Romanovskii ((7), p. 364) and T. A. Sarymsakov ((8), pp. 59, 61, 166). Lemma 2 generalizes a result of Yu. V. Prokhorov (9, 10).

Faculty of Mathematics and Physics, Parhon University
Bucharest, Romania

Received
28 XI 1960

REFERENCES

1. R. L. Dobrushin, Theory of Probability and Its Applications, 1, no. 1, 72 (1956).
2. R. L. Dobrushin, Theory of Probability and Its Applications, 1, no. 4, 365 (1956).
3. A. Kolmogoroff, C. R., 191, 910 (1930).
4. A. Kolmogoroff, Math. Ann., 99, 309 (1928).
5. A. N. Kolmogorov, Basic Concepts of Probability Theory, Moscow–Leningrad, 1936.
6. B. V. Gnedenko, A Course in Probability Theory, Moscow, 1954.
7. V. I. Romanovskii, Discrete Markov Chains, Moscow–Leningrad, 1949.
8. T. A. Sarymsakov, Foundations of the Theory of Markov Processes, Moscow, 1954.
9. Yu. V. Prokhorov, Dokl. AN, 69, no. 5, 607 (1949).
10. Yu. V. Prokhorov, Izv. AN SSSR, Ser. Matem., 14, 523 (1950).