Full Text
V. A. Rokhlin
AN AXIOMATIC DEFINITION OF THE ENTROPY OF A TRANSFORMATION WITH INVARIANT MEASURE
(Presented by Academician A. N. Kolmogorov on 17 VIII 1962)
The aim of this note is to give a simple axiomatic definition of the entropy of an automorphism of a Lebesgue space. The proof of the equivalence of this definition with the Kolmogorov–Sinai definition takes little space, but in fact rests on the entire preceding development of the theory. One may hope that the new definition will make it possible to understand better the place of entropy in ergodic theory.
1. Preliminary information. A survey of the state of the theory of transformations with invariant measure as of 1960 is given in \((^1)\). Of the results obtained after that survey, we shall need the following rather deep proposition:
Sinai’s Theorem \((^2)\). If \(T\) is an ergodic automorphism of a Lebesgue space and \(S\) is a Bernoulli automorphism with finite entropy not exceeding the entropy of the automorphism \(T\), then \(S\) is a homomorphic image of \(T\).
Notation. \(M\) is a Lebesgue space; \(Z\) is the set of measurable partitions of the space \(M\) having finite entropy; \(H(\xi\mid\eta)\) is the mean conditional entropy of a measurable partition \(\xi\) relative to a measurable partition \(\eta\) (for the definition and properties of the function \(H(\xi\mid\eta)\), see \((^3)\)). If \(T\) is an automorphism of the space \(M\) and \(\xi\) is a measurable partition, then the partitions \(\xi_T\) and \(\xi_T^{-}\) are defined by the formulas:
\[ \xi_T=\prod_{k=-\infty}^{\infty} T^k\xi,\qquad \xi_T^{-}=\prod_{k=1}^{\infty} T^{-k}\xi . \]
The functions \(h(T,\xi)\) and \(h(T)\) are defined by the formulas:
\[ h(T,\xi)=H(\xi\mid \xi_T^{-}),\qquad h(T)=\sup h(T,\xi)\quad(\xi\in Z); \]
\(h(T)\) is the entropy of the automorphism.
A measurable partition \(\zeta\) is called invariant with respect to the automorphism \(T\) if \(T\zeta\ge \zeta\), and completely invariant with respect to \(T\) if \(T\zeta=\zeta\) (the relations \(=\) and \(\ge\) between measurable partitions are always understood modulo zero). The factor-automorphism induced by the automorphism \(T\) in the factor-space \(M/\zeta\) of the space \(M\) with respect to a completely invariant partition \(\zeta\) is denoted by \(T_\zeta\).
By \(D\) we denote the Bernoulli automorphism whose state space is the pair of points with measures \(1/2\).
Pinsker’s Lemma \((^4)\). If \(T\) is an automorphism of the space \(M\) and \(\xi\in Z,\ \eta\in Z\), then
\[ h(T,\xi\eta)=h(T,\xi)+H(\eta\mid \xi_T\eta_T^{-}). \]
A compact proof of this lemma (in a somewhat more general formulation) is given in \((^5)\).
2. Principal factor-automorphisms. The factor-automorphism \(T_\xi\) of the automorphism \(T\) is called principal if every invariant partition finer than \(\xi\) is completely invariant.
If \(T_\zeta\) is a principal factor-automorphism of the automorphism \(T\), then \(h(T_\zeta)=h(T)\).
Proof. Since \(h(T)\ge h(T_\zeta)\), it suffices to consider the case \(h(T_\zeta)<\infty\). We shall show that if \(l\) is an arbitrary number less than \(h(T)\), then \(h(T_\zeta)>l\).
Let \(\xi_1,\xi_2,\ldots\) be an increasing sequence of partitions from \(Z\) converging to \(\zeta\), and let \(\eta\) be a partition from \(Z\) such that \(h(T,\eta)>l\). By Pinsker’s lemma,
\[ h(T,\xi_n)+H\bigl(\eta\mid(\xi_n)_T\eta_T^{-}\bigr) = h(T,\xi_n\eta)\ge h(T,\eta)>l. \]
The first term on the left converges to \(h(T_\zeta)\), and it remains only to prove that the second term converges to zero. Since the sequence
\[ (\xi_1)_T\eta_T^{-},\quad (\xi_2)_T\eta_T^{-},\ldots \]
is increasing and converges to \(\zeta\eta_T^{-}\), we have
\[ \lim_{n\to\infty} H\bigl(\eta\mid(\xi_n)_T\eta_T^{-}\bigr) = H\bigl(\eta\mid \zeta\eta_T^{-}\bigr), \]
while the partition \(\zeta\eta_T^{-}\) is invariant and finer than \(\zeta\), and therefore completely invariant. Consequently, \(\eta\le \zeta\eta_T^{-}\) and \(H(\eta\mid\zeta\eta_T^{-})=0\).
If \(h(T)<\infty\) and \(h(T_\zeta)=h(T)\), then \(T_\zeta\) is the principal factor-automorphism of the automorphism \(T\).
Proof. Let \(\zeta'\) be an invariant measurable partition finer than \(\zeta\). We shall show that it is completely invariant.
If this is not so, then there exists a measurable set \(A\), composed of whole elements of the partition \(T\zeta'\), but differing by a set of positive measure from every measurable set composed of whole elements of the partition \(\zeta'\). Let \(\eta\) be the partition of the space \(M\) into the sets \(A\) and \(M-A\), and let \(\delta\) be a positive number. If \(\xi\) is a partition from \(Z\) such that \(\xi\le \zeta\) and \(h(T,\xi)>h(T)-\delta\), then \(\xi_T\le \zeta\), and, by Pinsker’s lemma,
\[ H(\eta\mid \xi\eta_T^{-}) \le H(\eta\mid \xi_T\eta_T^{-}) = h(T,\xi\eta)-h(T,\xi) \le h(T)-h(T,\xi)<\delta. \]
Consequently, \(H(\eta\mid \zeta\eta_T^{-})=0\) and \(\eta\le \zeta\eta_T^{-}\). But from the definition of \(\eta\) it is clear that \(\eta\le T\zeta'\). Hence \(\eta_T^{-}\le \zeta'\), and
\[ \eta\le \zeta\eta_T^{-}\le \zeta\zeta'=\zeta'. \]
Thus the set \(A\), contrary to its definition, consists, up to a set of measure zero, of whole elements of the partition \(\zeta'\).
3. Axiomatic definition of entropy. As the basic properties of the entropy of an automorphism we single out the following four properties:
\((\alpha)\) If \(S\) is a homomorphic image of the automorphism \(T\), then \(h(S)\le h(T)\).
\((\beta)\) \(h(S\times T)=h(S)+h(T)\).
\((\gamma)\) \(h(D)=1\).
\((\delta)\) If \(S\) is the principal factor-automorphism of the automorphism \(T\), then \(h(S)=h(T)\).
The first three properties are well known; the fourth was established in §2.
Main theorem. \(h\) is the unique nonnegative function of an ergodic automorphism of Lebesgue space possessing properties \((\alpha)\)–\((\delta)\).
By a function we mean a real-valued function which may also take infinite values.
Proof of the theorem. Let \(h'\) be an arbitrary nonnegative function of an ergodic automorphism of Lebesgue space possessing properties \((\alpha)\)–\((\delta)\). We shall show that \(h'=h\).
Denote by \(\mathfrak{B}\) the set of Bernoulli automorphisms with finite entropy. If \(S\in\mathfrak{B}\), \(T\in\mathfrak{B}\), and \(h(S)=h(T)\), then, by Sinai’s theorem (§1), each of the automorphisms \(S,T\) is a homomorphic image of the other, and therefore \(h'(S)=h'(T)\) (axiom \((\alpha)\)). Thus, on \(\mathfrak{B}\).
the function \(h'\) is determined by the function \(h\). Since the function \(h\) maps \(\mathfrak{B}\) onto the interval \(0 \leq t < \infty\), this means that on the interval \(0 \leq t < \infty\) there exists a (nonnegative) function \(\varphi\) satisfying the relation
\(h'(T)=\varphi(h(T))\), \(T \in \mathfrak{B}\).
From axiom \((\alpha)\) it follows that the function \(\varphi\) is increasing. Indeed, if \(0 \leq s < t < \infty\) and \(S,T\) are automorphisms from \(\mathfrak{B}\) such that \(h(S)=s\), \(h(T)=t\), then, by Sinai’s theorem, \(S\) is a homomorphic image of \(T\), and therefore
\[ \varphi(s)=\varphi(h(S))=h'(S)\leq h'(T)=\varphi(h(T))=\varphi(t). \]
From axiom \((\beta)\) it follows that the function \(\varphi\) is additive. Indeed, if \(0 \leq s < \infty\), \(0 \leq t < \infty\), and \(S,T\) are automorphisms from \(\mathfrak{B}\) such that \(h(S)=s\), \(h(T)=t\), then \(S \times T \in \mathfrak{B}\), and
\[ \begin{aligned} \varphi(s+t) &= \varphi(h(S)+h(T))=\varphi(h(S\times T))=h'(S\times T)\\ &=h'(S)+h'(T)=\varphi(h(S))+\varphi(h(T))=\varphi(s)+\varphi(t). \end{aligned} \]
From axiom \((\gamma)\) it follows that \(\varphi(1)=1\). Indeed, \(D\in\mathfrak{B}\), and therefore
\[ \varphi(1)=\varphi(h(D))=h'(D)=1. \]
Since the function \(\varphi\) is additive and monotone and \(\varphi(1)=1\), we have \(\varphi(t)=t\), \(0 \leq t < \infty\). Consequently, if \(T\in\mathfrak{B}\), then \(h'(T)=h(T)\).
Let now \(T\) be an arbitrary ergodic automorphism of a Lebesgue space with \(h(T)<\infty\). By Sinai’s theorem, \(T\) has a factor-automorphism \(S\in\mathfrak{B}\) such that \(h(S)=h(T)\), and by item 2 this factor-automorphism is principal. Consequently, by axiom \((\delta)\),
\[ h'(T)=h'(S)=h(S)=h(T). \]
Finally, if \(T\) is an ergodic automorphism of a Lebesgue space with \(h(T)=\infty\), then for every natural \(n\) there exists a factor-automorphism \(S\) of the automorphism \(T\) such that \(n>h(S)<\infty\). From the ergodicity of \(T\) follows the ergodicity of \(S\), and by axiom \((\alpha)\)
\[ h'(T)\geq h'(S)=h(S)<n. \]
Thus, if the automorphism \(T\) is ergodic and \(h(T)=\infty\), then also \(h'(T)=\infty\).
- Addenda. The nonnegativity of the function \(h\) can be derived from axioms \((\alpha)\), \((\beta)\), if one assumes that \(h(T)\ne -\infty\).
Axiom \((\alpha)\) is equivalent to the collection of two axioms: if the automorphisms \(S\) and \(T\) are isomorphic, then \(h(S)=h(T)\); if \(S\) is a factor-automorphism of the automorphism \(T\), then \(h(S)\leq h(T)\).
Axiom \((\beta)\) may be replaced by the axiom: \(h(T^n)=n h(T)\), \(n\leq 0\) (if \(h(T)=\infty\), and \(n=0\), then the right-hand side is considered equal to zero).
Axiom \((\delta)\) can probably be simplified. It is not excluded that it may be replaced by the axiom: if every partition invariant with respect to \(T\) is completely invariant, then \(h(T)=0\).
It is not difficult also to give an axiomatic definition of entropy suitable for all (not only ergodic) automorphisms of a Lebesgue space. For this it suffices to adjoin to axioms \((\alpha)\)—\((\delta)\) the following axiom:
\((\varepsilon)\) If the partition \(\xi\) decomposes the automorphism \(T\) into components \(T_C\), then \(h(T)\) is the mean value (the integral over \(M/\xi\)) of the function \(h(T_C)\).
Received
12 VII 1962
References
\(^{1}\) V. A. Rokhlin, Uspekhi Mat. Nauk, 15, No. 4 (1960).
\(^{2}\) Ya. G. Sinai, Dokl. Akad. Nauk SSSR, 147, No. 4 (1962).
\(^{3}\) A. N. Kolmogorov, Dokl. Akad. Nauk SSSR, 119, No. 5 (1960); 124, No. 4 (1959).
\(^{4}\) M. S. Pinsker, Information stability of random variables and processes, Publishing House of the Academy of Sciences of the USSR, 1960.
\(^{5}\) V. A. Rokhlin, Ya. G. Sinai, Dokl. Akad. Nauk SSSR, 141, No. 5 (1961).