Abstract
Full Text
Reports of the Academy of Sciences of the USSR
- Volume 117, No. 5
MATHEMATICS
A. G. VITUSHKIN
ABSOLUTE (\varepsilon)-ENTROPY OF METRIC SPACES
(Presented by Academician A. N. Kolmogorov, 19 X 1957)
This note is devoted to improving A. N. Kolmogorov’s definitions of the (\varepsilon)-entropy of a metric space ((^{1})) and to estimating the (\varepsilon)-entropy for certain classes of functions.
Definition 1. Let (F) be a compact metric space, and let (\Phi) be its metric extension, i.e., a space that contains (F) as its subset and has on it a metric identical with that of (F). Denote by (N_{\varepsilon}^{\Phi}(F)) the number of elements of a minimal (in the sense of the number of elements) subset of (\Phi) approximating (F) with accuracy up to (\varepsilon).
The number (H_{\varepsilon}^{\Phi}(F)=\log N_{\varepsilon}^{\Phi}(F)) is called the (\varepsilon)-entropy of the space (F) relative to (\Phi), or simply the relative (\varepsilon)-entropy of the space (F)*.
Definition 2. Denote by (N_{\varepsilon}(F)) the number of sets participating in the most economical covering of the space (\Phi) by its subsets of diameter not exceeding (2\varepsilon). The number (H_{\varepsilon}(F)=\log N_{\varepsilon}(F)) is called the absolute (\varepsilon)-entropy of the space (F).
The name “absolute (\varepsilon)-entropy” is justified by the following fact.
Theorem 1. For every compact metric space (F) and every (\varepsilon>0), the quantity (H_{\varepsilon}(F)) coincides with the lower bound of the quantity (H_{\varepsilon}^{\Phi}(F)) over all possible metric extensions (\Phi) of the space (F), i.e.
[
H_{\varepsilon}(F)=\inf_{\Phi \supset F} H_{\varepsilon}^{\Phi}(F).
]
It turns out that the lower bound mentioned in Theorem 1 is attained, and the best metric extension in this sense for every compact space (F) is the space (C) of functions continuous on the interval ([0 \leq x \leq 1]). This is easily obtained from the Banach–Mazur theorem on the possibility of an isometric embedding in (C) of every separable metric space and from the following lemma of V. D. Erokhin:
Lemma 1. For every compact subset of the space (C) of diameter (2r), one can indicate an element of the same space (C) that is at distance no more than (r) from every point of the subset under consideration.
In terms of the definitions given above, it is possible to indicate not only the order for the lower bound of the volume of tables, but also the exact value of the lower bound of this quantity.
Theorem 2. Let (f) be an element of some metric space (F), and let (T(f)) be a table of this element, reconstructing some element (\varphi) from the metric extension (\Phi) of the space (F), at distance from (f) no more than (\varepsilon). Then
[
P[T(f)] \geq H_{\varepsilon}^{\Phi}(F)
]
* Everywhere below the logarithm will be understood to have base (2). In A. N. Kolmogorov’s notation, our (H_{\varepsilon}^{F}(F)) is (J_{F}^{a}(\varepsilon)), and our (H_{\varepsilon}(F)) is (J_{F}^{c}(2\varepsilon)).
and there exists a method of compiling the table (T(f)) such that
[
P[T(f)] \leqslant H_\varepsilon^\Phi(F)+1,
]
where (P[T(f)]) is the volume of the table (T(f)), i.e. the total number of binary digits required to record all the parameters of the table.
Theorem 3. The volume of any table (see Theorem 2) that reconstructs (f\in F) with accuracy up to (\varepsilon) must be no less than (H_\varepsilon(F)). Moreover, the indicated estimate is attainable.
Thus, estimating the volume of the table reduces to computing the absolute entropy of the corresponding space.
Notation. (E_n^z) is the space of (n) complex variables ((z_1,z_2,\ldots,z_n)) ((z_n=x_k+i y_k)); (E_n^x) is the space of variables ((x_1,x_2,\ldots,x_n)); (\mathfrak{D}{\rho_k}^{a_k b_k}) is the domain of the plane (z_k) bounded by an ellipse with semiminor axis (\rho_k) and with foci at the points (x_k=a_k,\ x_k=b_k); (\mathfrak{D}}^{a,b}=\mathfrak{D{\rho_1}^{a_1,b_1}\times\cdots\times\mathfrak{D}) is the space of entire functions of (n) complex variables of order (\sigma=(\sigma_1,\sigma_2,\ldots,\sigma_n)) and type (s=(s_1,s_2,\ldots,s_n)) ((\sigma_k>0,\ s_k>0)), i.e. such functions whose growth is bounded by the inequality}^{a_n,b_n}); (I_n^{a,b}) is the parallelepiped ((a_k\leqslant x_k\leqslant b_k)) ((k=1,2,\ldots,n)); (P_d) is the strip ((-d_k\leqslant y_k\leqslant d_k)); (P_d=P_{d_1}\times P_{d_2}\times\cdots\times P_{d_n}); (B_{\rho_k}^{r_k}) is the annulus ((r_k\leqslant |z_k|\leqslant \rho_k)); (B_\rho^r=B_{\rho_1}^{r_1}\times B_{\rho_2}^{r_2}\times\cdots\times B_{\rho_n}^{r_n}); (F_{\rho,c}^{r}) is the space of all functions analytic in (B_\rho^r), whose modulus in (B_\rho^r) is bounded by the constant (c); (F_{d,c,2\pi}) is the space of real (2\pi)-periodic functions analytic on (E_n^x), whose analytic continuations to (P_d) are bounded by the constant (c); (F_{\rho,c}^{a,b}) is the space of real functions analytic on (I_n^{a,b}), whose continuations to (\mathfrak{D}_{\rho}^{a,b}) are bounded by the constant (c); (F_s^{\sigma,c
[
|f(z_1,z_2,\ldots,z_n)|\leqslant c\prod_{k=1}^{n}\exp(\sigma_k|z_k|^{s_k}).
]
As the norm in these spaces one takes the maximum of the modulus of the function on the sets (B_{\rho'}^{r'},\ E_n^x,\ I_n^{a,b}), and (B_\rho^0), respectively. The following relations are valid*:
[
(H_\varepsilon(F_{\rho,c}^{r})=
\frac{2}{(n+1)!}\prod_{k=1}^{n}
\left(
\frac{1}{\log \dfrac{\rho_k''}{\rho_k'}}
+
\frac{1}{\log \dfrac{r_k'}{r_k''}}
\right)
\left(\log \frac{c}{\varepsilon}\right)^{n+1}
+
]
[
+\,O\left[\left(\log\frac{1}{\varepsilon}\right)^n
\log\log\frac{1}{\varepsilon}\right];
\tag{1}
]
[
H_\varepsilon(F_{d,c,2\pi})=
\frac{2^n}{(n+1)!(\log e)^n}
\prod_{k=1}^{n}\frac{1}{d_k}
\left(\log\frac{c}{\varepsilon}\right)^{n+1}
+
O\left[\left(\log\frac{1}{\varepsilon}\right)^n
\log\log\frac{1}{\varepsilon}\right];
\tag{2}
]
* Relations (1)—(3) are refinements of estimate II from the note by A. N. Kolmogorov ((^1)), and relation (4), in the case (n=1), is a refinement of the estimate
[
H_\varepsilon(F_s^{\sigma,c})\succ
\frac{\left(\log \dfrac{1}{\varepsilon}\right)^2}
{\log\log \dfrac{1}{\varepsilon}}
]
from the note by V. M. Tikhomirov ((^2)).
[
H_{\varepsilon}!\left(F_{\rho,c}^{a,b}\right)
=
\frac{1}{(n+1)!}
\prod_{k=1}^{n}
\frac{1}{\log!\left(\frac{2}{b_k-a_k}\rho_k\right)}
\left(\log\frac{c}{\varepsilon}\right)^{n+1}
+
]
[
+
O!\left[\left(\log\frac{1}{\varepsilon}\right)^{n}\log\log\frac{1}{\varepsilon}\right];
\tag{3}
]
[
H_{\varepsilon}!\left(F_{s}^{\sigma,c}\right)
=
\frac{2}{(n+1)!}
\prod_{k=1}^{n}s_k\,
\frac{\left(\log\frac{c}{\varepsilon}\right)^{n+1}}
{\left(\log\log\frac{c}{\varepsilon}\right)^{n}}
+
O!\left[
\frac{\left(\log\frac{1}{\varepsilon}\right)^{n+1}}
{\left(\log\log\frac{1}{\varepsilon}\right)^{n}}
\right].
\tag{4}
]
These estimates are easily obtained by counting the total number of binary digits needed for a sufficiently accurate storage of the coefficients of the polynomial that best approximates a function of one or another family.
Received
1 VIII 1957
CITED LITERATURE
¹ A. N. Kolmogorov, DAN, 108, No. 3 (1956). ² V. M. Tikhomirov, DAN, 117, No. 2 (1957).