MATHEMATICS

V. M. TIKHOMIROV

ON THE $\varepsilon$-ENTROPY OF CERTAIN CLASSES OF ANALYTIC FUNCTIONS

(Presented by Academician A. N. Kolmogorov, 17 V 1957)

Considering various classes $F$ of analytic functions $f(z)$, we shall be especially interested in their behavior on the segment of the real axis
$\Delta_T=\{z;\ -T\leq z\leq +T\}$ (always assuming that it lies in the domain of definition of $f\in F$) and, accordingly, shall use the metric
$\rho_T(f_1,f_2)=\max |f_1(z)-f_2(z)|,\quad z\in \Delta_T$.

As in the note of A. N. Kolmogorov (¹), we shall denote by $N_\varepsilon^T(F)$ the minimal number of elements of an $\varepsilon$-covering of $F$, i.e. a covering of $F$ by sets whose diameter in the metric $\rho_T$ is $\leq \varepsilon$. The binary logarithm
$\log N_\varepsilon^T(F)=H_\varepsilon^T(F)$ will be called the $\varepsilon$-entropy of the class $F$ on the segment $\Delta_T$. It seems to us essential, along with $N_\varepsilon^T(F)$, to consider the quantity $R_\varepsilon^T(F)$, equal to the maximal number of elements of an $\varepsilon$-distinguishable subset of $F$, i.e. a set contained in $F$ for any two elements $f_1$ and $f_2$ of which $\rho_T(f_1,f_2)>\varepsilon$.
$\log_2 R_\varepsilon^T(F)=C_\varepsilon^T(F)$ will be called the $\varepsilon$-capacity of the class $F$ on the segment $\Delta_T$.

It is easy to show that

\[ C_\varepsilon^T(F)\leq H_\varepsilon^T(F). \tag{1} \]

We shall consider the following classes of analytic functions:
$A_h^T(M)$—the class of functions analytic and bounded by the constant $M$ in the domain $G_h^T$ of points $z$ of the form $z=t+u,\ t\in\Delta_T,\ |u|\leq h$;

$F_{s,\sigma}^T(M)$—the class of entire functions satisfying, for every $t$ from the segment $\Delta_T$ and every $u$, the inequality $|f(t+u)|\leq Me^{\sigma|u|^s},\ s\geq 1$;

$B_\sigma(M)$—the class of entire functions satisfying the relation $|f(z)|\leq Me^{\sigma|\operatorname{Im}z|}$. The class $B_\sigma(M)$ would naturally be denoted by $F_{1\sigma}^{\infty}(M)$, since it coincides with the intersection of all classes $F_{1\sigma}^T(M)$ with finite $T$.

In formulating the results we shall use Bourbaki’s notation $\asymp$ and $\sim$ for strong and weak equivalence, explained in (¹). In some development of these, we shall say that $f(\varepsilon,T)$ and $g(\varepsilon,T)$ are weakly equivalent as $\varepsilon\to 0$ under one or another condition $S$ on $\varepsilon$ and $T$, uniformly in $T$, if there exist constants $0<a\leq A$ and $\varepsilon\leq\varepsilon_0$ such that
$a\leq f(\varepsilon,T)/g(\varepsilon,T)\leq A$ for $\varepsilon\leq\varepsilon_0$ and pairs $(\varepsilon,T)$ satisfying the condition $S$.

Theorem 1*.

\[ \frac{2\sigma}{\pi}\log\frac{1}{\varepsilon}\asymp \liminf_{T\to\infty}\frac{1}{2T}H_\varepsilon^T(B_\sigma(M)) \asymp \limsup_{T\to\infty}\frac{1}{2T}H_\varepsilon^T(B_\sigma(M)). \]

* Theorems 1–3 remain unchanged also for the functions $C_\varepsilon^T$.

Theorem 2. Uniformly in \(T \geqslant 0\),

\[ H_\varepsilon^T\left(A_h^T(M)\right)\asymp \left(\frac{\log(1/\varepsilon)}{\log(1/T+1)}+1\right)\log \frac1\varepsilon . \]

Theorem 3. For \(s \geqslant 1\), uniformly in \(T \geqslant 0\),

\[ H_\varepsilon^T\left(F_{s,\sigma}^T(M)\right)\asymp \left(\frac{\log(1/\varepsilon)} {\log\left(\dfrac{(\log(1/\varepsilon))^{1/\alpha}}{T}+1\right)}+1\right) \log \frac1\varepsilon . \]

The right-hand sides of the equivalences in Theorems 2 and 3 for \(T=0\) are taken to be equal to \(\log(1/\varepsilon)\).

We note that the class \(B_\sigma(M)\) in the metric \(\rho_T\) (for any \(T\)) is the closure of the class \(B'_\sigma(M)\) of finite sums

\[ f(z)=\sum_{k=-n}^{+n} c_k e^{i\lambda_k z}, \]

satisfying on the real axis the condition \(|f|\leqslant M\), and with frequencies \(\lambda_k\) from the interval \(-\sigma\leqslant \lambda\leqslant +\sigma\) ((\(^{2}\), pp. 160–164). Therefore Theorem 1 may be regarded as one of the justifications of the idea (apparently first advanced by V. A. Kotelnikov (\(^{3}\)) and widely accepted in information theory) that the amount of information contained in specifying, on an interval of length \(\tau\), a function with spectrum bounded by a frequency band of width \(2\sigma\), for large \(\tau\), is equivalent to the amount of information in specifying \(\dfrac{2\sigma}{\pi}\tau\) real numbers.

We also note some weak equivalences following from Theorems 2 and 3, valid uniformly in \(T\), under various restrictions:

\[ 2_{\mathrm a}.\qquad H_\varepsilon^T\left(A_h^T(M)\right)\asymp \log\frac1\varepsilon \left(1+\frac{\log\varepsilon}{\log T}\right) \quad \text{for } T\leqslant T_0 . \]

\[ 2_{\mathrm b}.\qquad H_\varepsilon^T\left(A_h^T(M)\right)\asymp \log^2\frac1\varepsilon \quad \text{for } 0<T_0\leqslant T\leqslant T_1 . \]

\[ 2_{\mathrm c}.\qquad H_\varepsilon^T\left(A_h^T(M)\right)\asymp T\left(\log\frac1\varepsilon\right)^2 \quad \text{for } 0<T_0\leqslant T . \]

\[ 3_{\mathrm a}.\qquad H_\varepsilon^T\left(F_{s\sigma}^T(M)\right)\asymp \log\frac1\varepsilon \left(1+\frac{\log\varepsilon}{\log T}\right) \quad \text{for } T\leqslant \frac{c}{(\log(1/\varepsilon))^\alpha},\quad c>0,\quad \alpha>\frac1s . \]

\[ 3_{\mathrm b}.\qquad H_\varepsilon^T\left(F_{s\sigma}^T(M)\right)\asymp \frac{(\log(1/\varepsilon))^2}{\log\log(1/\varepsilon)} \quad \text{for } \frac{c}{(\log(1/\varepsilon))^\alpha}\leqslant T\leqslant c'\left(\log\frac1\varepsilon\right)^\beta, \]

\[ \alpha>0,\quad \beta<\frac1s,\quad c,\ c'>0 . \]

\[ 3_{\mathrm c}.\qquad H_\varepsilon^T\left(F_{s\sigma}^T(M)\right)\asymp T\log^{\,2-1/s}\left(\frac1\varepsilon\right) \quad \text{for } T\geqslant c_1\left(\log\frac1\varepsilon\right)^\beta,\quad \beta\geqslant \frac1s,\quad c_1>0 . \]

Relations \(2_{\mathrm b}\) and \(3_{\mathrm b}\) are valid, in particular, for constant \(T>0\) (for constant \(T\), the relations \(2_{\mathrm b}\) are indicated in (\(^{1}\))); their comparison shows that for constant \(T\) the passage from the class \(A_h\) to the seemingly much narrower class \(F_{s\sigma}\) has little effect on the order of growth of \(H_\varepsilon\).

From \(2_{\mathrm c}\) and \(3_{\mathrm c}\) follow the relations

\[ 2.\qquad \liminf_{T\to\infty}\frac1{2T}H_\varepsilon^T\left(A_h^T(M)\right) \asymp \limsup_{T\to\infty}\frac1{2T}H_\varepsilon^T\left(A_h^T(M)\right) \asymp \left(\log\frac1\varepsilon\right)^2 . \]

\[ 3.\qquad \liminf_{T\to\infty}\frac1{2T}H_\varepsilon^T\left(F_{s,\sigma}^T(M)\right) \asymp \limsup_{T\to\infty}\frac1{2T}H_\varepsilon^T\left(F_{s\sigma}^T(M)\right) \asymp \left(\log\frac1\varepsilon\right)^{2-1/s}. \]

We note that 2 and 3 are preserved when passing to the classes \(A_h^\infty\) and \(F_{s\sigma}^{\infty}\). It is natural to compare them with Theorem 1.

Proof of Theorem 1. a) Upper estimate for the quantity \(H_\varepsilon(B_\sigma^T(M))\). It is known (see, for example, (4), p. 269) that for every function \(g(z)\in B_{\sigma'}(M)\), \(\sigma'<\pi\), the following interpolation formula holds:

\[ g(z)=\frac{\sin \pi z}{\pi\omega}\sum_{k=-\infty}^{+\infty} (-1)^k\frac{g(k)\sin \omega(k-z)}{(k-z)^2}, \tag{2} \]

where \(0<\omega<\pi-\sigma'\).

Obviously, if \(f(z)\in B_\sigma(M)\), then \(g(z)=f\!\left(\frac{\sigma'}{\sigma}z\right)\in B_{\sigma'}(M)\), whence from (2) we obtain, for \(f(z)\in B_\sigma(M)\),

\[ f(z)= \frac{\sin \frac{\pi\sigma}{\sigma'}z}{\pi\omega} \sum_{k=-\infty}^{+\infty} (-1)^k \frac{ f\!\left(k\frac{\sigma'}{\sigma}\right) \sin \frac{\omega\sigma}{\sigma'}\left(k\frac{\sigma'}{\sigma}-z\right) }{ \left(\frac{\sigma}{\sigma'}\right)^2 \left(k\frac{\sigma'}{\sigma}-z\right)^2 }, \qquad \sigma'<\pi,\quad 0<\omega<\pi-\sigma'. \]

Fix \(\omega\). Choose \(n=n(\varepsilon,\omega)\) so that
\(\sum_{|s|>n}\frac{1}{s^2}<\frac{\pi}{M}\varepsilon\omega\). Then for \(t\in \Delta_T\)*

\[ \left| \frac{\sin \frac{\pi\sigma}{\sigma'}t}{\pi\omega} \sum_{|k|>n+[T\sigma/\sigma']+1} \frac{ f\!\left(k\frac{\sigma'}{\sigma}\right) \sin \frac{\omega\sigma}{\sigma'}\left(k\frac{\sigma'}{\sigma}-t\right) }{ \left(\frac{\sigma}{\sigma'}\right)^2 \left(k\frac{\sigma'}{\sigma}-t\right)^2 } \right| \le \frac{M}{\pi\omega}\sum_{|s|>n}\frac{1}{s^2}<\varepsilon . \tag{3} \]

We approximate \(f(z)\) by a function \(\widetilde f(z)\) of the form

\[ \widetilde f(z)= \frac{\sin \frac{\pi\sigma}{\sigma'}z}{\pi\omega}\, \varepsilon \sum_{|k|<n+[T\sigma/\sigma']+1} (-1)^k \frac{ \left( \left[\frac{\operatorname{Re} f\!\left(k\frac{\sigma'}{\sigma}\right)}{\varepsilon}\right] +i\left[\frac{\operatorname{Im} f\!\left(k\frac{\sigma'}{\sigma}\right)}{\varepsilon}\right] \right) \sin \frac{\omega\sigma}{\sigma'}\left(k\frac{\sigma'}{\sigma}-z\right) }{ \left(\frac{\sigma}{\sigma'}\right)^2 \left(k\frac{\sigma'}{\sigma}-z\right)^2 }. \]

It is not hard to show that

\[ \rho_T(f,\widetilde f)\equiv \max_{t\in \Delta_T}|f(t)-\widetilde f(t)|\le C(\omega)\varepsilon, \]

where \(C(\omega)\) is a constant depending only on \(\omega\).

For each constructed function \(\widetilde f\), consider the set \(\{U_{\widetilde f}\}\) of functions \(f(z)\) such that \(\rho_T(f,\widetilde f)\le C(\omega)\varepsilon\). It is easy to see that \(\{U_{\widetilde f}\}\) forms a \(2C(\omega)\varepsilon\)-covering of the space \(B_\sigma^T(M)\). The number \(N_{\varepsilon T}\) of sets in this covering satisfies the inequality

\[ \overline{N}_{\varepsilon T} \le \bigl(2[M/\varepsilon]+1\bigr)^{2(2[T\sigma/\sigma']+2n+3)}, \]

whence

\[ H_{2C(\omega)\varepsilon}^T\bigl(B_\sigma^T(M)\bigr) \le (4[T\sigma/\sigma']+n')\log(2[M/\varepsilon]+1), \]

\[ \text{where} \]

\[ *[A]\text{ is the integer part of }A. \]

where \(n'=4n+6\), whence

\[ H_\varepsilon^T(B_\sigma(M))\ll \bigl(4[T\sigma'/\sigma']+n'\bigr)\log(2[2MC(\omega)/\varepsilon]+1). \tag{4} \]

From (4) it follows immediately that

\[ \lim_{\varepsilon\to0}\,\overline{\lim_{T\to\infty}}\, \frac{H_\varepsilon^T(B_\sigma(M))}{2T\log(1/\varepsilon)} \ll \frac{2\sigma}{\sigma'}\ll \frac{2\sigma}{\pi}, \]

since \(\sigma'\) is an arbitrary number less than \(\pi\).

b) A lower estimate for the quantity \(C_\varepsilon^T(B_\sigma(M))\). Let \(a>0\), \(\sigma'=\sigma-a\), \(n=[T\sigma'/\pi]\), and let
\(K=\{(k_{-n},k'_{-n}),\ldots,(k_0,k'_0),\ldots,(k_n,k'_n)\}\) be a set of integers, each of which does not exceed \([M'/\varepsilon]\) (we shall take care of \(M'\) later).

Consider the set of functions \(\{f_K(z)\}\), where

\[ f_K(z)=\varepsilon \sum_{|j|<n} \frac{(k_j+ik'_j)\sin\sigma'(z-j\pi/\sigma')\sin a(z-j\pi/\sigma')} {a\sigma'(z-j\pi/\sigma')^2}. \]

It is easy to prove that all \(f_K(t)\) are bounded by one and the same constant \(M'C(\sigma')\), independent of \(n\). Put \(M'=M/C(\sigma')\). Moreover, \(f_K(z)\) is of exponential growth with exponent \(\sigma'+a=\sigma\); consequently ((\(^2\)), p. 151), they all belong to \(B_\sigma(M)\).

The set of functions \(f_K(z)\) is \(\varepsilon\)-distinguishable. The number \(N_{\varepsilon T}\) of these functions is equal to

\[ (2[M/C\sqrt{2}\varepsilon]+1)^{2(2[T\sigma'/\pi]+1)}, \]

whence

\[ C_\varepsilon^T(B_\sigma(M))\gg \log N_{\varepsilon T} =(4[(\sigma-a)T/\pi]+2)\log(2[M/C\sqrt{2}\varepsilon]+1). \tag{5} \]

From (5) one may obtain that

\[ \lim_{\varepsilon\to0}\,\lim_{T\to\infty} \frac{C_\varepsilon^T(B_\sigma(M))}{2T\log(1/\varepsilon)} \gg \frac{2(\sigma-a)}{\pi}\gg \frac{2\sigma}{\pi}, \]

since \(a\) is arbitrary.

Using (1), we finally obtain

\[ \liminf_{T\to\infty}\frac{H_\varepsilon^T(B_\sigma(M))}{2T} \sim \limsup_{T\to\infty}\frac{H_\varepsilon^T(B_\sigma(M))}{2T} \sim \frac{2\sigma}{\pi}\log\frac{1}{\varepsilon}. \]

The scheme of proof of Theorems 2 and 3 is the same as that of Theorem 1. The upper estimates are obtained by approximating the coefficients of Taylor series; the lower estimates for small \(T\) (in Theorem 2 for \(T\ll \lambda h\), in Theorem 3 for \(T\ll \lambda(\log(1/\varepsilon))^{1/s}\), where \(\lambda\) is a certain number) are obtained by considering interpolation polynomials, and for large \(T\) by trigonometric interpolation.

In conclusion I consider it my pleasant duty to express gratitude to A. N. Kolmogorov, who posed the problems considered here and devoted much attention to the work.

Moscow State University
named after M. V. Lomonosov

Received
17 V 1957

CITED LITERATURE

\(^1\) A. N. Kolmogorov, DAN, 108, No. 3 (1956).
\(^2\) N. I. Akhiezer, Lectures on Approximation Theory, 1947.
\(^3\) V. A. Kotelnikov, Materials for the 1st All-Union Congress on the Problem of Communication Reconstruction.
\(^4\) B. Ya. Levin, Distribution of Zeros of Entire Functions, 1956.