CYBERNETICS AND CONTROL THEORY

O. B. LUPANOV

ON THE PRINCIPLE OF LOCAL CODING AND THE REALIZATION OF FUNCTIONS FROM CERTAIN CLASSES BY CIRCUITS OF FUNCTIONAL ELEMENTS

(Presented by Academician P. S. Novikov on 6 V 1961)

$1^\circ$. As is known, almost all functions and systems of functions of the algebra of logic admit only a very complicated circuit realization and therefore are in fact inaccessible. It is therefore important to single out classes of functions that can be realized more simply than the majority of functions, and to find methods for synthesizing circuits for them. The first results in this direction are contained in Shannon’s fundamental works ($^1$, $^2$). Subsequently a number of other classes were singled out, and more economical synthesis methods were also found for the classes introduced earlier ($^{3-9}$). S. V. Yablonskii constructed and studied a continuous family of classes of functions invariant with respect to substitutions of constants and renaming of arguments ($^{10}$, $^{11}$).

In the present paper we describe one general approach to the synthesis of circuits (the principle of local coding), which makes it possible, for very many classes of functions, to obtain substantially simpler circuits than in the general case. The idea of using an intermediate parameter, which underlies this principle, belongs to A. Sh. Blokh and was applied by him in the synthesis of contact circuits ($^{12}$). This principle is especially convenient in the synthesis of control systems ($^{13}$) of the “formulas with memory” type (circuits of functional elements ($^{14}$, $^{15}$)) or stronger ones (for example, automata). For a number of classes of functions, application of the principle of local coding makes it possible to obtain asymptotics for Shannon’s function.

$2^\circ$. Let $B_n$ be the set of all tuples $(a_0,\ldots,a_{n-1})$ of zeros and ones. A mapping of $B_n$ into $B_m$ will be called an $(n,m)$-operator. An $(n,1)$-operator may be regarded as a function of the algebra of logic in the arguments $x_1,\ldots,x_n$. We shall consider realization of $(n,m)$-operators by circuits of functional elements in an arbitrary finite basis ($^{14}$). To the elements of the basis are assigned weights—nonnegative numbers. The weight $P$ of each element $E$ realizing a function essentially depending on $k$ arguments, where $k>1$, is positive; the number

\[ \frac{P}{k-1} \]

is called the reduced weight of the element $E$. Let $\rho$ be the least of the reduced weights of the basis elements. The complexity of a circuit is defined as the sum of the weights of all elements entering into it. We introduce Shannon’s function in the usual way. Let $F$ be an operator; denote by $L(F)$ the least of the complexities of circuits realizing $F$. Let $\mathfrak{F}$ be some finite set of operators; by $L(\mathfrak{F})$ we shall denote

\[ \max_{F \in \mathfrak{F}} L(F). \]

In a number of cases, the $(n,m)$-operators of a certain class can be encoded by sequences of zeros and ones in such a way that the operator is determined uniquely by the code and, moreover, for each tuple $\tilde{\sigma}$ from $B_n$ and some piece of the code of not very great length, the value of the operator on the tuple $\tilde{\sigma}$ is determined (this is precisely the locality of the coding;

the indicated piece of code plays the role of an intermediate parameter in the process of computing the value of the operator). For example, for a symmetric function \(f(x_1,\ldots,x_n)\) the code may be the tuple \((\pi_0,\pi_1,\ldots,\pi_n)\), where \(\pi_i\) is the value of the function \(f\) on any tuple containing \(i\) ones and \(n-i\) zeros. In this example the piece of code consists of one digit, although in the general case these pieces may have a greater, even variable, length. The circuit for the operator consists of several parts. One part, from the tuple \(\tilde{\sigma}\), gives the “coordinates” of the piece of code (the circuit for \(A_F^{(1)}\) and \(A_F^{(2)}\)—see below), another, from the coordinates of the piece, gives the piece itself (this part usually contains most of the elements of the circuit—below, the circuit for \(A_F\)), and the third part, from the piece of code and the tuple \(\tilde{\sigma}\), “simply” computes the value of the operator on the tuple \(\tilde{\sigma}\) (below, the circuit for \(A_F^{(3)}\)).

For a more precise formulation of the principle of local coding, we introduce a number of notations. By \(|\tilde{\sigma}|\), where \(\tilde{\sigma}=(\sigma_0,\ldots,\sigma_{k-1})\), we denote the number
\(\sum_{i=0}^{k-1}\sigma_i2^i\), and by \(\tilde{\sigma}(k,q)\), where \(0\leq q<2^k\), the tuple \(\tilde{\sigma}=(\sigma_0,\ldots,\sigma_{k-1})\) such that \(|\tilde{\sigma}|=q\). Let \(\tilde{\pi}=(\pi_0,\ldots,\pi_{R-1})\) be some sequence; the sequence \((\pi_s,\pi_{s+1},\ldots,\pi_{s+t})\) (here \(\pi_i=0\) for \(i\geq R\)) will be denoted by the symbol \(S_{s,t}(\tilde{\pi})\), or \(S(\tilde{\pi},\tilde{\sigma}(r,s),\tilde{\sigma}(r,t))\), where* \(r=\lceil \log R\rceil\). Denote by \(B_{h,m}\bigl((h+1)\lceil\log h\rceil+m,2^m\bigr)\)-operator (here \(m\leq\lceil\log h\rceil\)), transforming the tuple \((\tilde{\alpha},\tilde{\beta},\tilde{\gamma})\), \(\tilde{\alpha}=(\alpha_0,\ldots,\alpha_{h-1})\), \(\tilde{\beta}=(\beta_0,\ldots,\beta_{\lceil\log h\rceil-1})\), \(\tilde{\gamma}=(\gamma_0,\ldots,\gamma_{m-1})\), into the tuple \((\delta_0,\ldots,\delta_{2^m-1})\) such that
\((\delta_0,\ldots,\delta_{|\tilde{\gamma}|})=S(\tilde{\alpha},\tilde{\beta},\tilde{\gamma})\) and \(\delta_i=0\) for \(i>|\tilde{\gamma}|\).

Principle of local coding. Let \(\mathfrak{F}=\{\mathfrak{F}_1,\ldots,\mathfrak{F}_n,\ldots\}\) be a sequence of sets of operators, where \(\mathfrak{F}_n\) contains only \((n,m_n)\)-operators. Suppose that to each operator \(F\) from \(\mathfrak{F}_n\) one can assign a sequence \(\tilde{\pi}^{(F)}=(\pi_0^{(F)},\pi_1^{(F)},\ldots,\pi_{h_n-1}^{(F)})\) of length \(h_n\), consisting of zeros and ones, and operators: an \((n,m_{1,n})\)-operator \(A_F^{(1)}\), \(m_{1,n}=\lceil\log h_n\rceil\); an \((n,m_{2,n})\)-operator \(A_F^{(2)}\); and an \((n+2^{m_{2,n}},m_n)\)-operator \(A_F^{(3)}\), such that

\[ F(\tilde{x})=A_F^{(3)}\bigl(\tilde{x},B_{h_n,m_{2,n}}(\tilde{\pi}^{(F)},A_F^{(1)}(\tilde{x}),A_F^{(2)}(\tilde{x}))\bigr). \tag{1} \]

Define the \((m_{1,n}-l_n,2^{l_n}+2^{m_{2,n}})\)-operator \(A_F\) as follows (here \(l_n\) is some number):

\[ A_F\bigl(\sigma(m_{1,n}-l_n,k)\bigr) = S_{k2^{l_n},\,2^{l_n}+2^{m_{2,n}}}\bigl(\tilde{\pi}^{(F)}\bigr). \]

Consider the operators

\[ A_F^{(11)}(\tilde{x})=S_{0,l_n}\bigl(A_F^{(1)}(\tilde{x})\bigr), \qquad A_F^{(12)}(\tilde{x})=S_{l_n,m_{1,n}-l_n}\bigl(A_F^{(1)}(\tilde{x})\bigr). \]

Obviously, a circuit realizing the operator \(A_F^{(1)}\) simultaneously realizes the operators \(A_F^{(11)}\) and \(A_F^{(12)}\). Then

\[ \begin{aligned} B_{h_n,m_{2,n}}\bigl(\tilde{\pi}^{(F)},A_F^{(1)}(\tilde{x}),A_F^{(2)}(\tilde{x})\bigr) &= B_{2^{l_n}+2^{m_{2,n}},\,m_{2,n}}^{\,l_n} \bigl(A_F(A_F^{(12)}(\tilde{x})),\\ &\qquad A_F^{(11)}(\tilde{x}),A_F^{(2)}(\tilde{x})\bigr). \end{aligned} \tag{2} \]

The synthesis of the circuit for \(F(\tilde{x})\) is carried out in accordance with (1)—(2).

Introduce the notation: \(H(\mathfrak{F}_n)\) is the logarithm to base 2 of the number of operators in \(\mathfrak{F}_n\); \(\lambda(n)=H(\mathfrak{F}_n)/\log H(\mathfrak{F}_n)\).

* Here and everywhere below, the symbol \(\log\) denotes the logarithm to base 2; the symbol \(\lceil a\rceil\) denotes the least integer not less than \(a\).

Theorem 1. Suppose

\[ \frac{n+m_n}{\lambda(n)} \to 0,\qquad h_n \sim H(\mathfrak F_n),\qquad 2^{m_{2,n}}=o(\lambda(n)), \]

\[ L\left(A_F^{(i)}\right)=o(\lambda(n)),\qquad i=1,2,3. \]

Then

\[ L(\mathfrak F_n)\sim \rho\,\frac{H(\mathfrak F_n)}{\log H(\mathfrak F_n)}. \]

An important special case of the principle of local coding is the so-called principle of parametrization: when \(m_{2,n}=0\) and each operator \(F\) from \(\mathfrak F_n\) can be represented in the form

\[ F(\tilde x)=A'_F\bigl(\tilde x,\tilde \pi_{A''_F}^{(F)}(\tilde x)\bigr). \]

3°. Let us give some results obtained by means of the principle of local coding.

A. Functions with a given number of ones. The number of sets from \(B_n\) on which the function \(f(x_1,\ldots,x_n)\) assumes the value 1 will be denoted by \(\|f\|\). Let \(\mathfrak R_{n,k}\) be the class of functions \(f(x_1,\ldots,x_n)\) such that \(*\ \|f\|=k\). Introduce the notation \(l(n,k)=\min(k,2^n-k)\).

Theorem 2. Suppose the sequence \(\{k(n)\}\) is such that

\[ \frac{l(n,k(n))}{\log n}\to\infty. \]

Then

\[ L(\mathfrak R_{n,k(n)})\sim \rho\,\frac{\log C_{2^n}^{k(n)}}{\log\log C_{2^n}^{k(n)}}. \]

Let \(\mathfrak R_n\) be the class of all functions \(f(x_1,\ldots,x_n)\) of the algebra of logic, and let \(L(n)=L(\mathfrak R_n)\). Let \(M_n\) be the class of all functions \(f(x_1,\ldots,x_n)\) for which \(L(f)=L(n)\). In the work of S. V. Yablonskii \((^{13})\) it is shown that the construction of functions from \(M_n\) in a certain natural class of algorithms is connected with an exhaustive search through all functions of the algebra of logic. It is of interest to study the properties of these “most difficult” functions. Let

\[ \Delta(n)=\max_{f\in M_n}\bigl|\|f\|-2^{n-1}\bigr|. \]

Theorem 3.

\[ \Delta(n)<C\cdot 2^n\left(\frac{\log n}{n}\right)^{1/2}, \]

where \(C\) is some constant.

This inequality follows from the estimate of the remainder term in the asymptotic expression for \(L(\mathfrak R_{n,k(n)})\) and from the fact that

\[ L(n)-\rho\,\frac{2^n}{n}\preccurlyeq \frac{2^n\log n}{n^2}. \]

Corollary. Let \(f_1,\ldots,f_n,\ldots\) be an arbitrary sequence such that \(f_n\in M_n\) \((n=1,2,\ldots)\). Then

\[ \|f_n\|\sim 2^{n-1}. \]

B. Monotone operators. An \((n;m)\)-operator \(F\) is called monotone \((^8)\) if from the fact that \(|\tilde \alpha|\le |\tilde \beta|\) it follows that \(|F(\tilde \alpha)|\le |F(\tilde \beta)|\). Let \(\mathfrak M_n\) be the class of all monotone \((n,n)\)-operators.

\[ \text{* Earlier, B. I. Finikov studied the realization of functions from }\mathfrak R_{n,k}\text{ in the case of small }k \]
by parallel-sequential contact circuits \((^7)\).

Theorem 4*.

\[ L(\mathfrak{M}_n)\sim \rho\,\frac{2^{n+1}}{n}. \]

B. Symmetric operators. Let \(\mathfrak{S}_{n,m}\) be the class of all systems
\[ (f_1(x_1,\ldots,x_n),\ldots,f_m(x_1,\ldots,x_n)) \]
of symmetric functions of the algebra of logic in \(n\) arguments.

Theorem 5.

\[ L(\mathfrak{S}_{n,1})\lesssim n. \]

Theorem 6. If

\[ \frac{m(n)}{\log n}\to\infty, \]

then

\[ L(\mathfrak{S}_{n,m(n)})\sim \rho\,\frac{nm(n)}{\log(nm(n))}. \]

V. A. Steklov Mathematical Institute
Academy of Sciences of the USSR

Received
27 IV 1961

CITED LITERATURE

1. C. E. Shannon, Trans. AIEE, 57, 713 (1938).
2. C. E. Shannon, Bell Syst. Tech. J., 28, No. 1, 59 (1949).
3. M. A. Gavrilov, Theory of relay-contact circuits, 1950.
4. S. V. Yablonskii, DAN, 95, No. 4, 805 (1954).
5. G. N. Povarov, DAN, 94, No. 6, 1075 (1954).
6. V. K. Korobkov, DAN, 109, No. 2, 260 (1956).
7. B. I. Finikov, DAN, 115, No. 2, 247 (1957).
8. B. A. Trakhtenbrot, DAN, 127, No. 2, 281 (1959).
9. V. I. Reznik, DAN, 139, No. 3 (1961).
10. S. V. Yablonskii, UMN, 12, issue 6, 189 (1957).
11. S. V. Yablonskii, DAN, 124, No. 1, 44 (1959).
12. A. Sh. Blokh, DAN, 111, No. 5, 1017 (1956).
13. S. V. Yablonskii, Problems of Cybernetics, issue 2, 7 (1959).
14. B. Lupanov, Izv. higher educational institutions, Radiophysics, 1, No. 1, 120 (1958).
15. O. B. Lupanov, DAN, 128, No. 3, 464 (1959).

* This theorem is a strengthening of Theorem 1 from (8).