CYBERNETICS AND CONTROL THEORY

A. A. MUCHNIK and S. G. GINDIKIN

ON THE COMPLETENESS OF A SYSTEM OF UNRELIABLE ELEMENTS REALIZING FUNCTIONS OF THE ALGEBRA OF LOGIC

(Presented by Academician P. S. Novikov on 23 I 1962)

In the synthesis of circuits realizing functions of the algebra of logic, alongside other questions, questions of completeness and reliability arise. The first of these was investigated in detail by E. Post (¹). The problem of reliability is the subject of works by J. von Neumann (²), E. F. Moore and C. E. Shannon (³), and a number of other authors. The present note is devoted to the study of the question of the completeness of a system of elements that unreliably realize functions of the algebra of logic; namely, we determine the conditions under which every function of the algebra of logic can be realized with arbitrarily high reliability by a circuit composed of such elements.

Let us proceed to the precise formulation of the problem. Suppose there is a system of functions of the algebra of logic consisting of two disjoint parts \(A\) and \(B\):

\[ A=\{g_1,g_2,\ldots,g_r,\ldots\},\qquad B=\{f_1,f_2,\ldots,f_s,\ldots\}. \]

Let the functions from the system \(A\) be realized by completely reliable (i.e., error-free) functional elements (f. e.), while each function \(f_s\in B\) is realized by functional elements \(T_{f_s}\) with upper bounds on the probability of error for them equal to \(\varepsilon_s\). We shall assume that \(\varepsilon_s<1/2\). Put \(\chi_0=\inf\{\varepsilon_s\}\), \(B(\nu)=\{f_s:\varepsilon_s<\nu\}\) for every \(0<\nu<1/2\), and

\[ \chi(B')=\inf_{f_s\in B'}\{\varepsilon_s\} \]

for every subset \(B'\) of the set \(B\). Denote by \(\widetilde B\) the totality of the functions \(f_s\) and the numbers \(\varepsilon_s\) assigned to them.

We are interested in the following question. Is it possible, for each function of the algebra of logic, to construct an arbitrarily reliable circuit* from f. e. corresponding to functions from the system \(A\cup \widetilde B\) (with the indicated bounds on the probabilities of error), i.e., for arbitrarily small \(\gamma>0\), to construct a circuit realizing the given function whose probability of error (on any set of arguments) remains less than \(\gamma\), when errors of the circuit elements corresponding to \(B\) occur independently with probabilities not exceeding the corresponding \(\varepsilon_s\)? If such a realization is possible for all functions of the algebra of logic, then we shall call the system \(A\cup \widetilde B\) \(h\)-complete.

As J. von Neumann (²) showed, the probability of error of the whole circuit cannot be smaller than the probability of error of the output element. Hence it follows that, for \(h\)-completeness of the system \(A\cup \widetilde B\), it is necessary that the system \(A\) be nonempty. It also turns out that the system \(A\) need not be complete in the ordinary sense (see (⁴)).

Let us begin with the case where \(A\) contains the constants 0 and 1.

Theorem 1. For completeness of the system \(A\cup \widetilde B\) with completely reliable constants it is necessary and sufficient that:

1) the system of functions \(A\cup B\) be complete in the ordinary sense;

* We consider here deterministic circuits, i.e., circuits of f. e. whose connections do not change during operation (in contrast to the work (²)), and in which the operations of superposition of f. e. and identification of inputs occur without error.

2) \(A\) contained: a) a nonlinear function; b) a function not belonging to the class \(S_6^*\), consisting of all disjunctions \(x_1 \vee x_2 \vee \cdots \vee x_n\), and also 0 and 1; c) a function not belonging to the class \(P_6\), consisting of conjunctions and constants \((0\) and \(1)^*\).

Necessity. The necessity of condition 1) is obvious. To prove the necessity of condition 2), we shall apply the operation of extracting a “reliable output subcircuit.” Consider a circuit \(S\) realizing some function not representable by superposition of functions from \(A\), with error probability \(\gamma < \chi_0\). Then the output element of the circuit \(S\) must be completely reliable, and therefore it realizes a function from \(A\).

Fig. 1. Extraction of a reliable output subcircuit

By a reliable output subcircuit of the circuit \(S\) we shall mean a maximal subcircuit \(P\) of the circuit \(S\), containing the output of the circuit \(S\) and consisting only of reliable elements. It follows from the definition that the circuit \(P\) quite reliably realizes some function from the closed class generated by the system \(A\), and that the inputs of the subcircuit \(P\) are connected with the outputs of unreliable elements or are inputs of the entire circuit \(S\) (see Fig. 1). If all functions from \(A\) were linear functions or belonged to one of the classes \(S_6, P_6\), then in each of these cases one could indicate such a set of input signals for the circuit \(S\) that an error of only one element, whose output is connected to an input of the circuit \(P\), necessarily entails an error at the output of the entire circuit \(S\); and in this case the error probability of the circuit \(S\) cannot be less than \(\chi_0\), whereas it has become greater than \(\gamma\).

Sufficiency. Let \(F_1, F_2, F_3\) be functions satisfying conditions 2a), 2b), and 2c), respectively. Then, by means of Post’s methods (see also \((^4)\)), one can construct from them the “mixer”
\[ m(x,y,z)=xy \vee yz \vee xz, \]
which, as Neumann showed, makes it possible to increase without bound the reliability of any functional element whose error probability is less than \(1/2\). Since all \(\varepsilon_s < 1/2\), we can realize the functions of the system \(B\) with arbitrarily high reliability, and, by virtue of the completeness of the system \(A \cup B\), also all functions of the algebra of logic.

A generalization of Theorem 1 is Theorem 2, which completely resolves the question of the \(h\)-completeness of the system \(A \cap \widetilde{B}\).

We shall need some definitions introduced by Post**. A set of \(n\)-place binary tuples satisfies the condition \([A_\mu:]\) \(([:a_\mu])\) if, for any \(\mu\) tuples from this set, one can indicate a coordinate at which each of these tuples has 1 (respectively 0).

A function \(f(x_1,\ldots,x_n)\) satisfies the condition \([A_\mu:]\) \(([:a_\mu])\) if the set of tuples on which \(f=1\) \((f=0)\) satisfies the condition \([A_\mu:]\) \(([:a_\mu])\).

The classes of functions satisfying the conditions \([A_\mu:]\) and \([:a_\mu]\) will be denoted, respectively, by \(G^\mu\) and \(F^\mu\). As Post showed, all these classes are closed, and
\[ G^2 \supset G^3 \supset \cdots \supset G^\mu \cdots,\qquad F^2 \supset F^3 \supset \cdots \supset F^\mu \supset \cdots . \]

Theorem 2. In order that the system \(A \cup \widetilde{B}\) be \(h\)-complete, it is necessary and sufficient that:

* \(S_6\) and \(P_6\) are Post’s notation \((^1)\).
** The notation used here is somewhat simpler than Post’s notation.

1) the system \(A \cup B\) was complete in the ordinary sense;

2) the system \(A\) contained: a) a nonlinear function; b) a function not belonging to the class \(S_6\); c) a function not belonging to the class \(P_6\);

3) there existed a system of integers \(\mu_1 > \mu_2 > \cdots > \mu_t > 0\) such that, if \(B_i = B\left(\dfrac{1}{\mu_i}\right)\) and \(\chi_i = \chi(B \setminus B_i)\), then: a) \(\chi_i < \dfrac{1}{\mu_{i+1}}\); b) the system \(A_i = A \cup B_{i-1}\) contains a function not belonging to the class \(G^{\mu_i+1}\); c) the system \(A_i\) contains a function not belonging to the class \(F^{\mu_i+1}\).

Necessity. The necessity of condition 2) is proved in the same way as in Theorem 1. The proof of the necessity of condition 3), by means of analogous considerations, reduces to the following lemma.

Lemma. Let a circuit \(S\) realize a function not representable by superpositions of functions from \(A\) and distinct from \(0\) (1), and let its reliable output subcircuit \(P\) be a function of the class \(G^\mu\) (\(F^\mu\)), with \(\dfrac{1}{\mu} \leq \chi_0\). Then the upper bound of the error probability of the entire circuit \(S\) is not less than \(\dfrac{1}{\mu}\).

It follows from this lemma that, with the aid of functions of the class \(G^\mu\) (\(F^\mu\)), it is impossible to increase the reliability of a circuit without bound if \(\dfrac{1}{\mu} \leq \chi_0\).

Sufficiency. Let the system \(A\) satisfy conditions 2), 3). From Post’s results it follows that either a “selector” is expressible in terms of the functions of the system \(A\), or they generate the class of functions \(G^{\mu_1}_2\), or the class \(F^{\mu_1}_2\), where \(G^{\mu_1}_2\) is the class of monotone functions satisfying the condition \([A_{\mu_1}]\) and preserving 0 and 1, and \(F^{\mu_1}_2\) is the class dual to it. The class \(G^{\mu_1}_2\) contains the monotone symmetric function \(S^k_{\mu_1 k+1}(x_1,\ldots,x_{\mu_1 k+1})\), equal to 1 on the tuples containing no more than \(k\) zeros, and 0 on the remaining tuples, while the class \(F^{\mu_1}_2\) contains the function \(S^{\mu_1 k-k+1}_{\mu_1 k+1}(x_1,\ldots,x_{\mu_1 k+1})\). For any \(\varepsilon_0 < \dfrac{1}{\mu_1}\) one can indicate a \(k\) for which the indicated functions increase reliability without bound when \(\varepsilon < \varepsilon_0\), just as the “selector” increases reliability without bound for any \(\varepsilon < \dfrac{1}{2}\) (see (2)). We note that this fact follows directly from the law of large numbers. In synthesizing a circuit realizing an arbitrary function \(g(x_1,\ldots,x_n)\), not representable by superpositions of functions from \(A\), we first construct a sufficiently reliable circuit for \(f_j \in B_1\), then for \(f_l \in B_2\), etc., using the selector or the functions \(S^k_{\mu_i k+1}\), \(S^{\mu_i k-k+1}_{\mu_i k+1}\) \((i=1,2,\ldots)\), and finally, regarding these circuits as p.e., realizing \(f_s\), we construct a circuit for \(g\). This circuit will be sufficiently reliable if sufficiently reliable circuits for the \(f_s\) have been constructed, since the error probability of the whole circuit does not exceed the sum of the error probabilities of all its elements.

We note that, since the number of different \(\mu_i\) is finite, from any \(h\)-complete system \(A \cup \widetilde{B}\) one can select a finite \(h\)-complete subsystem by taking from the system \(A\) a part of the functions \(A' \subseteq A\) (no more than two) that satisfies condition 2), and from the system \(\widetilde{B}\) a finite part \(\widetilde{B}'\), so that for \(A' \cup \widetilde{B}'\) condition 3) is satisfied. Now, adjoining to \(A' \cup \widetilde{B}'\) a finite set \(C\) of functions from \(A \cup B\) up to a complete system in the ordinary sense, we obtain an \(h\)-complete system \(A' \cup \widetilde{B}' \cup \widetilde{C}\).

Following Post, define the closed classes of functions \(G^\infty\) and \(F^\infty\). The class \(G^\infty\) consists of functions of the form \(x_1 \cdot g(x_2,\ldots,x_n)\), the class \(F^\infty\) of functions \(x_1 \vee g(x_2,\ldots,x_n)\).

Theorem 3. In order that a system \(A\) generate a function that raises reliability for some \(\varepsilon > 0\), it is necessary and sufficient that the system \(A\) satisfy condition 2) of Theorem 2 and, moreover, co-

would contain: a) a function not belonging to the class \(G^\infty\); b) a function not belonging to the class \(F^\infty\).

Received
29 XII 1961

REFERENCES

1. E. L. Post, Two-valued Iterative Systems of Mathematical Logic, Princeton, 1941.
2. J. Neumann, in: Automata, IL, 1956, p. 68.
3. E. F. Moore, K. E. Shannon, Cybernetics Collection, 1, IL, 1960, p. 109.
4. S. V. Yablonskii, Tr. Mat. Inst. im. V. A. Steklova AN SSSR, 51, 5 (1958).