CYBERNETICS AND CONTROL THEORY

V. I. VARSHAVSKII

FUNCTIONAL CAPABILITIES AND SYNTHESIS OF THRESHOLD ELEMENTS

(Presented by Academician P. S. Novikov on 31 III 1961)

We shall call a threshold element an element whose behavior in a discrete circuit is described by the expression

\[ y = \operatorname{sign}\left(\sum_{j=0}^{n-1}\xi_j x_j - \eta\right), \tag{1} \]

where \(y\) is a binary variable characterizing the state of the element output; \(x_j\) is a binary variable characterizing the state of the \(j\)-th input; \(\xi_j\) and \(\eta\) are certain integers;

\[ \operatorname{sign} z = \begin{cases} 0, & z < 0,\\ 1, & z \geqslant 0. \end{cases} \tag{2} \]

Under certain natural restrictions, expression (1) can be used to describe: a biological neuron, a ferrite–transistor cell with magnetization, a parametron, a cryotron, a polarized relay and, in general, any summing circuit with a phase-sensitive null indicator \((1^{-3})\).

If the state of each input of the element is put in correspondence with a certain coordinate of an \(n\)-dimensional space, then the set of all possible input actions is represented as the set of vertices of the unit \(n\)-dimensional hypercube \(S^n\), and expression (1) defines a closed half-space separated by a hyperplane with equation

\[ \sum_{j=0}^{n-1}\xi_j x_j - \eta = 0, \tag{3} \]

in which lie the vertices of the hypercube corresponding to those input actions that cause excitation of the element output. Thus, the problem of determining the functional capabilities of a threshold element reduces to the problem of determining the properties of subsets of vertices \(S^n\), \(T\) and \(F\), such that \(T \cap F = 0\), \(T \cup F = S^n\), and there exists a hyperplane separating them.

Let \(t = (t_0, \ldots, t_{n-1}) \in T\), \(f = (f_0, \ldots, f_{n-1}) \in F\), and

\[ \sum_{j=0}^{n-1}\xi_j t_j - \eta \geqslant 0, \]

\[ \sum_{j=0}^{n-1}\xi_j f_j - \eta < 0. \]

Suppose that \(\xi_j > 0\) and \(\eta \geqslant 0\).

Lemma 1. The set \(T\) has the following property: if the vector

\[ t = (t_0, \ldots, t_{j-1}, 0, t_{j+1}, \ldots, t_{n-1}) \in T, \]

then the vector

\[ t = (t_0, \ldots, t_{j-1}, 1, t_{j+1}, \ldots, t_{n-1}) \in T. \]

Lemma 2. The set \(F\) has the following property: if the vector

\[ f = (f_0, \ldots, f_{j-1}, 1, f_{j+1}, \ldots, f_{n-1}) \in F, \]

then the vector

\[ f = (f_0, \ldots, f_{j-1}, 0, f_{j+1}, \ldots, f_{n-1}) \in F. \]

Theorem 1. A logical function specified by a set of vectors \(T\) is monotone.

For the concept of monotonicity for logical functions, see (4).

Let us call a set of subcubes \(S^n\) having at least one common vertex a star.

Theorem 2. The set \(T\) is a star with vertex \((1,1,1,\ldots,\ldots,1,1)\). It is not difficult to show that the set \(T\) is a star with vertex \((0,0,0,\ldots,0,0)\).

Let the support set \(T_0\) of the set \(T\) be the set of vertices lying on the maximal diagonals of the subcubes forming a star, relative to their common vertex. Define \(F_0\) analogously.

Theorem 3. A necessary and sufficient condition for the existence of a hyperplane strictly separating the set \(F\) from the set \(T\) is the existence of a hyperplane strictly separating the set \(F_0\) from the set \(T_0\).

The coefficients of the separating hyperplane are determined from the solution of the system of inequalities

\[ \sum_{j=0}^{n-1}\xi_j t_j-\eta \geqslant 0 \quad \text{for all } t\in T_0, \]

\[ \sum_{j=0}^{n-1}\xi_j f_j-\eta < 0 \quad \text{for all } f\in F_0, \tag{4} \]

or, taking into account the integrality condition on the coefficients and variables:

\[ \sum_{j=0}^{n-1}\xi_j t_j-\eta \geqslant 0 \quad \text{for all } t\in T_0, \]

\[ -\sum_{j=0}^{n-1}\xi_j f_j+\eta \geqslant 1 \quad \text{for all } f\in F_0. \tag{5} \]

Theorem 4. If the star corresponding to the logical function \(g(x_0,x_1,\ldots,x_{n-1})\) is separable, then the star corresponding to the function

\[ \varphi(x_0,\ldots,x_{n-1},x_n,\ldots,x_{m-1}) = x_n \vee \cdots \vee x_{m-1}\vee g(x_0,\ldots,x_{n-1}) \]

is also separable, and \(\eta_\varphi=\eta_g\) and \(\xi_k=\eta_g\) for all \(k\) from \(n\) to \(m-1\).

Theorem 5. If the star corresponding to the logical function \(g(x_0,x_1,\ldots,x_{n-1})\) is separable, then the star corresponding to the function

\[ \varphi(x_0,\ldots,x_{n-1},x_n,\ldots,x_{m-1}) = x_n \wedge \cdots \wedge x_{m-1}\wedge g(x_0,\ldots,x_{n-1}) \]

is also separable, and

\[ \xi_k=\sum_{j=0}^{n-1}\xi_j-\eta_g+1, \qquad \eta_\varphi=\eta_g+(m-n)\xi_k \]

(for all \(k\) from \(n\) to \(m-1\)).

Theorems 4 and 5 make it possible to substantially restrict the class of sets under consideration.

If a separating hyperplane exists, then for at least one \(\xi_k\) it is true that \(\xi_k\leqslant \xi_j\) \((j=0,1,\ldots,n-1)\).

Theorem 6. If \(\xi_k=\xi_{\min}\), then there is no vector \(f\in(F\cap \overline{F}_0)\) adjacent in direction \(k\) to a vector \(t\in T_0\).

Theorem 7 (converse of Theorem 6). If all vectors \(f\in F\) adjacent to vectors \(t\in T_0\) in direction \(k\) belong to \(F_0\), then \(\xi_k=\xi_{\min}\).

If \(\xi_k=1\), then all vertices satisfying the condition of Theorem 7 lie in a hyperplane, i.e., they determine a system of equations. If the system is inconsistent, then \(\xi_k=2\), and from the resulting system some maximal consistent subsystem of equations is selected.

If at the first step we obtain \(\xi_k=\xi_{\min}\), then from the sets \(T_0\) and \(F_0\) the sets \(T'_0\) and \(F'_0\) can be formed by excluding pairs of adjacent vectors. By excluding the coordinates for those \(\xi_k\) that are determined at the first step, the space of dimension \(n\) is transformed into a space of dimension \(n-c_1\), where \(c_1\) is the number of \(\xi_k=\xi_{\min}\), and \(T'_0\) and \(F'_0\) are transformed into \(T'_{01}\) and \(F'_{01}\), respectively.

Theorem 8. If all vectors \(f'\in F'_1\) adjacent to vectors \(t'\in T'_{01}\) in direction \(l\) belong to \(F'_{01}\), and \(c_1\) direction coefficients are equal to \(\xi_{\min}\), then
\[ \xi_l \ge 1+c_1\xi_{\min}. \]

In the case \(\xi_l=1+c_1\xi_{\min}\), the vertices determining \(\xi_l\) lie in a hyperplane and determine a system of equations. If it is inconsistent with the system obtained earlier, \(\xi_l=2+c_1\xi_{\min}\), and from the newly obtained equalities one selects the maximal set which, together with the previously obtained system, forms a consistent system of equations. The assertion of Theorem 8 is easily extended to the case of any step of the process. The process is continued until a consistent system of equations is obtained that determines all the remaining coefficients of the hyperplane. Convergence of the process is a condition for realizing the logical function by a threshold element. The indicated process is valid only for monotone functions. In the case of a nonmonotone function, an additional condition for realizability is the possibility of transforming the given function into a monotone one by inverting variables. After the equation of the separating hyperplane has been obtained for the monotone function, it is transformed into the equation of the separating hyperplane for the original function by substituting \((1-x_j)\) instead of \(x_j\) for the inverted variables.

Let us consider an example. Suppose the minimal disjunctive form of a monotone logical function is given (for a monotone logical function the reduced form coincides with the minimal one (4)):
\[ g(x_0,x_1,x_2,x_3,x_4,x_5) = x_5x_4x_3+x_5x_4x_2+x_5x_4x_1+x_5x_4x_0+x_5x_3x_2+x_5x_3x_1 +x_5x_3x_0+x_4x_3x_2+x_5x_2x_1x_0+x_4x_3x_1x_0. \]

It is not difficult to obtain:
\[ \begin{aligned} \overline{g}(x_0,x_1,x_2,x_3,x_4,x_5) ={}& \bar{x}_5\bar{x}_4+\bar{x}_5\bar{x}_3+\bar{x}_3\bar{x}_2\bar{x}_1\bar{x}_0+\bar{x}_5\bar{x}_2\bar{x}_1\\ &+\bar{x}_5\bar{x}_2\bar{x}_0+\bar{x}_4\bar{x}_2\bar{x}_1\bar{x}_0+\bar{x}_4\bar{x}_3\bar{x}_2+\bar{x}_4\bar{x}_3\bar{x}_1+\bar{x}_4\bar{x}_3\bar{x}_0 . \end{aligned} \]

The vertices of the sets \(T_0\) and \(F_0\) correspond to the terms of the minimal disjunctive form, whence:
\[ T_0=\{56,52,50,49,44,42,41,39,28,27\}^{*}, \]
\[ F_0=\{48,40,38,37,35,26,25,23,15\}. \]

Consider Table 1. In the table, the rows correspond to vertices from \(T_0\), the columns to vertices from \(F_0\). To each vertex number there is assigned a numbered set of coordinates, and to each coordinate the corresponding power of two is assigned. If for some vertex from \(T_0\) some \(t_j=1\), then from this vertex in direction \(j\) there emerges a one-dimensional element ending at a vertex from \(F\), with \(N(t)-2^j=N(f)\) (\(N(t)\) is the number of the vertex \(t\)). If \(f\in F_0\), then at the intersection of the corresponding row and column the number \(j\) is entered.

The condition of Theorem 7 is satisfied by those directions for which all ones in column \(j\) from \(T_0\) correspond to marks \(j\) in the cells of the table. Such directions in Table 1 are the zeroth and the first; hence \(\xi_0=\xi_1=\min\). The vertices 50, 49, 42, 41, 39, and 27 may lie in the hyperplane, whence
\[ \xi_5+\xi_4=\eta-1,\qquad \xi_5+\xi_3=\eta-1,\qquad \xi_5+\xi_2=\eta-2,\qquad \xi_3+\xi_4=\eta-2. \]

The system is consistent and, consequently, \(\xi_0=\xi_1=1\). The sets \(T'_0\) and \(F'_0\) will be respectively \(\{28\}\) and \(\{23,15\}\), and the sets \(T'_{01}\) and \(F'_{01}\), respectively, \(\{7\}\) and \(\{5,3\}\).

* The numbers are decimal equivalents of binary codes formed by the ordered-by-\(j\) sets of coordinates of the vertices.

Table 1

Let us consider Table 2. The construction of Table 2 is analogous to Table 1. The conditions of Theorem 8 correspond to directions 3 and 4, whence \(\xi_3 = \xi_4 \geq 1 + 2 \cdot 1 = 3\). For \(\xi_3 = \xi_4 = 3\) we obtain the additional equality \(\xi_2 = \eta - 6\), solving which jointly with the system obtained earlier we have

\[ \xi_0 = \xi_1 = 1, \qquad \xi_2 = 2, \]

\[ \xi_3 = \xi_4 = 3, \qquad \xi_5 = 4, \qquad \eta = 8 \]

or we obtain the excitation condition of the element

\[ x_0 + x_1 + 2x_2 + 3x_3 + 3x_4 + 4x_5 \geq 8. \]

Table 2

Since at each step of the process we obtain at least one coefficient and at least one equality, in one step we in fact determine at least two parameters of the hyperplane; consequently, the synthesis procedure must have no more than

\[ \left] \frac{n+1}{2} \right[ \]

steps.

Leningrad Branch
of the V. A. Steklov Mathematical Institute
Academy of Sciences of the USSR

Received
25 III 1961

CITED LITERATURE

1. K. Ganzhorn, Electr. Rundsch., 11, No. 8 (1957).
2. S. Muroga, Proc. ICIP UNESCO (1959).
3. M. C. Paull, E. I. McCluskey, Proc. IRE, 48, No. 7 (1960).
4. S. V. Yablonskii, Tr. Mat. Inst. im. V. A. Steklova, AN SSSR, 51, 5 (1958).