CYBERNETICS AND CONTROL THEORY

E. I. NECHIPORUK

ON VALVE CIRCUITS

(Presented by Academician M. V. Keldysh, 4 VII 1962)

Let us consider the realization of Boolean matrices by valve circuits. We shall call the depth of a valve circuit the greatest of the lengths of chains connecting the input and output poles of the circuit. Denote by \(B_m(p,q)\) Shannon’s function for realizations of \((p,q)\)-matrices* by circuits of depth not exceeding \(m\), and by \(B(p,q)\) Shannon’s function for realizations by circuits of arbitrary depth. In \((^1)\) an asymptotic estimate was obtained for

\[ B(p_n,q_n) \]

under the condition

\[ v_n=\frac{\lg_2 q_n}{\lg_2 p_n}\to 0 \]

(and some others). At the same time it turns out that \(B(p_n,q_n)\sim B_2(p_n,q_n)\). Below we succeed in obtaining the estimate \(B(p_n,q_n)\sim B_3(p_n,q_n)\) under the condition that \(\lim v_n\) exists and belongs to some nowhere dense set of points of the interval \([0,1]\).

Estimates of the complexity of valve circuits have numerous applications; for example, the realization of linear transformations is almost uniquely described by valve circuits.

\(1^\circ\). Denote by \(\chi(\mathfrak A)\) the matrix realized by the circuit \(\mathfrak A\). Let \(\mathfrak A'\) be a \((p,r)\)-circuit and \(\mathfrak A''\) an \((r,q)\)-circuit. Denote by \(\mathfrak A'\times \mathfrak A''\) the \((p,q)\)-circuit obtained as a result of identifying the \(l\)-th output pole of the circuit \(\mathfrak A'\) with the \(l\)-th input pole of the circuit \(\mathfrak A''\) \((l=1,\ldots,r)\).

Lemma.

\[ \chi(\mathfrak A'\times \mathfrak A'')=\chi(\mathfrak A')\times \chi(\mathfrak A''). \]

(The symbol \(\times\) also denotes multiplication of matrices.)

Denote by \(\|A\|\) the number of ones in the matrix \(A\). Denote by \(\mathfrak B(p_n,q_n,\alpha_n)\) the class of all Boolean \((p_n,q_n)\)-matrices \(A\) such that \(\|A\|=\alpha_n p_n q_n\) (the conditions \(0\le \alpha_n\le 1\) and that \(\alpha_n p_n q_n\) is a natural number will be understood henceforth without explicit mention).

Theorem 1. Suppose the following conditions are satisfied:

a) \(q_n\le p_n\);

b) \(\alpha_n q_n\to\infty\);

c) \(\lg_2 p_n/\lg_2 \dfrac{1}{\alpha_n}\to \rho\), where \(\rho\) is an integer greater than zero;

d) \(p_n\alpha_n^\rho\to\infty\).

Then for \(\mathfrak B(p_n,q_n,\alpha_n)\)

\[ B_2(p_n,q_n)\sim \frac{\alpha_n p_n q_n}{\rho}. \]

Proof. The lower bound is obtained as in \((^1)\).

Upper bound. Let \(D_n\in \mathfrak B(p_n,q_n,\gamma_n)\), \(\beta_n\le \gamma_n\le \alpha_n\), where \(\beta_n\) is an arbitrary parameter satisfying the conditions

\[ \frac{\beta_n}{\alpha_n}\to 0,\qquad \beta_n q_n\to\infty,\qquad p_n\beta_n^\rho\to\infty. \]

* In the present paper the definition of a valve \((p,q)\)-circuit introduced in \((^1)\) is used.

** By a \((p,q)\)-matrix we mean a matrix having \(p\) rows and \(q\) columns.

If \(n\) is sufficiently large, then in the matrix \(D_n\) there exists a \((t_n,\rho)\)-submatrix consisting entirely of ones, with \(t_n \to \infty\). Indeed, the number of all \((1,\rho)\)-submatrices of the matrix \(D_n\) consisting entirely of ones is not less than \(p_n C^\rho_{[\gamma_n q_n]}\) (because the smallest number of such submatrices occurs in the case of a uniform distribution of the ones among the rows of the matrix \(D_n\)). Let us call the type of a \((1,\rho)\)-submatrix the system of numbers of the columns of the matrix \(D_n\) in which this submatrix is located. The \(t_n\) different \((1,\rho)\)-submatrices of one type form a \((t_n,\rho)\)-submatrix. Since the number of types is equal to \(C^\rho_{q_n}\), there exists at least one type to which belong at least

\[ t_n=\left]\frac{p_n C^\rho_{[\gamma_n q_n]}}{C^\rho_{q_n}}\right[ \]

\((1,\rho)\)-submatrices consisting entirely of ones. \(t_n \sim p_n\gamma_n^\rho\), since \(\gamma_n q_n \to \infty\).

We describe the process of constructing the matrices \(D_n^{(k)}\), \(k=1,\ldots,N_n(\beta_n)+1\). Denote by \(\gamma_n^{(k)}\) the number \(\|D_n^{(k)}\|/p_n q_n\). Put \(D_n^{(1)}=A_n\). If \(\beta_n \leqslant \gamma_n^{(k)}\), then, by the preceding, we select in the matrix \(D_n^{(k)}\) a \((t_n^{(k)},\rho)\)-submatrix consisting entirely of ones; denote it by \(a_n^{(k)}\). Denote by \(A_n^{(k)}\) the matrix obtained from \(D_n^{(k)}\) by replacing all elements by zeros except the elements of the submatrix \(a_n^{(k)}\), and by \(D_n^{(k+1)}\) the matrix obtained from \(D_n^{(k)}\) by replacing by zeros all elements of the submatrix \(a_n^{(k)}\); we have \(D_n^{(k)}=A_n^{(k)}\vee D_n^{(k+1)}\). If \(\gamma_n^{(k)}<\beta_n\), then we put \(k=N_n(\beta_n)+1\). Thus, the process consists in successively selecting submatrices filled with ones; the matrices formed become increasingly “sparse,” and the process terminates as soon as a matrix is formed containing fewer than \(\beta_n p_n q_n\) ones.

Denote by \(H_n\) the matrix \(D_n^{(N_n(\beta_n)+1)}\). We have
\(A_n=A_n^{(1)}\vee\ldots\vee A_n^{(N_n(\beta_n))}\vee H_n\),
\(\min_{k=1,\ldots,N_n(\beta_n)} t_n^{(k)} \gtrsim p_n\beta_n^\rho\),
\(\|H_n\|<\beta_n p_n q_n\).
Represent each matrix \(A_n^{(k)}\) in the form \(A_n^{(k)}=F_n^{(k)}\times G_n^{(k)}\), where \(F_n^{(k)}\) is a \((p_n,1)\)-matrix, \(G_n^{(k)}\) is a \((1,q_n)\)-matrix, \(\|F_n^{(k)}\|=t_n^{(k)}\), \(\|G_n^{(k)}\|=\rho\), and let

\[ F_n=\bigl(F_n^{(1)},\ldots,F_n^{(N_n(\beta_n))}\bigr),\qquad G_n= \begin{pmatrix} G_n^{(1)}\\ \vdots\\ G_n^{(N_n(\beta_n))} \end{pmatrix}. \]

Then

\[ A_n=F_n\times G_n\vee H_n, \tag{1} \]

\[ \|F_n\|\leqslant \frac{\alpha_n p_n q_n}{\rho},\qquad \|G_n\|\leqslant \frac{\alpha_n q_n}{\beta_n^\rho},\qquad \|H_n\|<\beta_n p_n q_n. \tag{2} \]

Realizing each matrix \(F_n\), \(G_n\), \(H_n\) by a circuit of depth 1, we obtain, by the lemma, a circuit for \(A_n\). By (2),
\(B_2(p_n,q_n)\lesssim \alpha_n p_n q_n/\rho\). The theorem is proved.

Remark. If conditions a), b), d) are satisfied and in condition c) \(\rho\) is not an integer, then

\[ B_2(p_n,q_n)\lesssim \frac{\alpha_n p_n q_n}{[\rho]}. \]

It can be shown that in this case the trivial power lower bound is ineffective, i.e., it can be improved.

2°. Theorem 2. Suppose the following conditions are satisfied:

a) \(q_n \leqslant p_n\);

b) \(q_n \to \infty\);

c)
\[ \lim_{n\to\infty}\frac{\lg_2 q_n}{\lg_2 p_n} = \frac{\mu}{\mu(\rho-1)+\rho}, \]
where \(\mu,\rho\) are positive integers.*

Then
\[ B(p_n,q_n)\sim B_3(p_n,q_n)\sim \frac{p_n q_n}{\lg_2(p_n q_n)}. \]

Proof. The lower estimate is the cardinality estimate.

The upper estimate. Let \(A_n\) be a \((p_n,q_n)\)-matrix. Introduce the parameter \(\vartheta_n\) and divide the matrix \(A_n\) into
\[ T_n=\left]\frac{q_n}{\vartheta_n}\right[ \]
nonoverlapping \((p_n,\vartheta_{n,k})\)-submatrices
\[ A_n^{(k)},\qquad A_n=\bigl(A_n^{(1)},\ldots,A_n^{(T_n)}\bigr), \]
so that \(\vartheta_{n,k}=\vartheta_n\) for \(k=1,\ldots,T_n-1\) and \(\vartheta_{n,T_n}\leqslant\vartheta_n\). Form a \((2^{\vartheta_{n,k}},\vartheta_{n,k})\)-matrix \(\Sigma_{n,k}\), whose rows are all possible distinct vectors, taken in arbitrary order. Obviously, there exist \((p_n,2^{\vartheta_{n,k}})\)-matrices \(B^{(k)}(A_n)\), having exactly one one in each row, such that
\[ A_n^{(k)}=B^{(k)}(A_n)\times \Sigma_{n,k},\qquad k=1,\ldots,T_n. \]
Introduce the matrices
\[ B(A_n)=\bigl(B^{(1)}(A_n),\ldots,B^{(T_n)}(A_n)\bigr), \]
\[ \Sigma_n= \begin{pmatrix} \Sigma_{n,1} & 0\\ \ddots & \\ 0 & \Sigma_{n,T_n} \end{pmatrix}. \]
Then
\[ A_n=B(A_n)\times \Sigma_n. \]

Denote by \(q'_n\) the number
\[ \sum_{k=1}^{T_n}2^{\vartheta_{n,k}}. \]
Introduce the parameter \(\lambda_n\) and divide the matrix \(B(A_n)\) into
\[ U_n=\left]\frac{p_n}{\lambda_n}\right[ \]
nonoverlapping \((\lambda_{n,i},q'_n)\)-submatrices
\[ B_i(A_n), \]
\[ B(A_n)= \begin{pmatrix} B_1(A_n)\\ \vdots\\ B_{U_n}(A_n) \end{pmatrix} \]
so that \(\lambda_{n,i}=\lambda_n\) for \(i=1,\ldots,U_n-1\) and \(\lambda_{n,U_n}\leqslant\lambda_n\). Form a \((\lambda_{n,i},C_{\lambda_{n,i}}^{\mu+1})\)-matrix \(\mathfrak S^{(n,i)}\), whose columns are all possible distinct vectors containing exactly \(\mu+1\) ones, taken in arbitrary order (if \(\lambda_{n,i}<\mu+1\), then we put \(C_{\lambda_{n,i}}^{\mu+1}=0\)). Obviously, there exist \((C_{\lambda_{n,i}}^{\mu+1},q'_n)\)-matrices \(C_i(A_n)\) \((i=1,\ldots,U_n)\) such that
\[ B_i(A_n)=\mathfrak S^{(n,i)}\times C_i(A_n)\vee B_i^*(A_n), \]
where
\[ \|B_i^*(A_n)\|\leqslant(\mu+1)q'_n,\qquad \frac{\lambda_n T_n}{\mu+1}-q'_n\leqslant \|C_i(A_n)\|\leqslant \frac{\lambda_n T_n}{\mu+1} \]
(the product of an \((a,0)\)-matrix by a \((0,b)\)-matrix is defined as an \((a,b)\)-matrix consisting entirely of zeros). Introduce the matrices
\[ \mathfrak S^{(n)}= \begin{pmatrix} \mathfrak S^{(n,1)} & 0\\ \ddots & \\ 0 & \mathfrak S^{(n,U_n)} \end{pmatrix}, \qquad C(A_n)= \begin{pmatrix} C_1(A_n)\\ \vdots\\ C_{U_n}(A_n) \end{pmatrix}, \qquad B^*(A_n)= \begin{pmatrix} B_1^*(A_n)\\ \vdots\\ B_{U_n}^*(A_n) \end{pmatrix}. \]

* Condition c) includes the important case for applications when \(p_n \succ q_n\).

Then

\[ B(A_n)=\mathfrak S^{(n)}\times C(A_n)\vee B^*(A_n). \tag{4} \]

Denote by \(p'_n\) the number \(\displaystyle \sum_{i=1}^{U_n} C_{\lambda_n,i}^{\mu+1}\); \(C(A_n)\) is a \((p'_n,q'_n)\)-matrix; denote by \(\alpha_n\) the number \(\|C(A_n)\|/p'_nq'_n\), and by \(v_n\) the number \(\lg_2 q_n/\lg_2 p_n\).

Introduce, for the parameters \(\vartheta_n,\lambda_n\), the conditions: 1) \(\lambda_n/2^{\vartheta_n}\to\infty\); 2) \(q_n/\vartheta_n\lambda_n^\mu\to\infty\). Then
\[ T_n\sim q_n/\vartheta_n,\quad q'_n\sim q_n2^{\vartheta_n}/\vartheta_n,\quad U_n\sim p_n/\lambda_n,\quad p'_n\sim p_n\lambda_n^\mu/(\mu+1)!,\quad \|C(A_n)\|\sim p_nq_n/(\mu+1)\vartheta_n, \]
\[ \alpha_n\sim \mu!/2^{\vartheta_n}\lambda_n^\mu,\quad \alpha_nq'_n\to\infty. \]

Introduce, further, the conditions: 3) \(\lg_2 p'_n\big/\lg_2\dfrac{1}{\alpha_n}\to\rho\); 4) \(p'_n\alpha_n^\rho\to\infty\).

Introduce the parameter \(\beta_n\) and the conditions: 5) \(\beta_n2^{\vartheta_n}\lambda_n^\mu\to0\); 6) \(\beta_nq_n2^{\vartheta_n}/\vartheta_n\to\infty\); 7) \(p_n\lambda_n^\mu\beta_n^\rho\to\infty\).

Apply to \(C(A_n)\) the construction of Theorem 1; by virtue of (1), (3), (4),
\[ C(A_n)=F_n\times G_n\vee H_n \]
and
\[ A_n=\mathfrak S^{(n)}\times F_n\times(G_n\times\Sigma_n)\vee (\mathfrak S^{(n)}\times H_n\vee B^*(A_n))\times\Sigma_n. \]

We obtain the circuit for \(A_n\) from circuits of depth 1 for \(F_n,\mathfrak S^{(n)},G_n\times\Sigma_n,H_n,B^*(A_n),\Sigma_n\) (lemma). We have
\[ \|\mathfrak S^{(n)}\|\le p_n\lambda_n^\mu,\quad \|\Sigma_n\|\le q_n2^{\vartheta_n},\quad \|B^*(A_n)\|\le \frac{p_nq_n2^{\vartheta_n}}{\vartheta_n\lambda_n}, \]
and, by virtue of (2),
\[ \|F_n\|\le p_nq_n(\mu+1)\rho^{\vartheta_n},\quad \|G_n\times\Sigma_n\|\le \vartheta_n\|G_n\|\le \frac{q_n}{\lambda_n^\mu p_n^{\beta_n^\rho}}, \]
\[ \|H_n\|\le \frac{\beta_np_nq_n2^{\vartheta_n}\lambda_n^\mu}{\vartheta_n}. \]

In order that the relations
\[ B_3(p_n,q_n)\sim\|F_n\|\le p_nq_n/\lg_2(p_nq_n) \]
hold, it is sufficient that the following additional conditions hold:
\[ \begin{gathered} 8)\;(\mu+1)\rho^{\vartheta_n}\ge(1+v_n)\lg_2p_n;\qquad 9)\;\lambda_n^\mu\lg_2p_n/q_n\to0;\qquad 10)\;2^{\vartheta_n}\lg_2p_n/p_n\to0;\\ 11)\;2^{\vartheta_n}\lg_2p_n/\vartheta_n\lambda_n\to0;\quad 12)\;\lg_2p_n/p_n\lambda_n^\mu\beta_n^\rho\to0;\quad 13)\;\beta_n2^{\vartheta_n}\lambda_n^\mu\lg_2p_n/\vartheta_n\to0. \end{gathered} \]

By the conditions of the theorem,
\[ v_n=\frac{\mu}{\mu(\rho-1)+\rho}+\varphi_n, \]
where \(\varphi_n\to0\). Put, for sufficiently large \(n\),
\[ \lambda_n=\left[p_n^{\frac{1}{\mu(\rho-1)+\rho}+\delta_n}\right],\quad \vartheta_n=\left[\left(\frac{1}{\mu(\rho-1)+\rho}+\xi_n\right)\lg_2p_n\right]; \]
\[ \frac{1}{\beta_n}=p_n^{\frac{\mu+1}{\mu(\rho-1)+\rho}+\eta_n}; \]
\[ \delta_n=\frac{\varphi_n-3\,\lg_2\lg_2p_n/\lg_2p_n}{\mu}; \quad \xi_n=-3\,\frac{\lg_2\lg_2p_n}{\lg_2p_n} +\min\left[\delta_n,\mu\left(\frac{1}{\rho}-1\right)\delta_n\right]; \]
\[ \eta_n=\mu\delta_n+\xi_n+\frac{\lg_2\lg_2p_n}{\lg_2p_n}. \]

Then conditions 1)—13) are fulfilled. The theorem is proved.

Leningrad State University
named after A. A. Zhdanov

Received
3 VII 1962

REFERENCES

1. O. B. Lupanov, DAN, 111, 6, 1171 (1956).