UDC 519.95

CYBERNETICS AND THE THEORY OF CONTROL

G. S. Plesnevich

ESTIMATES OF THE MEAN COMPUTATION TIME ON ONE-DIMENSIONAL ONE-SIDED ITERATIVE SYSTEMS

(Presented by Academician V. M. Glushkov, January 24, 1966)

This note considers one-dimensional one-sided iterative systems with combinational cells and with delays in the side channels \((^{1})\). An arbitrary system of this type is specified by a tuple \(\mathscr{S}=\langle X,Y,S,s^{\#},f,g\rangle\), where \(X\) and \(Y\) are the input and output alphabets of the system; \(S\) is the alphabet of side signals; \(s^{\#}\) is the boundary signal \((s^{\#}\in S)\); \(f:X\times S\to S\) is the function transforming side signals and \(g:X\times S\to Y\) is the output function. We shall call the state of an \(n\)-network of the iterative system \(\mathscr{S}\) at some instant of discrete time the collection of the states of all side channels of the \(n\)-network at that instant (including the state of the side channel entering the first cell of the network and the state of the side channel leaving the last cell of the network). It is convenient to represent the state of an \(n\)-network by a word of length \(n+1\) in the alphabet \(S\), assuming that the \(i\)-th letter of this word is the side signal stored in the \(i\)-th side channel of the \(n\)-network. (The numbering of the side channels of the network is carried out in the direction in which side signals pass through the network, the first channel being assigned number 0.)

Let an arbitrary input \(n\)-word \(\xi=x_1x_2\ldots x_n\) be given, and let the state of the \(n\)-network at some instant of time \(t\) be \(\sigma=s_0s_1\ldots s_n\). Then the state of the \(n\)-network at time \(t+1\) is represented by the word \(\sigma'=s'_0s'_1\ldots s'_n\), where \(s'_i=f(x_i,s_{i-1})\), if \(1\le i\le n\), and \(s'_0=s^{\#}\). Thus, each input \(n\)-word \(\xi\) determines a transformation \(T_\xi\) of the states of the \(n\)-network. A state of the \(n\)-network is called a stable state under the input word \(\xi\) if \(T_\xi\sigma=\sigma\). For each input \(n\)-word there exists a unique stable state of the \(n\)-network.

The \(n\)-network of the iterative system \(\mathscr{S}\) defines, in the following way, a mapping \(\varphi\) of all \(n\)-words in the alphabet \(X\) into \(n\)-words in the alphabet \(Y\). Let \(\xi=x_1x_2\ldots x_n\) be an arbitrary input \(n\)-word and let \(\sigma=s_0s_1\ldots s_n\) be the stable state of the \(n\)-network for this input word. Then \(\varphi(\xi)=y_1y_2\ldots y_n\), where \(y_i=g(x_i,s_{i-1})\).

Let \(\sigma\) be the initial state of the \(n\)-network, i.e., the state of the network at time \(t=0\), and let \(\xi\) be some input \(n\)-word. Consider the sequence \(\sigma(0)=\sigma,\ \sigma(1)=T_\xi\sigma(0),\ldots,\sigma(t+1)=T_\xi\sigma(t),\ldots\) of states through which the \(n\)-network passes at successive instants of time. At least for \(t=n\), the state \(\sigma(t)\) will be stable under the input word \(\xi\). Denote by \(\tau(\xi,\sigma)\) the smallest such \(t\) for which the state \(\sigma(t)\) is stable under the input word \(\xi\). (For example, if \(\sigma\) is a stable state under the input word \(\xi\), then \(\tau(\xi,\sigma)=0\). Conversely, if \(\tau(\xi,\sigma)=0\), then \(\sigma\) is a stable state under the word \(\xi\).)

Suppose that probability distributions \(p(\xi)\) and \(p(\sigma)\) are given on input \(n\)-words and on states of the \(n\)-network. Then the mean computation time on the \(n\)-network \(\varepsilon_n\) (the mathematical expectation of the computation time) is defined by the formula \(\varepsilon_n=\sum \tau(\xi,\sigma)p(\xi)p(\sigma)\), where the summation is over all \(n\)-words \(\xi\) and all \((n+1)\)-words \(\sigma\).

In this note we shall assume that each letter \(x \in X\) has a constant probability \(p_x > 0\) of appearing in any position of any input word. Then, if \(\xi = x_1x_2 \ldots x_n\), then \(p(\xi)=p_{x_1}p_{x_2}\ldots p_{x_n}\). The initial state will be taken to be \(\sigma^{\#}=s^{\#}s^{\#}\ldots s^{\#}\), i.e. \(p(\sigma)=0\), if \(\sigma \ne \sigma^{\#}\), and \(p(\sigma^{\#})=1\).

Remark 1. In physical realizations of iterative networks the boundary signal is encoded by a set of zeros, i.e. by the absence of pulses. Thus, setting the network to the initial state \(\sigma^{\#}\) corresponds to the operation of “resetting” the states of the delays. Such a choice of the initial state is not the only natural choice. For example, one may assume that before each computation the initial state of the network is the (stable) state in which the network settled after the preceding computation. In this situation the distribution \(p(\sigma)\) is determined by the formula \(p(\sigma)=\sum p(\xi)\), where the summation is over all input \(n\)-words \(\xi\) such that \(\tau(\xi,\sigma)=0\).

Theorem 1. Let the distribution of input words \(p(\xi)\) be chosen as indicated above. Then with each iterative system \(\mathcal S\) there is associated a number \(\rho\) such that: 1) \(0 \le \rho \le 1\); 2) if \(\rho=0\), then \(\varepsilon_n=O(1)\), \(n \to \infty\); 3) if \(\rho=1\), then \(\varepsilon_n=n-O(1)\), \(n \to \infty\); 4) if \(0<\rho<1\), then, as \(n\to\infty\),
\[ \varepsilon_n=\left(1/\log \frac1\rho\right)\log n+O(\log\log n). \]
(Recall that, by assumption, the computations are carried out from the initial state \(\sigma^{\#}\).)

In the proof of this theorem the following two lemmas are used.

Lemma 1. Let \(R\) be the set of all input words \(\widetilde{\xi}\) such that \(\tau(\widetilde{\xi},\sigma^{\#})=n\), where \(n\) is the length of the word \(\widetilde{\xi}\). Then \(R\) is a complete regular event, and for an arbitrary input word \(\xi\)
\[ \tau(\xi,\sigma^{\#})=\max\{k:\exists \widetilde{\xi}\ (\text{the word } \widetilde{\xi}\text{ is contained in the word } \xi)\land(\text{the length of the word } \widetilde{\xi}\text{ is }k)\land(\widetilde{\xi}\in R)\}. \]
(If \(N\) is any set of natural numbers, then \(\max N\) denotes the greatest element of \(N\) when \(N\ne\varnothing\), and \(\max\varnothing=0\).)

Lemma 2. Let \(R\) be an arbitrary complete regular event and
\[ \alpha_k=\sum p(\xi), \]
where \(\xi\) ranges over all \(k\)-words belonging to \(R\). Then there exist numbers \(\rho\) and \(r\) such that: a) \(0\le \rho\le 1\); b) \(r\) is a nonnegative integer; c) if \(\rho=0\), then \(\alpha_k=0\) for all sufficiently large \(k\); d) if \(\rho=1\), then
\[ \lim_{k\to\infty}\alpha_k>0; \]
e) if \(0<\rho<1\), then
\[ \alpha_k=a(k)k^r\rho^k+O(\rho^k)\quad\text{as }k\to\infty, \]
where \(a(k)\) is a periodic function taking positive values.

We shall briefly set forth, omitting details, the proof of item 4) of Theorem 1. Let
\[ q_n(k)=\sum p(\xi), \]
where \(\xi\) ranges over all input \(n\)-words \(\xi\) such that \(\tau(\xi,\sigma^{\#})>k\). It is easy to see that
\[ \varepsilon_n=\sum_{k=0}^{n-1}q_n(k). \]
Take an arbitrary input \(n\)-word \(\xi=x_1x_2\ldots x_n\). If \(\tau(\xi,\sigma^{\#})>k\), then, by Lemma 1, for some index \(i\), \(1\le i\le n-k\), the \((k+1)\)-word \(x_ix_{i+1}\ldots x_{i+k}\) belongs to the regular event \(R\). Hence it is not difficult to conclude that
\[ q_n(k)=P\{\tau(\xi,\sigma^{\#})>k\}\le (n-k)\alpha_{k+1}. \]
If, however, \(\tau(\xi,\sigma^{\#})\le k\), then for every \(i\), \(1\le i\le n-k\), the \((k+1)\)-word \(x_ix_{i+1}\ldots x_{i+k}\) does not belong to \(R\). Hence, using the independence of the random events
\[ \{x_ix_{i+1}\ldots x_{i+k}\notin R\} \quad\text{and}\quad \{x_jx_{j+1}\ldots x_{j+k}\notin R\} \]
for \(|i-j|>k\), we obtain
\[ P\{\tau(\xi,\sigma^{\#})\le k\}\le (1-\alpha_{k+1})^{[\,n/(k+1)\,]}. \]
Thus we have
\[ 1-(1-\alpha_{k+1})^{[\,n/(k+1)\,]}\le q_n(k)\le (n-k)\alpha_{k+1}. \tag{*} \]

Now define two functions \(k_0(n)\) and \(k^0(n)\) by putting
\[ k_0(n)=\min\left\{k:\left[\frac{n}{k+1}\right]\alpha_{k+1}<1\right\} \]
\[ k^0(n)=\min\left\{k:(n-k)\alpha_{k+1}<1\right\}. \]

It is easy to show, using item d) of Lemma 2, that as \(n \to \infty\)

\[ k_0(n)=\left(1/\log \frac{1}{\rho}\right)\{\log n+(r-1)\log\log n\}+O(1), \]

\[ k^0(n)=\left(1/\log \frac{1}{\rho}\right)\{\log n+r\log\log n\}+O(1). \]

From inequality \((*)\) we obtain

\[ k_0(n)-\sum_{k=0}^{k_0(n)-1}(1-\alpha_{k+1})^{\left[\frac{n}{k+1}\right]} +\sum_{k=k_0(n)}^{n-1}\left\{1-(1-\alpha_{k+1})^{\left[\frac{n}{k+1}\right]}\right\}\le \]

\[ \le \varepsilon_n \le k^0(n)+\sum_{k=k^0(n)}^{n-1}(n-k)\alpha_{k+1}. \]

The proof is completed by establishing that the expressions

\[ \sum_{k=0}^{k_0(n)-1}(1-\alpha_{k+1})^{\left[\frac{n}{k+1}\right]},\qquad \sum_{k=k_0(n)}^{n-1}\left\{1-(1-\alpha_{k+1})^{\left[\frac{n}{k+1}\right]}\right\},\qquad \sum_{k=k^0(n)}^{n-1}(n-k)\alpha_{k+1} \]

are bounded functions of \(n\).

We now show how the number \(\rho\) can be computed. Consider the directed graph \(G\), whose set of vertices consists of unordered pairs \(\{s,s'\}\), \(s\ne s'\) \((s,s'\in S)\), and whose vertex adjacency mapping \(\Gamma\) is defined as follows: \(\{s'',s'''\}\in\Gamma\{s,s'\}\) if and only if there exists a letter \(x\in X\) such that either \(f(x,s)=s''\) and \(f(x,s')=s'''\), or \(f(x,s)=s'''\) and \(f(x,s')=s''\). Let \(Z\) be the set of vertices of the graph \(G\) having the form \(\{s^\#, f(x,s^\#)\}\), where \(x\) is an arbitrary input letter. If the set \(Z\) is empty, then \(f(x,s^\#)=s^\#\) for all \(x\in X\). It is easy to see that then \(\tau(\xi,\sigma^\#)=0\) for any input word \(\xi\), and \(\rho\), of course, is equal to zero.

Let \(Z\ne\varnothing\), and let \(H=(E,\Gamma)\) be the subgraph of the graph \(G\) consisting of exactly those vertices of the graph \(G\) that are reachable by directed paths from the vertices of the set \(Z\). The graph \(H\) may be regarded as a weighted graph if to each of its arcs \((e,e')\), \(e=\{s,s'\}\), \(e'=\{s'',s'''\}\), we assign the number \(\nu(e,e')=\sum p_x\), where the summation is over all letters \(x\in X\) such that \(f(x,s)=s''\) and \(f(x,s')=s'''\), or \(f(x,s)=s'''\) and \(f(x,s')=s''\).

Let \(M\) be the adjacency matrix of the weighted graph \(H\), i.e. the matrix whose rows and columns are in one-to-one correspondence with the vertices of the graph; moreover, at the intersection of the row associated with the vertex \(e\) and the column associated with the vertex \(e'\) there stands \(0\) if \(e'\notin\Gamma e\), and \(\nu(e,e')\) if \(e'\in\Gamma e\). The number \(\rho\) is equal to the maximum of the moduli of the characteristic roots of the matrix \(M\).

Since it is important to distinguish the cases \(\rho=0\), \(\rho=1\), and \(0<\rho<1\), it is useful to have criteria for \(\rho=0\) or \(\rho=1\) that are simpler than criteria based on computing the characteristic roots of the matrix.

Theorem 2. \(\rho=0\) if and only if the graph \(H\) contains no contours (directed cycles).

Theorem 3. \(\rho=1\) if and only if the graph \(H\) contains a maximal strongly connected subgraph such that for every vertex \(\{s,s'\}\) of this subgraph and every input letter \(x\) one has \(f(x,s)\ne f(x,s')\).

In conclusion, let us consider estimates of the mean computation time on \(n\)-nets of iterative systems realizing the two mappings indicated.

A. Addition of two \(p\)-ary numbers. The alphabet \(X\) consists of \(p^2\) letters corresponding to pairs of integers \(\langle\alpha,\beta\rangle\), \(0\le\alpha,\beta\le p-1\). The alphabet \(Y\) consists of \(p\) letters corresponding to integers \(\gamma\), \(0\le\gamma\le\)

\(\leq p-1\). Let \(\xi=x_1x_2\ldots x_n,\ x_i=\langle \alpha_i,\beta_i\rangle\) be an arbitrary input \(n\)-word, and
\[ (\alpha_1+p\alpha_2+\ldots+p^{n-1}\alpha_n)+(\beta_1+p\beta_2+\ldots+p^{n-1}\beta_n) = \gamma_1+p\gamma_2+\ldots+p^{n-1}\gamma_n+p^n\gamma_{n+1}. \]
Put \(\varphi_1(\xi)=\gamma_1\gamma_2\ldots\gamma_n\). The mapping \(\varphi_1\) is realized by an iterative system \(\mathcal S_1\), in which the alphabet \(S\) consists of two letters corresponding to the numbers 0 and 1, \(s^\#=0\),
\[ f(\langle\alpha,\beta\rangle,\delta)=\left[\frac{\alpha+\beta+\delta}{p}\right] \]
and
\[ g(\langle\alpha,\beta\rangle,\delta)=\alpha+\beta+\delta \pmod p \qquad (0\leq \alpha,\beta\leq p-1,\ \delta=0,1). \]
Obviously, the graph \(H\) for \(\mathcal S_1\) consists of one vertex \(e=\{0,1\}\) and a loop \((e,e)\). Further, it is easy to find that
\[ v(e,e)=\sum_{\alpha+\beta=p-1} p_{\alpha\beta}, \]
where \(p_{\alpha\beta}\) is the probability of occurrence at the input of the pair \(\langle\alpha,\beta\rangle\). Hence it follows that
\[ \rho=\sum_{\alpha+\beta=p-1} p_{\alpha\beta}. \]
In particular, if all pairs \(\langle\alpha,\beta\rangle\) occur with equal probabilities, then \(\rho=1/p\), and, consequently, \(\varepsilon_n\sim \log_p n\).

B. Comparison of two \(p\)-ary numbers. The alphabet \(X\) is the same as in Example A. The alphabet \(Y\) consists of three letters \(a,b,c\). Let \(\xi=x_1x_2\ldots x_n,\ x_i=\langle\alpha_i,\beta_i\rangle\) be an arbitrary input \(n\)-word. Put \(\varphi_2(\xi)=y_1y_2\ldots y_n\), where \(y_i=a\) if the two numbers
\[ u_i=\alpha_1+p\alpha_2+\ldots+p^{i-1}\alpha_i \quad\text{and}\quad v_i=\beta_1+p\beta_2+\ldots+p^{i-1}\beta_i \]
coincide with one another; \(y_i=b\) if \(u_i>v_i\); \(y_i=c\) if \(u_i<v_i\). The mapping \(\varphi_2\) is realized by an iterative system \(\mathcal S_2\), for which \(S=Y,\ s^\#=a\), and the functions \(f\) and \(g\) are defined as follows:
\[ f(\langle\alpha,\beta\rangle,a)= \begin{cases} a, & \text{if } \alpha=\beta,\\ b, & \text{if } \alpha>\beta,\\ c, & \text{if } \alpha<\beta; \end{cases} \qquad f(\langle\alpha,\beta\rangle,b)= \begin{cases} b, & \text{if } \alpha\geq\beta,\\ c, & \text{if } \alpha<\beta; \end{cases} \]
\[ f(\langle\alpha,\beta\rangle,c)= \begin{cases} b, & \text{if } \alpha>\beta,\\ c, & \text{if } \alpha\leq\beta; \end{cases} \qquad g(\langle\alpha,\beta\rangle,y)=y. \]
Just as in Example A, the graph \(H\) for \(\mathcal S_2\) consists of one vertex \(e=\{a,b\}\) and a loop \((e,e)\). In this case
\[ v(e,e)=\sum_{\alpha=\beta} p_{\alpha\beta}. \]
Thus,
\[ \rho=\sum_{\alpha=\beta} p_{\alpha\beta}. \]
In particular, if the pairs \(\langle\alpha,\beta\rangle\) are equiprobable, then \(\rho=1/p\), and, consequently, \(\varepsilon_n\sim \log_p n\).

Remark 2. The \(n\)-nets of the iterative system \(\mathcal S_1\) for \(p=2\) are known in computer engineering under the name of \(n\)-digit adders of parallel action. If the adder is equipped with a device for signaling a stable state, then the time for adding two numbers coincides with the computation time on the \(n\)-net of the iterative system \(\mathcal S_1\). Von Neumann showed that for any \(n\), \(\varepsilon_n<\log_2 n\) \({}^{(2)}\). The fact that \(\log_2 n\) is an asymptotic estimate for \(\varepsilon_n\) was shown in \({}^{(3)}\).

Institute of Mathematics
Siberian Branch of the Academy of Sciences of the USSR

Received
14 XII 1965

CITED LITERATURE

\({}^{1}\) F. C. Hennie, Iterative Arrays of Logical Circuits, 1961.

\({}^{2}\) J. v. Neumann, A. W. Burks, H. Goldstine, Preliminary Discussion of the Logical Design of an Electronic Computing Instrument, In: J. v. Neumann, Collected Works, 5, 1963, p. 45.

\({}^{3}\) V. V. Shkurba, N. Z. Shor, Proceedings of the All-Union Conference on Probability Theory and Mathematical Statistics, 1962, p. 269.