CYBERNETICS AND CONTROL THEORY

Ya. M. BARZDIN’

THE CAPACITY OF A MEDIUM AND THE BEHAVIOR OF AUTOMATA

(Presented by Academician P. S. Novikov on 29 VI 1964)

There exist various concepts of automata, for example, logical networks \((^1)\), Turing machines, the Kolmogorov–Uspenskii algorithm \((^2)\), Neumann–Church automata \((^{3,4})\), growing automata \((^5)\), etc. What they have in common is that they consist of individual elements that are connected with one another in a definite way. Automata, both within the framework of a single concept and within the framework of different concepts, may differ from one another by various parameters. One of the tasks of automata theory is to determine which parameters can be compensated for by others, and at what cost. In the present note we investigate the influence of the topology of automata on their capabilities*. For this purpose the notions of an automaton’s medium and its capacity are introduced, as well as the notions of modeling and strong modeling (see below). In the first part of the note, the possibilities of modeling automata are clarified as a function of the capacity of the medium. In particular, in this way one partially succeeds in estimating the comparative capabilities of the K.–U. (Kolmogorov–Uspenskii) algorithms and the N.–Ch. (Neumann–Church) automata. The second part of the note is devoted to strong modeling. Theorems are given which show how the number of elements increases and the speed of computation decreases under strong modeling of automata with arbitrary topology by N.–Ch. automata. We note that N.–Ch. automata are automata with a very simple topology. The simplicity of the topology substantially facilitates their technical realization (see, for example, \((^8)\), where by a computing medium is meant an automaton very close to an N.–Ch. automaton).

1°. Automata and their behavior. In the present note, by automata we shall mean a certain natural generalization of logical networks built from elements with delays. We give their exact definition. An element is an object that has states \(\Lambda, q_1, \ldots, q_N\), input channels \(R_1, \ldots, R_M\), and one output channel. The input and output channels use the alphabet \(Q=\langle \Lambda, q_1, \ldots, q_N\rangle\), and the state of the output channel always coincides with the state of the element itself. The operation of an element is specified by a function \(\varphi(a_0, a_1, \ldots, a_M)\): if at the moment \(t=0,1,2,\ldots\) the element is in state \(a_0\) and the letter \(a_i\) is fed along channel \(R_i\), then at the moment \(t+1\) it passes into the state \(\varphi(a_0, a_1, \ldots, a_M)\). It is assumed that \(\varphi(\Lambda,\Lambda,\ldots,\Lambda)=\Lambda\). Let \(A\) be some element. A network over \(A\) is any collection (in general infinite) of elements \(A\), for which it is specified which input channels are joined to which output channels. It is assumed here that each input channel may be joined to no more than one output channel. Those input channels that are not joined to any output channels will be called free. We single out some finite sequence of free input channels and declare them to be the input channels of the given network. Along the input channels of the network, information is supplied from outside in the form of letters (all—

* As applied to logical networks, some related questions were considered by Lupanov \((^6)\) and Holland \((^7)\).

possible) from \(Q\), while the letter \(\Lambda\) is always supplied through the remaining free input channels. We shall also single out some finite sequence of output channels and declare them to be the output channels of the given network. Then we shall call such a network an automaton over \(A\).

We now define the behavior of an automaton. The behavior of an automaton is defined under the assumption that some initial state of it has been fixed (at \(t=0\)), and moreover in such a way that only a finite number of elements have states \(\ne \Lambda\).* The feeding of any input sequence
\(x(0), x(1), \ldots, x(t), \ldots\) to the “glued” input channel of the automaton \(\mathfrak A\) gives rise, on its “glued” output channel, to a completely determined output sequence
\(z(0), z(1), \ldots, z(t), \ldots\), where \(x(t)\) (\(z(t)\)) is a tuple of letters of the input (output) alphabet of the automaton, which at time \(t\) is fed to (issued on) the “glued” input (output) channel of the automaton. The operator induced in this way will be called the behavior of the automaton \(\mathfrak A\).

\(2^\circ\). Modeling. We shall say that an automaton \(\mathfrak A\) can be modeled (modeled with a shift, modeled with a stretching) by an automaton \(\mathfrak B\), if the behavior of \(\mathfrak A\) (some \(\tau\)-shift of the behavior of \(\mathfrak A\), some \(\sigma\)-stretching of the behavior of \(\mathfrak A\)) can be realized in \(\mathfrak B\) (for the definitions of a \(\tau\)-shift and a \(\sigma\)-stretching see (1), p. 128; the definition of realizability of an operator \(\theta\) in an automaton \(\mathfrak B\) is analogous to the definition of realizability of \(\theta\) in a network \(\mathfrak B(a)\) in (5), only in our case the feeding of letters begins at \(t=0\)).

\(3^\circ\). Strong modeling. Let there be given an automaton \(\mathfrak A\) over \(A\), with elements
\(\alpha_1,\ldots,\alpha_n\) and input channels
\(\xi_1,\ldots,\xi_m\), and an automaton \(\mathfrak B\) over \(B\), with elements
\(\beta_1,\ldots,\beta_n,\ldots,\beta_{n'}\) and input channels
\(\eta_1,\ldots,\eta_m,\ldots,\eta_{m'}\). With respect to the state alphabets of the element \(A\) and the element \(B\) we shall assume that the second includes the first. Denote by
\(\alpha_i(t)\), \(\beta_i(t)\), \(\xi_j(t)\), and \(\eta_j(t)\), respectively, the states of the elements \(\alpha_i,\beta_i\) and of the input channels \(\xi_j,\eta_j\) at time \(t\). Suppose the following condition is satisfied: there exist such initial states of the elements
\(\beta_{n+1},\ldots,\beta_{n'}\) that, whatever the initial states \(\alpha_i(0)\) of the elements \(\alpha_i\) may be, if only
\(\beta_i(0)=\alpha_i(0)\) and, for all
\(t=0,1,2,\ldots\), \(\eta_j(Dt)=\xi_j(t)\), then
\(\beta_i(Dt)=\alpha_i(t)\)
\((i=1,\ldots,n;\ j=1,\ldots,m)\). In this case we shall say that the automaton \(\mathfrak A\) can be strongly modeled with stretching \(D\) by the automaton \(\mathfrak B\).

\(4^\circ\). Environment and its capacity. To each automaton \(\mathfrak A\) over \(A\) there corresponds a directed graph whose vertices are the elements of the automaton and whose edges are the nonfree input channels of the elements. This graph is characterized by the property that there is a number \(M\) such that no more than \(M\) edges enter each vertex. Directed graphs (in general infinite) possessing this property will be called environments. An environment will be called symmetric if, whenever an edge \((\alpha,\beta)\) exists, the edge \((\beta,\alpha)\) also exists. Let \(G\) be an environment and let \(\alpha \in G\) be a vertex. Define the functions \(f_\alpha^G\) and \(f^G\): \(f_\alpha^G(r)\) is the number of those vertices that are at distance \(\le r\) from \(\alpha\) (the distance of the vertex \(\beta\) from \(\alpha\) is the least number of edges along which one can get from \(\beta\) to \(\alpha\), going in the direction of the edges); \(f^G(r)=\max f_\alpha^G(r)\) (the maximum is taken over all \(\alpha \in G\)). The function \(f^G(r)\) will be called the capacity of the environment \(G\). It is easy to see that
\(f_\alpha^G(r)\le f^G(r)\le M^0+M^1+\cdots+M^r\).

\(5^\circ\). We shall say that an automaton \(\mathfrak A\) is embedded in an environment \(G\) if the graph corresponding to it is a subgraph of the environment \(G\).

As an automaton embeddable in an environment with degree capacity, let us consider the N—C automaton (Neumann—Church). The automaton \(\mathfrak A\) over \(A\) (where \(A\)

* If an element \(\alpha\) of the automaton \(\mathfrak A\) is in the state \(\Lambda\), this should be interpreted as its absence. If at time \(t\) \(\alpha \ne\) was in the state \(\Lambda\), and at time \(t+1\) passed into a state \(\ne \Lambda\), this should be interpreted as the generation of the given element. Thus, on our automata one can interpret growing automata.

has input channels \(R_1, \ldots, R_{2k}\)) is called a \(k\)-dimensional N.–Ch. automaton over \(A\) if it has the following properties: 1) its elements are placed at points of a \(k\)-dimensional integer lattice (at each point no more than one element); 2) suppose the element \(\alpha\) is at the point \((v_1, \ldots, v_k)\), and suppose the points \((v_1+1, v_2, \ldots, v_k), \ldots, (v_1, \ldots, v_{k-1}, v_k+1)\), \((v_1-1, v_2, \ldots, v_k), \ldots, (v_1, \ldots, v_{k-1}, v_k-1)\) are denoted respectively by \(C_1, \ldots, C_{2k}\); then the input channel \(R_i\) of the element \(\alpha\) is connected to the output channel of the element \(\beta\) if and only if \(\beta\) is located at the point \(C_i\).

Fig. 1

In Fig. 1 a 2-dimensional N.–Ch. automaton is shown, with input channels \(P_1, P_2\) and output channels \(S_1, S_2\).

\(6^\circ\). We shall say that a medium \(G'\) is \(l\) times less capacious than a medium \(G\) if \(f_\alpha^{G'}(3lr) < f_\alpha^G(r)\).

Theorem 1. Let \(G\) be an arbitrary symmetric medium; then there exists an automaton that can be embedded in \(G\) and that cannot be modeled with stretch \(l\) by any automaton embeddable in a medium \(G'\), \(l\) times less capacious than the medium \(G\).

From this theorem it follows that: 1) there exists an automaton that cannot be modeled with stretch by any N.–Ch. automaton; 2) for every \(k \geq 2\) there exists a \(k\)-dimensional N.–Ch. automaton that cannot be modeled with stretch by any \(k'\)-dimensional N.–Ch. automaton, if \(k' < k\).

We shall say that an automaton \(\mathfrak A\) operates in a focal manner if there exists an \(r\) such that at each moment no more than \(r\) elements change their state. Theorem 1 in some cases admits strengthening. In particular, the following holds.

Theorem 2. Let \(G\) be an arbitrary symmetric medium with exponential capacity; then there exists an automaton \(\mathfrak A\), operating in a focal manner and embeddable in \(G\), that cannot be modeled with stretch by any automaton embeddable in any medium with polynomial capacity.

This theorem shows that the capacity of a medium, generally speaking, cannot be compensated by the universality of the operation of the elements. Slightly generalizing the K.–U. algorithm, one can introduce for it the notions of inputs and outputs, and then, using Theorem 2, it is easy to show that such an automaton in general cannot be modeled by any N.–Ch. automaton.

\(7^\circ\). Universal media. This point is partly connected with the work \((^9)\) and in a certain sense is its continuation. We shall consider boundedly deterministic operators (b.d. operators) and automata constructed from a finite number of elements (for the definition of b.d. operators see \((^1)\), p. 98). Without essential restriction of generality, we shall consider only b.d. operators with two-letter input and output alphabets. We denote the class of all such b.d. operators by \(\mathscr E\). Suppose a medium \(G\) has the following property: there exists an element \(A\) such that for any operator \(\theta \in \mathscr E\) one can construct an automaton \(\mathfrak A\) over \(A\), consisting of a finite number of elements, having one input and one output channel, embeddable in the medium \(G\), and realizing some \(\tau\)-shift (\(\sigma\)-stretch) of the operator \(\theta\). Then we shall say that the medium \(G\) is universal up to shift (universal up to stretch).

In Fig. 2 a medium is shown that has a very simple topology and linear capacity.

Theorem 3. The medium shown in Fig. 2 is universal with accuracy up to a shift.

Theorem 4. A medium \(G\) is universal with accuracy up to a stretching if and only if it contains arbitrarily long cycles.

§0. Strong simulation of automata with arbitrary topology by \(k\)-dimensional Neumann–Church automata. Fix an arbitrary element \(A\). Along with \(A\), we shall consider elements \(B\). For simplicity we shall assume that all \(B\) are such that, for every automaton \(\mathfrak A\) over \(A\), there is a \(k\)-dimensional Neumann–Church automaton over \(B\) that strongly simulates the former.

Fig. 2

Introduce the following functions: \(L_B^{(k)}(\mathfrak A)\) is the least number of elements of \(k\)-dimensional Neumann–Church automata over \(B\) that strongly simulate the automaton \(\mathfrak A\); \(D_B^{(k)}(\mathfrak A)\) is the least stretching with which \(\mathfrak A\) can be strongly simulated on some \(k\)-dimensional Neumann–Church automaton over \(B\); \(L_B^{(k)}(n)=\max L_B^{(k)}(\mathfrak A)\) and \(D_B^{(k)}(n)=\max D_B^{(k)}(\mathfrak A)\) (the maximum is taken over all automata \(\mathfrak A\) over \(A\) with number of elements \(\le n\)).

\[ L_B^{(k)}(n) \succcurlyeq n\log n,\qquad D_B^{(k)}(n) \succcurlyeq \frac{\sqrt[k]{\,n\log n\,}}{\log n}. \]

Theorem 6. There exists an element \(B\) such that

\[ L_B^{(k)}(n) \preccurlyeq n\log n,\qquad D_B^{(k)}(n) \preccurlyeq \sqrt[k]{\,n\log n\,}, \]

and both these estimates are attained simultaneously.

I express my gratitude to B. A. Trakhtenbrot for his attention to the present work and for valuable advice.

Institute of Mathematics
Siberian Branch of the Academy of Sciences of the USSR

Received
29 VI 1964

REFERENCES

1. N. E. Kobrinskii, B. A. Trakhtenbrot, Introduction to the Theory of Finite Automata, Moscow, 1962.
2. A. N. Kolmogorov, V. A. Uspenskii, Uspekhi Mat. Nauk, 13, 4 (1958).
3. E. F. Moore, Proc. Symposia in Appl. Math., 14 (1962).
4. A. Church, Summer Institute for Symbolic Logic, Cornell University (Summaries), 1957, Second Ed., 1960, p. 3.
5. Ya. M. Barzdin, DAN, 157, No. 3 (1964).
6. O. B. Lupanov, Problems of Cybernetics, vol. 7, 61 (1962).
7. J. H. Holland, J. Franklin Inst., 270, No. 3, 202 (1960).
8. E. V. Evreinov, Computing Systems, vol. 4, Novosibirsk, 1962.
9. M. L. Minskii, in: Automata, IL, 1956, pp. 163–178.