Reports of the Academy of Sciences of the USSR
1963. Volume 148, No. 6

CYBERNETICS AND CONTROL THEORY

I. D. KOCHUBIEVSKII, G. M. ULANOV

INFORMATION CONDITIONS FOR THE INVARIANCE OF LINEAR AUTOMATIC CONTROL SYSTEMS

(Presented by Academician B. N. Petrov on VII 23, 1962)

The theory of invariance, developed up to the present time in the works of V. S. Kulebakin \((^1)\), N. N. Luzin \((^4)\), B. N. Petrov \((^2)\), A. Yu. Ishlinskii \((^5)\), and others, corresponds to the case of an action \(\xi_x(x,t)\) defined as a deterministic function of time \(t\). For a given, in the probabilistic sense, \(\xi_x(x,t)\), the effect of invariance was investigated by A. G. Ivakhnenko \((^8)\) and A. G. Shevelev \((^3)\), without consideration of information processes.

Fig. 1

At the same time, the theory of invariance—absolute and up to \(\varepsilon\)—depends essentially on the information characteristics of the ensemble of actions \(\xi_x(x,t)\). In the present note an attempt is made to apply information theory and statistical optimization to the determination of invariance conditions \((^6,^7,^9,^10)\).

Let us consider the information conditions for invariance of automatic control systems (ACS). Let a process \(\xi_x(x,t)\) be given, taking values on a bounded set \(x\) with metric \(\rho(x)=(x_1-x_2)\), which must be reproduced by the automatic system under the condition of an interference in the form of an action \(\xi_f(f,t)\) (\(f \in F\) bounded), applied at another point (Fig. 1). \(\xi'_{x'}(x',t)\) is the process at the output of the automatic control system, taking values on a bounded set with the same metric. The consideration of the majority of automatic systems can be reduced to this form if a preliminary functional transformation of the control signal \((\xi)\) is required.

In the case of realization of the condition of absolute invariance,

\[ x^i \equiv x'^i, \tag{1} \]

where \(i\) is the number of a realization of the process of the ensemble \(\xi\).

With such a general formulation one may regard the ACS as a system for transmitting information. Since the actions are applied to the system at different points, then, by virtue of its linearity, the transmission of each action may be considered separately. Then, considering only the transmission of the control action without taking interference into account, for the ACS the information-transmission equation \((^{9-12})\) has the form

\[ \mathcal{H}'_2(\xi')=\mathcal{H}'_1(\xi)+\frac{1}{2W}\int_W \log |\Phi(j\omega)|^2 d\omega, \tag{2} \]

where \(W\) is the frequency band of the ensemble \(\xi\); \(\mathcal{H}'_1(\xi)\) is the entropy of the ensemble \(\xi\) per degree of freedom,

\[ \mathcal{H}'_1(\xi)=-\lim_{n\to\infty}\frac{1}{n}\int \cdots \int_{\underbrace{\scriptstyle n}} p(x_1,x_2,\ldots,x_n)\log p(x_1,x_2,\ldots,x_n)\,dx_1 dx_n; \tag{3} \]

\(p(x_1,x_2,\ldots,x_n)\) is the multidimensional distribution density; \(\mathcal{H}'_2(\xi')\) is the entropy per degree of freedom of the signal at the output; \(\Phi(j\omega)\) is the frequency-

characteristic of the filter, in our case the frequency characteristic of the closed automatic control system.

From the conditions of absolute invariance (1), as a consequence, the nondistorting information is equal to

\[ \mathcal{H}'_1(\xi)=\mathcal{H}'_2(\xi'); \tag{4} \]

\[ \int_W \log |\Phi(j\omega)|\,d\omega=0. \tag{5} \]

Expression (5) may be regarded as an informational condition of absolute invariance, taking into account the frequency band of the ensemble of signals. A more stringent requirement, considered in the theory of invariance, consists in the fact that the frequency \(W\) must be infinite.

From the condition of absolute invariance it follows that the frequency characteristic of the automatic control system must possess the property

\[ |\Phi(j\omega)|=1 \quad \text{for } 0<\omega<W. \tag{6} \]

The frequency characteristic of an absolutely invariant automatic control system must have a rectangular form. But, as shown in \((^{10})\), a filter with a rectangular frequency characteristic is physically unrealizable, since it does not satisfy the Paley—Wiener condition (i.e., the integral

\[ \int_0^\infty \frac{\log |\Phi(\omega)|}{1+\omega^2}\,d\omega \]

is a finite number). This proposition shows that in a single-loop system the conditions of absolute invariance in the general case, for an arbitrary signal, cannot be attained even when the signal frequency band is limited.

For a two-channel system the frequency characteristic \(\Phi\) can be represented in the form of the sum of two characteristics: \(\Phi_n(j\omega)\), the transfer function of the closed loop, and \(\Phi_k(j\omega)\), the transfer function of the correcting circuit; then condition (5) can be represented in the form

\[ |\Phi_n(j\omega)+\Phi_k(j\omega)|=1 \quad \text{for } 0\leq \omega \leq W. \tag{7} \]

This condition does not impose stringent rectangularity requirements on each frequency characteristic \(\Phi_n\) and \(\Phi_k\), and therefore it is possible to find a physically realizable system satisfying this condition. Condition (7) may be represented in a stronger form

\[ \Phi_n(j\omega)+\Phi_k(j\omega)=1, \tag{8} \]

which will impose additional requirements on the phase-frequency characteristics. Hence it follows that, in order to satisfy the conditions of absolute invariance, the frequency characteristic of the second channel must be

\[ \Phi_k(j\omega)=1-\Phi_n(j\omega). \tag{9} \]

The right-hand side of (9) represents the expression for the frequency characteristic of the error of the closed system, i.e., the error of the channel

\[ \Phi_{k\delta}(j\omega)=1-\Phi_n(j\omega). \tag{10} \]

Consequently, the frequency characteristic of the second channel must be the frequency characteristic of the error of the first channel,

\[ \Phi_k(j\omega)=\Phi_{k\delta}(j\omega). \tag{11} \]

This means that information is transmitted through the first main channel, and error correction through the second. Obviously, one can select two physically realizable filters with frequency characteristics \(|\Phi_n|\) and \(|\Phi_k|\) and obtain the conditions of absolute invariance.

Consequently, in reproducing systems the conditions of absolute invariance for an arbitrary signal of general form are realizable if and only if there are two channels for information transmission satisfying the conditions set forth above. This is a confirmation of the well-known theorem of B. N. Petrov \((^2)\).

Let us consider, in this same system, the conditions for absolute invariance under stabilization with respect to the action \(\xi_f(f,t)\). From the point of view of information theory, the latter means that at the output of the system no information about the application of the disturbance should be observed, which can be written in the form

\[ 0 \equiv \mathcal{H}'_f(\xi_f) + \frac{1}{2W}\int\limits_W \log |v(j\omega)|^2\,d\omega, \tag{12} \]

where \(v(j\omega)\) is the frequency characteristic of the system (closed-loop) with respect to the action \(\xi_f(f,t)\).

Hence follows the condition of absolute invariance for the stabilization system under an arbitrary signal of general form; consequently, for any \(\mathcal{H}\) it must be

\[ |\mathcal{H}'_f(\xi_f)| = \left|\frac{1}{2W}\int\limits_W \log |v(j\omega)|^2 d\omega\right|. \]

If deterministic signals \(f(t)\) act on the system, then from the point of view of information theory such signals carry no information, since their probability distribution density is zero. Such information is known in advance. However, for technical automatic-control systems such a problem is widespread. In this case the correction is carried out by choosing \(\Phi_{\mathrm{opt}}\), the operator of the automatic-control system (the case \(\Phi_{\mathrm{opt}}\ne 0\)) in accordance with the known deterministic action, and is achieved with the aid of the \(K(D)\)-image (the third form of invariance), so that \(K(D)f(t)=0\), \(K(D)\ne 0\), \(f(t)\ne 0\) \({}^{(1)}\). If one considers stationary processes and systems with infinite memory and takes the observation interval to be infinite, the Wiener-optimal transfer function of the automatic-control system \(\Phi_{\mathrm{opt}}\) is characterized by an error \(\bar{\varepsilon}^2\) equal to

\[ \varepsilon^2 = \frac{1}{2\pi}\int\limits_{-\infty}^{\infty}\left\{S_h(\omega)-|\Phi_{\mathrm{opt}}(j\omega)|^2 S_f(\omega)\right\}d\omega, \]

where \(S_f(\omega)\) is the spectral density of \(f(t)\), and \(S_h(\omega)\) is the spectral density of the desired signal.

In control problems with optimal filtering, \(\bar{\varepsilon}^2=0\), \(S_h(\omega)=0\); then

\[ |\Phi_{\mathrm{opt}}(j\omega)|^2 S_f(\omega)=0. \]

If \(\Phi(D)\ne 0\), \(f(t)\ne 0\), and the variance of \(f(t)\) is zero (the deterministic case), the latter is possible in the case

\[ \Phi_{\mathrm{opt}}(D)f(t)=0, \]

i.e., \(\Phi_{\mathrm{opt}}(D)\) must be the \(K(D)\)-image of \(f(t)\), or contain it as a factor. Thus, in this case the Wiener-optimal system satisfies the optimal solution obtained on the basis of invariance theory.

Received
1 VII 1962

CITED LITERATURE

\({}^{1}\) V. S. Kulebakin, Transactions of the First International Congress of IFAC, 1, Publishing House of the Academy of Sciences of the USSR, 1961.
\({}^{2}\) B. N. Petrov, ibid.
\({}^{3}\) A. G. Shevelev, ibid.
\({}^{4}\) N. N. Luzin, P. I. Kuznetsov, DAN, 51, No. 4 (1946).
\({}^{5}\) A. Yu. Ishlinskii, Proceedings of the Conference “Theory of Invariance and Its Application in Automatic Devices”, Academy of Sciences of the Ukrainian SSR, 1959.
\({}^{6}\) V. S. Kulebakin, ibid.
\({}^{7}\) B. N. Petrov, ibid.
\({}^{8}\) A. G. Ivakhnenko, Avtomatika, No. 3 (1957).
\({}^{9}\) K. Shannon, “Statistical Theory of the Transmission of Electrical Signals,” in: Theory of the Transmission of Electrical Signals in the Presence of Noise, IL, 1953, p. 59.
\({}^{10}\) S. Goldman, Information Theory, IL, 1957, pp. 170, 212.
\({}^{11}\) A. N. Kolmogorov, Theory of Information Transmission, Session of the Academy of Sciences of the USSR on Scientific Problems of Automation of Production in 1957, Moscow, 1957.
\({}^{12}\) A. N. Kolmogorov, A. N. Tikhonov, UMN, 14, issue 2 (86) (1959).