Full Text
Reports of the Academy of Sciences of the USSR
1967. Volume 172, No. 6
UDC 62-50:517.948.35
CYBERNETICS AND CONTROL THEORY
V. V. SOLODNIKOV, V. L. LENSKII
REGULARIZATION OF PROBLEMS IN THE STATISTICAL DYNAMICS OF AUTOMATIC CONTROL SYSTEMS
(Presented by Academician A. A. Dorodnitsyn, 25 IV 1966)
In the theory of optimal automatic control systems, ill-posed problems often arise. The numerical solution of these problems, as well as the physical implementation of the optimal systems found, presents substantial difficulties. In the present paper it is shown that the application of the regularization methods developed in (¹, ²) to certain problems in the statistical dynamics of automatic control systems not only yields stable algorithms, but also reduces the complexity of the physical implementation of the optimal systems found.
One of the fundamental problems of the statistical dynamics of automatic control systems is the determination of a functional \(F[x(t_1);\, t-a \leq t_1 \leq t]\) that minimizes the functional
\[ \overline{\left|y(t)-F[x(t_1);\, t-a \leq t_1 \leq t]\right|^2}, \tag{1} \]
where \(x(t)\) is a random signal at the input of the filter specified by the functional \(F\); \(y(t)\) is the desired output signal of the filter; \(a\) is a parameter determining the memory of the filter (if \(a < \infty\), we shall say that the filter has finite memory; if \(a=\infty\), then we shall say that the filter has infinite memory); the bar denotes the averaging operation.
Let the solution of the variational problem for the functional (1) be sought in the class of polynomial functionals of the form
\[ F[x(t_1);\, t-a \leq t_1 \leq t] = \int_{0}^{a} h_1(p_1)x(t-p_1)\,dp_1+ \]
\[ +\int_{0}^{a}\int_{0}^{a} h_2(p_1,p_2)x(t-p_1)x(t-p_2)\,dp_1\,dp_2+\cdots \]
\[ \cdots+ \int_{0}^{a}\cdots\int_{0}^{a} h_s(p_1,\ldots,p_s)x(t-p_1)\cdots x(t-p_s)\,dp_1\cdots dp_s. \tag{2} \]
Then the system of functions \(h_1, h_2,\ldots,h_s\), corresponding to the optimal polynomial functional, satisfies a system of integral
equations of the first kind
\[ \sum_{i=1}^{s}\int_{0}^{a}\cdots\int_{0}^{a} h_i(p_1,\ldots,p_i)\, \Gamma_x(p_1,\ldots,p_i,q_1,\ldots,q_j)\,dp_1\cdots dp_i = \]
\[ = \Gamma_{yx}(q_1,\ldots,q_j),\quad j=1,2,\ldots,s; \]
\[ \Gamma_x(p_1,\ldots,p_i,q_1,\ldots,q_j) = \overline{x(t-p_1)\cdots x(t-p_i)\,x(t-q_1)\cdots x(t-q_j)}, \tag{3} \]
\[ \Gamma_{yx}(q_1,\ldots,q_j) = \overline{y(t)\,x(t-q_1)\cdots x(t-q_j)} \]
moments of the random signal \(x(t)\) and mixed moments of the random signals \(y(t)\) and \(x(t)\).
System (2) is equivalent to the functional equation
\[ \Gamma_x h = \Gamma_{yx}, \tag{4} \]
where \(h\) and \(\Gamma_{yx}\) are elements of the Hilbert space \(\mathcal H\), which is the direct sum of the spaces \(\mathcal H^1,\mathcal H^2,\ldots,\mathcal H^s\), and each \(\mathcal H^i\) is a space of functions of \(i\) variables, square-integrable on the \(i\)-dimensional hypercube with side \(a\).
It can be shown that the operator \(\Gamma_x\), acting in \(\mathcal H\), will be self-adjoint and positive. Let \(E_\lambda\) be the resolution of the identity of the operator \(\Gamma_x\). Then the solution of equation (4) can be represented in the form
\[ h=\int_{0}^{\infty}\frac{1}{\lambda}\,dE_\lambda \Gamma_{yx} \tag{5} \]
if and only if
\[ \int_{0}^{\infty}\frac{1}{\lambda^2}\,d\left(E_\lambda\Gamma_{yx},\Gamma_{yx}\right)<\infty. \tag{6} \]
Let the element \(\Gamma_{yx}\) be given with some error \(\eta\). In this case the solution of equation (4) will have the form
\[ \overline{h}=\int_{0}^{\infty}\frac{1}{\lambda}\,dE_\lambda(\Gamma_{yx}+\eta). \tag{7} \]
The norm of the error of the solution will obviously be determined by the formula
\[ \|h-\overline{h}\|= \left[ \int_{0}^{\infty}\frac{1}{\lambda^2}\,d(E_\lambda\eta,\eta) \right]^{1/2}. \tag{8} \]
It follows from this that the norm of the error of the solution may be arbitrarily large depending on the distribution of the spectral measure \((E_\lambda\eta,\eta)\). Thus, the problem of determining the optimal functional of the form (2) is not well posed. To obtain stable algorithms, one may apply the methods of regularization of ill-posed problems proposed in works \((^{1,2})\). If the regularizing functional is given in the form \(\Omega(h)=(h,h)\), then regularization of problem (4) leads to the equation of the second kind
\[ \mu h+\Gamma_x h=\Gamma_{yx}. \tag{9} \]
One of the possible regularizing algorithms for problem (4) may be the following:
\[ R_{\delta}\,[\Gamma_{yx}]=\int_{\alpha}^{\infty}\frac{1}{\lambda}\,dE_{\lambda}\Gamma_{yx}. \tag{10} \]
If the element \(\Gamma_{yx}\) is given with error \(\eta\), then
\[ \begin{aligned} \|h-\bar h\| &=\left\|\int_{0}^{\infty}\frac{1}{\lambda}\,dE_{\lambda}\Gamma_{yx} -\int_{\alpha}^{\infty}\frac{1}{\lambda}\,dE_{\lambda}(\Gamma_{yx}+\eta)\right\| \leq \\ &\leq \left\|\int_{0}^{\alpha}\frac{1}{\lambda}\,dE_{\lambda}\Gamma_{yx}\right\| +\left\|\int_{\alpha}^{\infty}\frac{1}{\lambda}\,dE_{\lambda}\eta\right\| \leq \left[\int_{0}^{\alpha}\frac{1}{\lambda^{2}}\,d(E_{\lambda}\Gamma_{yx},\Gamma_{yx})\right]^{1/2} +\frac{1}{\alpha}\|\eta\|. \end{aligned} \tag{11} \]
It follows from this:
Theorem. For any \(\varepsilon>0\) there exist such \(\delta(\varepsilon)\) and \(\alpha(\varepsilon)\) that, if \(\|\eta\|<\delta(\varepsilon)\), then \(\|h-\bar h\|\leq \varepsilon\).
In the case of a discrete spectrum, the regularizing algorithm (10) is equivalent to defining the solution in the form of a linear combination of eigenvectors whose eigenvalues are greater than \(\alpha\).
Regularization of ill-posed problems of the statistical dynamics of automatic control systems is expedient not only because it yields stable algorithms, but also because regularization reduces the complexity of the physical realization of optimal filters.
Indeed, regularization of problem (4) is equivalent to the variational problem of determining the element \(h\) that minimizes the functional \(\Omega\) for a fixed value of the functional (1). Minimization of the functional \(\Omega\) is equivalent to narrowing the set \(A_t=\{h\mid \Omega(h)\leq t\}\). Let us define the functional \(C_{\varepsilon}(h)\)—the cost of the physical realization of a filter specified by the element \(h\), with accuracy \(\varepsilon\). As a characteristic of the complexity of the physical realization of a filter specified by an element \(h\in A_t\), with accuracy \(\varepsilon\), one may take the number
\[ \max_{h\in A_t} C_{\varepsilon}(h). \]
It is obvious that
\[ \max_{h\in A_{t_1}} C_{\varepsilon}(h) \leq \max_{h\in A_{t_2}} C_{\varepsilon}(h), \quad \text{if } A_{t_1}\subset A_{t_2}. \]
Hence it is clear that regularization reduces the maximum possible cost of the physical realization of the solution found.
An illustration of this conclusion may be the circumstance that the solution of equation (9), as well as the solution found by applying the regularizing algorithm (10), has a finite norm and therefore does not contain generalized functions in its composition. Filters specified by generalized functions are physically unrealizable.
Moscow Higher Technical School
named after Bauman
Received
28 I 1966
REFERENCES
- A. N. Tikhonov, DAN, 151, No. 3 (1963).
- A. N. Tikhonov, DAN, 153, No. 1 (1963).