Reports of the Academy of Sciences of the USSR
1960. Volume 134, No. 2

CYBERNETICS AND CONTROL THEORY

Ya. Z. Tsypkin

ON OPTIMAL PROCESSES IN PULSE AUTOMATIC SYSTEMS

(Presented by Academician V. A. Kotelnikov on 26 IV 1960)

By optimal processes in pulse automatic systems (p.a.s.) are meant those processes for which one or another functional, determining in some sense the quality of the system, attains an extremal value.

P.a.s. are described by difference equations, and as applied to these equations the problem of optimal processes was considered in the works of N. N. Krasovskii on the basis of the moment problem (¹); by R. Bellman on the basis of the dynamic programming method developed by him (²); by R. Kalman, J. Bertram, and V. Kelke on the basis of the concepts of vector space (³–⁵) and the dynamic programming method (⁵); and by L. I. Rozonoer on the basis of L. S. Pontryagin’s maximum principle (⁶).

The solution of the problem of optimal processes was reduced to finding, in a certain phase space of the system, a trajectory connecting the origin of coordinates with the point corresponding to the initial conditions and minimizing one or another functional.

Below we present a somewhat different approach to the solution of this problem, based on using, instead of a system of first-order difference equations, an equation in convolution form. This makes it possible to operate with the physical concept of the time characteristic and, consequently, to consider systems both with lumped and with distributed parameters.

In addition, this approach makes it possible to solve the problem on the basis of the elementary apparatus of the theory of extrema of functions of many variables.

The equation of an open-loop p.a.s., consisting of a pulse element and a continuous part under any kind of modulation, may be written in the form

\[ z[n] = \sum_{m=0}^{n} \Psi(n, m, u[m]), \tag{1} \]

where \(\Psi(n, m, u[m])\) is the response of the continuous part of the system at the instant \(\bar t = n\) to a pulse applied at the instant \(\bar t = m\); \(u[m]\) and \(z[m]\) are lattice functions determining, respectively, the control action and the output quantity of the p.a.s. at discrete instants of time. For a p.a.s. with constant parameters, \(\Psi(n, m, u[m]) = \Psi(n - m, u[m])\). The expressions for \(\Psi(n - m, u[m])\) depend on the type of modulation, and for amplitude, width, and time pulse modulations they are given in (⁷).

As in (⁷), in what follows processes are considered on the relative time scale \(\bar t = t/T_p\), where \(T_p\) is the repetition interval of the p.a.s.

Let \(z^0[n]\) denote the desired course of the process; then the measure of deviation of the process from the desired one is the error

\[ \varepsilon[n] = z^0[n] - z[n]. \tag{2} \]

The dynamic properties of a sampled-data automatic control system may be evaluated by a functional of the grid function \(u[n]\)

\[ J\{u[n]\}=\sum_{k=0}^{s} F\left(\varepsilon[k],\varepsilon^{(1)}[k],\ldots,\varepsilon^{(r)}[k];\,u[k]\right), \tag{3} \]

where \(F\) is some prescribed function determined by the chosen measure of quality of the sampled-data automatic control system, and

\[ \varepsilon^{(\nu)}[k]=\left[\frac{d^\nu \varepsilon(\bar t)}{dt^\nu}\right]_{\bar t=k}. \tag{4} \]

The number \(s\), in particular, may be equal to infinity, and the greatest value of \(r\) is determined by the number of degrees of freedom of the continuous part.

Suppose that the control action is constrained by the restrictions

\[ |u[n]|\leq P_n. \tag{5} \]

The case of additional conditions imposed, for example, on \(\varepsilon^{(\nu)}[k]\), is considered below.

The main problem in the theory of optimal processes consists in determining a control action \(u_{\mathrm{opt}}[n]\) satisfying the constraints (5), for which the functional (3), by virtue of equations (1), (2), attains an extremal value.

In order to apply to the solution of this problem the elementary apparatus of the theory of extrema of functions of many variables, we introduce in (3) the change of variables

\[ u[n]=P_n\sin v[n]. \tag{6} \]

The necessary condition of optimality for the functional \(J\{P_n\sin v[n]\}\) with respect to the grid function \(v[n]\), which may be regarded as a function of many variables \(v[n]\) \((n=0,1,2,\ldots,s)\), is written in the form

\[ \frac{\partial J\{P_n\sin v[n]\}}{\partial v[r]} = \left[ \frac{\partial J\{u[n]\}}{\partial u[r]} \right]_{u[r]=P_r\sin v[r]} P_n\cos v[r]=0 \quad (r=0,1,2,\ldots,s). \tag{7} \]

These equations are equivalent to two groups of equations:

\[ \left[ \frac{\partial J\{u[n]\}}{\partial u[r]} \right]_{u[r]=P_r\sin v[r]}=0; \tag{8} \]

\[ P_r\cos v[r]=0. \tag{9} \]

Equations (8) determine the necessary conditions for an extremum in the absence of constraints. Equations (9) determine the boundary values of the optimal control

\[ u[n]=\pm P_n. \tag{10} \]

From the systems of equations (8), (9) the optimal control actions lying inside and on the boundary of the domain (5) are determined. If all solutions of equations (8) do not satisfy the constraints (5), then the optimal control actions coincide with the boundary ones.

It is often necessary to find the extremum of the functional (5) in the presence of additional conditions, for example, under the requirement of a finite duration of the process

\[ \varepsilon^{(\nu)}[s]=0 \quad (\nu=0,1,\ldots,l-1), \tag{11} \]

where \(l\) is determined by the number of degrees of freedom of the continuous part, or under an energy constraint

\[ \sum_{k=0}^{s} u^{2}[k]=M \tag{12} \]

and so on.

This problem reduces to finding a constrained extremum and can be solved by the same method. However, in this case the optimal control actions will depend on undetermined Lagrange multipliers, which are determined from conditions (11) or (12), taking account of the constraints (5).

For linear pulse automatic systems and quadratic functionals, the optimal control actions can be found by methods of linear programming.

In the particular case of the functional (3)

\[ J\{u[n]\}=\sum_{k=0}^{s} F(u[k]) \tag{13} \]

equations (8) have the form

\[ \frac{\partial J\{u[n]\}}{\partial u[r]} = \frac{\partial F(u[r])}{\partial u[r]} =0 \qquad (r=0,\,1,\,2,\,\ldots\,s); \tag{14} \]

they are independent. Denoting the solutions of this system of equations by \(u^{0}[n]\), we arrive at the following theorem:

Theorem. If the control action is optimal, then it is determined by the expression

\[ u_{\mathrm{opt}}[n]= \begin{cases} u^{0}[n], & \text{when } |u^{0}[n]|<P_{n};\\ \pm P_{n}, & \text{when } |u^{0}[n]|\geq P_{n}. \end{cases} \tag{15} \]

In this case, in order to determine the optimal control actions, it is necessary to find the optimal control actions in the absence of the constraints (5), and to replace those values that do not satisfy these constraints by the limiting values.

Unlike continuous systems, for which the optimal control action in the absence of a constraint of the form (5) is usually expressed by means of a \(\delta\)-function, i.e., is unbounded, in linear pulse automatic systems this case does not arise. The change of the control action only at discrete instants of time, which takes place in pulse automatic systems, is a peculiar constraint that excludes this possibility.

As the repetition interval \(T_p\) tends to zero, a pulse automatic system tends to the equivalent continuous system \((^{8})\). In this case the optimal values \(u^{0}(\bar t)\) must coincide with the limiting values. This conclusion also follows from L. S. Pontryagin’s maximum principle.

Institute of Automation and Telemechanics
Academy of Sciences of the USSR

Received
25 IV 1960

REFERENCES

\(^{1}\) N. N. Krasovskii, Prikl. matem. i mekh., 21, No. 5 (1957).
\(^{2}\) R. Bellman, Dynamic Programming, Princeton, 1957.
\(^{3}\) R. E. Kalman, The WESCON Convention Record, 1, part 4, 1957.
\(^{4}\) R. E. Kalman, J. E. Bertram, Proc. Computers in Control Systems Conference, 1958.
\(^{5}\) R. E. Kalman, R. W. Koepcke, Trans. Am. Soc. Mech. Eng., 80, No. 8 (1958).
\(^{6}\) L. I. Rozonoer, Avtomatika i telemekh., 10, No. 12 (1959).
\(^{7}\) Ya. Z. Tsypkin, DAN, 124, No. 4 (1956).
\(^{8}\) Ya. Z. Tsypkin, Theory of Pulse Systems, Moscow, 1958.