MATHEMATICS

G. L. KHARATISHVILI

THE MAXIMUM PRINCIPLE IN THE THEORY OF OPTIMAL PROCESSES WITH DELAY

(Presented by Academician L. S. Pontryagin, 4 VII 1960)

In the present note, L. S. Pontryagin’s maximum principle (see ((^1))) is generalized to the case of optimal processes with delay, and some applications of the result obtained to linear optimal systems with delayed argument are given.*

1°. Statement of the problem. Let (\Omega) be an arbitrary subset of the linear real (r)-dimensional space (E^r). By the class of admissible controls we shall mean the set of all piecewise-continuous functions, with discontinuities of the first kind, (u(t)) of the real argument (t) (time), defined on an arbitrary interval (t_0 \leq t \leq t_1) and taking values in (\Omega) at each instant of time.

Let (X^n) be the (n)-dimensional phase space of the optimal problem formulated below, and let (x=(x^1,\ldots,x^n)) be a point of this space. Suppose, further, that scalar functions (f^i(x,y,u)), (i=1,\ldots,n), are given, continuous in the totality of the arguments (x,y,u) and continuously differentiable with respect to all coordinates of the vectors (x,y) of the space (X^n). Let the system of equations of motion of the phase point (x(t)=(x^1(t),\ldots,x^n(t))) have the form

[
\dot{x}^i(t)=f^i(x(t),x(t-\tau),u(t)),\qquad i=1,\ldots,n;\ \tau=\mathrm{const}>0.
\tag{1}
]

Here, on the right-hand side of system (1), under the sign of the functions (f^i(x,y,u)), (i=1,\ldots,n), in place of the arguments (x,y,u) there stand, respectively, (x(t), x(t-\tau), u(t)). Thus, the delayed argument is contained only in the phase coordinates and is absent in the control. For the unique determination of the trajectory (x(t)) of system (1), it is necessary to prescribe not only an admissible control (u(t)), but also an initial function (\varphi(t)\in X^n), defined on some interval of length (\tau). Speaking of the trajectory of system (1) corresponding to an admissible control (u(t)), (t_0\leq t\leq t_1), and to a prescribed continuous initial function (\varphi(t)), (t_0-\tau\leq t\leq t_0), we shall always assume that the corresponding solution (x(t)) of system (1) is defined and continuous on the interval (t_0\leq t\leq t_1) and at the point (t_0) coincides with (\varphi(t_0)): (x(t_0)=\varphi(t_0)). Therefore it is convenient to speak of a continuous trajectory (x(t)) of system (1), defined on the interval (t_0-\tau\leq t\leq t_1), corresponding to an admissible control (u(t)), (t_0\leq t\leq t_1), and to the initial function (\varphi(t)), (t_0-\tau\leq t\leq t_0).

Formulation of the optimal problem. Suppose that in (X^n) a smooth (k)-dimensional manifold (M^k), (0\leq k\leq n), and a continuous initial function (\varphi(t)) (defined on an interval of length (\tau)) are given. It is required, in the class of admissible controls, to choose such a control (u(t)), (t_0\leq t\leq t_1), that the trajectory (x(t)), (t_0-\tau\leq t\leq t_1), of system (1), corresponding to the chosen control (u(t)) and to the prescribed initial function (\varphi(t)), (t_0-\tau\leq t\leq t_0), should satisfy the boundary condition (x(t_1)\in M^k), and the integral

[
\int_{t_0}^{t_1} f^0(x(t),x(t-\tau),u(t))\,dt
\tag{2}
]

* The results set forth in the present note were obtained in L. S. Pontryagin’s seminar on mathematical problems of the theory of oscillations and automatic control.

attain a minimum, where the scalar function (f^0(x,y,u)) satisfies the same conditions as the scalar functions (f^i(x,y,u)), (i=1,\ldots,n), and in computing the integral (2), under the sign of the function (f^0), instead of the arguments (x,y,u) we substitute, respectively, (x(t), x(t-\tau), u(t)). We note that the limits of integration (t_0,t_1) in (2) are not fixed; only the boundary conditions are fixed: the initial function (\varphi(t)), (t_0-\tau \leq t \leq t_0), is given, and (x(t_1)\in M^k).

If (f^0(x,y,u)\equiv 1), then we obtain the time-optimal problem for systems with delay. If the dimension of the manifold (M^k) is zero, then (M^k) becomes a point, and we obtain the optimal problem with fixed right endpoint; if the dimension (k=n), then the boundary condition on the right is absent altogether, since (M^k) in this case is an open set of the space (X^n).

We shall now give another (equivalent) formulation of our optimal problem, more convenient for the formulation and proof of the main result.

Introduce the ((n+1))-dimensional phase space (X^{n+1}) of variables (x^0,\ldots,x^n), and denote points of this space, in contrast to points of the space (X^n), by boldface letters
[
\mathbf{x}=(x^0,\ldots,x^n).
]
Let (L^{k+1}) denote the direct product of the manifold (M^k) by the axis (x^0). Then our optimal problem is equivalent to the following problem.

The system of equations of motion of the phase point
[
\mathbf{x}(t)=(x^0(t),\ldots,x^n(t))
]
has the form
[
\dot{x}^i(t)=f^i(x(t),x(t-\tau),u(t)),\qquad i=0,1,\ldots,n. \tag{3}
]
It is required to choose an admissible control (u(t)), (t_0\leq t\leq t_1), such that the trajectory
[
\mathbf{x}(t)=(x^0(t),x(t)),\qquad t_0-\tau\leq t\leq t_1,
]
of system (3), corresponding to this control and to the initial function
[
\vec{\varphi}(t)=(0,\varphi(t)),\qquad t_0-\tau\leq t\leq t_0,
]
satisfy the boundary condition (\mathbf{x}(t_1)\in L^{k+1}), and the coordinate (x^0(t_1)) attain a minimum. Any admissible control (u(t)) and corresponding trajectory (\mathbf{x}(t)) satisfying the formulated problem will be called optimal.

2°. The maximum principle for optimal processes with delay. Introduce the ((n+1))-dimensional covariant vector
[
\vec{\psi}=(\psi_0,\ldots,\psi_n)
]
of the space (X^{n+1}), and form the scalar function
[
H(\vec{\psi},x,y,u)=\sum_{\alpha=0}^{n}\psi_\alpha f^\alpha(x,y,u).
]

By (M(\vec{\psi},x,y)) we shall denote the exact upper bound of the function (H(\vec{\psi},x,y,u)) for fixed (\vec{\psi},x,y) and for (u) varying over the set (\Omega).

Theorem 1 (maximum principle). Let
[
\mathbf{x}(t)=(x^0(t),x(t)),\qquad t_0-\tau\leq t\leq t_1,
]
be an optimal trajectory of system (3), corresponding to an optimal control (u(t)), (t_0\leq t\leq t_1), and to the initial function
[
\vec{\varphi}(t)=(0,\varphi(t)),\qquad t_0-\tau\leq t\leq t_0.
]
Then there exists a nonzero continuous vector-function
[
\vec{\psi}(t)=(\psi_0(t),\psi(t))=(\psi_0(t),\ldots,\psi_n(t)),\qquad t_0\leq t\leq t_1,
]
satisfying on the interval (t_0\leq t\leq t_1-\tau) the system
[
\dot{\psi}_i(t)
=
-\frac{\partial H(\vec{\psi}(t),x(t),x(t-\tau),u(t))}{\partial x^i}
-\frac{\partial H(\vec{\psi}(t+\tau),x(t+\tau),x(t),u(t+\tau))}{\partial y^i},
\qquad
i=0,1,\ldots,n, \tag{4}
]
and on the interval (t_1-\tau\leq t\leq t_1) the system
[
\dot{\psi}_i(t)
=
-\frac{\partial H(\vec{\psi}(t),x(t),x(t-\tau),u(t))}{\partial x^i},
\qquad
i=0,1,\ldots,n. \tag{5}
]

that on the entire interval (t_0 \leqslant t \leqslant t_1) the maximum condition is satisfied
[
H(\vec\psi(t),\,x(t),\,x(t-\tau),\,u(t))
= M(\vec\psi(t),\,x(t),\,x(t-\tau)) \tag{6}
]
and the inequality (\psi_0(t)=\mathrm{const}\leqslant 0); moreover, at the terminal time (t_1)
[
H(\vec\psi(t_1),\,x(t_1),\,x(t_1-\tau),\,u(t_1))
= M(\vec\psi(t_1),\,x(t_1),\,x(t_1-\tau))=0, \tag{7}
]
and the vector (\psi(t_1)=(\psi_1(t_1),\ldots,\psi_n(t_1))) is orthogonal to the (k)-dimensional tangent plane (T^k) of the manifold (M^k) at the point
[
x(t_1)=(x^1(t_1),\ldots,x^n(t_1)).
]

Thus, in order to find optimal controls and optimal trajectories by means of Theorem 1, it is necessary to solve the joint system of differential equations (3), (4) on the interval (t_0 \leqslant t \leqslant t_1-\tau) and the system (3), (5) on the interval (t_1-\tau \leqslant t \leqslant t_1). Here a difficulty arises, connected with the circumstance that the unknown functions (x^1(t),\ldots,x^n(t)), (\psi_1(t),\ldots,\psi_n(t)) enter the system (3), (4) both with retarded and with advanced argument. For linear optimal systems with retarded argument this difficulty is absent (see § 4).

3°. Brief outline of the proof of Theorem 1. Let
[
x(t),\quad t_0-\tau\leqslant t\leqslant t_1,
]
be an optimal trajectory of the system (3), corresponding to the optimal control (u(t)), (t_0\leqslant t\leqslant t_1), and to the initial function (\vec\varphi(t)), (t_0-\tau\leqslant t\leqslant t_0). Construct a family (V) of admissible variations of the control (u(t)) by the method described in [1]. To each varied control (v\in V) and the prescribed initial function (\vec\varphi(t)), (t_0-\tau\leqslant t\leqslant t_0), there corresponds a trajectory (y(t)) of the system (3). It can be proved that
[
y(t_1)=x(t_1)+\varepsilon\,\delta x(t_1)+o(\varepsilon),
]
where (\varepsilon) is a positive infinitesimal quantity, and the totality of vectors (x(t_1)+\delta x(t_1)), corresponding to all possible (v\in V), forms a convex cone (K) with vertex at the point (x(t_1)).

The family (V) contains variations of the following form:
[
v(t)=
\begin{cases}
u(t), & \text{for } t_0\leqslant t\leqslant \theta-\varepsilon,\
u^, & \text{for } \theta-\varepsilon<t\leqslant \theta,\
u(t), & \text{for } \theta<t\leqslant t_1,\
u(t_1-0), & \text{for } t_10,
\end{cases}
]
where (\theta) is an arbitrary point of continuity of the control (u(t)), (t_0<t<t_1); (u^\in\Omega). To the varied control (v(t)) and the initial function (\vec\varphi(t)), (t_0-\tau\leqslant t\leqslant t_0), there corresponds a trajectory (y(t)) of the system (3), representable on the interval (\theta\leqslant t\leqslant t_1+\varepsilon\rho) in the form
[
y(t)=x(t)+\varepsilon\,\delta x(t)+o(\varepsilon),
]
where (\delta x(t)\equiv 0) for (\theta-\tau\leqslant t<\theta), while on the interval (\theta\leqslant t\leqslant t_1+\varepsilon\rho) it satisfies the system of equations in variations
[
\delta \dot x^{\,i}(t)=
\sum_{\alpha=0}^{n}\left[
\frac{\partial f^i(x(t),x(t-\tau),u(t))}{\partial x^\alpha}\,\delta x^\alpha(t)
+
\frac{\partial f^i(x(t),x(t-\tau),u(t))}{\partial y^\alpha}\,\delta x^\alpha(t-\tau)
\right],
]
[
i=0,1,\ldots,n,
]
and takes the initial value
[
\delta x(\theta)
=
f(x(\theta),\,x(\theta-\tau),\,u^*)
-
f(x(t),\,x(\theta-\tau),\,u(\theta)),
]
where (f=(f^0,\ldots,f^n)).

Since the trajectory (x(t)) is optimal, no ray (l\subset X^{n+1}), issuing from the point (x(t_1)) and directed along the negative axis (x^0), is interior to the cone (K). Consequently, through (x(t_1)) one can draw a supporting plane (P) to the cone (K), separating the ray (l) from the cone (K).

Let (\vec\chi=(\chi_0,\ldots,\chi_n)) be the covariant vector orthogonal to (P) and directed toward the ray (l), and let (\vec\psi(t)), (t_0\leqslant t\leqslant t_1), be a continuous solution of the system (4), (5), satisfying the boundary condition (\vec\psi(t_1)=\vec\chi).

Then the scalar product

[
\vec{\psi}(t_1)\cdot \delta x(t_1)=\vec{\chi}\cdot \delta x(t_1)\leqslant 0.
\tag{8}
]

Let us prove the equality

[
\vec{\psi}(\theta)\cdot \delta x(\theta)=\vec{\psi}(t_1)\cdot \delta x(t_1).
\tag{9}
]

We have

[
\begin{aligned}
\vec{\psi}(t_1)\cdot \delta x(t_1)-\vec{\psi}(\theta)\cdot \delta x(\theta)
&=\int_{\theta}^{t_1}\bigl(\vec{\psi}(t)\cdot \delta x(t)\bigr)' \, dt \
&=\sum_{\alpha=0}^{n}\left[
\int_{\theta+\tau}^{t_1}
\frac{\partial H\bigl(\vec{\psi}(t),x(t),x(t-\tau),u(t)\bigr)}
{\partial y^\alpha}\,
\delta x^\alpha(t-\tau)\,dt \right.\
&\qquad\left.
-\int_{\theta}^{t_1-\tau}
\frac{\partial H\bigl(\vec{\psi}(t+\tau),x(t+\tau),x(t),u(t+\tau)\bigr)}
{\partial y^\alpha}\,
\delta x^\alpha(t)\,dt
\right]=0.
\end{aligned}
]

From inequality (8) and equality (9) there follows equality (6)—the maximum condition. The inequality (\psi_0(t)=\mathrm{const}\leqslant 0), equality (7), and the orthogonality of the vector (\psi(t_1)) to (T^k) are proved in the same way as in paper ({}^{1}).

(4^\circ). Linear time-optimal systems with a delayed argument. The most important example for applications is a linear controlled system with a delayed argument

[
\dot{x}^{i}(t)=\sum_{\alpha=1}^{n}a_{\alpha}^{i}x^{\alpha}(t)
+\sum_{\alpha=1}^{n}b_{\alpha}^{i}x^{\alpha}(t-\tau)
+\sum_{\beta=1}^{r}c_{\beta}^{i}u^{\beta}(t),
\qquad i=1,\ldots,n,
\tag{10}
]

where (x=(x^1,\ldots,x^n)\in X^n) and (u=(u^1,\ldots,u^r)) is a point of a closed convex (r)-dimensional polyhedron (\Omega).

We shall consider only the time-optimal problem. In our case the system of equations for (\psi(t)) has the form

[
\begin{aligned}
\dot{\psi}i(t)&=-\sum\psi_\alpha(t)}^{n}a_i^{\alpha
-\sum_{\alpha=1}^{n}b_i^{\alpha}\psi_\alpha(t+\tau),
\qquad t_0\leqslant t\leqslant t_1-\tau,\
\dot{\psi}i(t)&=-\sum\psi_\alpha(t),}^{n}a_i^{\alpha
\qquad t_1-\tau\leqslant t\leqslant t_1,\quad i=1,\ldots,n.
\end{aligned}
\tag{11}
]

System (11) does not contain the unknowns (x^i(t)), (i=1,\ldots,n); therefore it can be solved independently of system (10), and the difficulty indicated at the end of Sec. 2 is absent in this case.

We shall call system (10) nondegenerate if, for every nonzero solution (\psi(t)) of system (11) and for every vector (w\in E^r) having the direction of one of the edges of the polyhedron (\Omega), the scalar product (\psi(t)\cdot Cw) does not vanish identically on any time interval (t), where (C=|c_j^i|). We have not succeeded in finding simple sufficient conditions for nondegeneracy analogous to the nondegeneracy conditions contained in papers ({}^{2,3}). However, for a broad class of linear systems with delay that have important technical applications, the nondegeneracy condition is easily established. For nondegenerate systems, uniqueness theorems analogous to the uniqueness theorems for linear systems without delay are valid.

Mathematical Institute named after V. A. Steklov
Academy of Sciences of the USSR

Received
2 VII 1960

REFERENCES

1. V. G. Boltyanskii, R. V. Gamkrelidze, L. S. Pontryagin, Izv. AN SSSR, Ser. matem., 24, 3 (1960).
2. R. V. Gamkrelidze, Izv. AN SSSR, Ser. matem., 22, 449 (1958).
3. L. S. Pontryagin, UMN, 14, no. 1 (1959).