Yu. V. Egorov, A. A. Milyutin

On Sufficient Conditions for a Strong Extremum in the Class of Curves with Bounded Derivative

(Presented by Academician L. S. Pontryagin on 15 VI 1964)

As is known, sufficient conditions for a strong extremum in the classical calculus of variations have a nonlocal character (see, for example, \((^1)\)). In the present note we show that this circumstance is connected not with the fact that the functional is not continuous in the space \(C(a,b)\), but with the fact that the range of variation of the derivatives is unbounded.

In the general theory of optimal control, typical problems are those with a compact domain of controls. This circumstance, apparently, makes it possible there too to give sufficient conditions for optimality connected only with the extremal. The point is that if the domain of controls is compact, then it is enough to establish a strong minimum for nearby curves, where nearness is understood in the sense of \((^2)\). The local investigation, apparently, can be carried out with the help of the variational methods proposed in \((^3)\).

1. Let

\[ \frac{dx(t)}{dt}=f(t,x(t),u(t)),\qquad x=(x_1,\ldots,x_n),\qquad f=(f_1,\ldots,f_n) \]
\[ (a\leq t\leq b). \tag{1} \]

and suppose it is required to find a function \(u(t)\) with values in a given compact set \(U\) in some topological space such that

\[ x(a)=x_0,\qquad x(b)=x_1 \tag{2} \]

and the functional

\[ I(x)=\int_a^b f_0(t,x(t),u(t))\,dt \]

takes the least possible value. (We note that we restrict ourselves to the conditions (2) only for simplicity of exposition.)

Let

\[ H(\psi,t,x;u)=\sum_{\alpha=0}^{n}\psi_\alpha f_\alpha(t,x,u) \]

be a function of \(u\in U\) attaining its maximum at an optimal \(u\) (see \((^2)\)).

Theorem 1. Let \(x=x_0(t)\) be a curve satisfying (1)—(2) for \(u=u_0(t)\), such that

\[ H(\psi(t),t,x_0(t),u_0(t))=\sup_{u\in U} H(\psi(t),t,x_0(t),u). \]

If a sequence of curves \(x=x_m(t)\) satisfies (1)—(2) for \(u=u_m(t)\), with

\[ \sup_{a\leq t\leq b}|x_m(t)-x_0(t)|\to 0\quad (m\to\infty) \]

and \(I(x_m)<I(x_0)\), then

\[ H(\psi(t),t,x_0(t),u_m(t)) - H(\psi(t),t,x_0(t),u_0(t))\to 0 \]

in measure.

Proof. The difference

\[ H(\psi(t),t,x_0(t),u_m(t)) - H(\psi(t),t,x_0(t),u_0(t)) \]

is nonpositive for almost all \(t\), while, on the other hand,

\[ \int_a^b \bigl[ H(\psi(t),t,x_0(t),u_m(t)) - H(\psi(t),t,x_0(t),u_0(t)) \bigr]\,dt = \]

\[ = \int_a^b \sum_{\alpha=0}^n \psi_\alpha(t)\,[f_\alpha(t,x_0,u_m)-f_\alpha(t,x_0,u_0)]\,dt = \psi_0[I(x_m)-I(x_0)] + \]

\[ + \int_a^b \psi_0[f_0(t,x_0,u_m)-f_0(t,x_m,u_m)]\,dt + \int_a^b \sum_{\alpha=1}^n \psi_\alpha(t) \left[\frac{dx_{\alpha m}}{dt}-\frac{dx_{\alpha 0}}{dt}\right]dt + \]

\[ + \int_a^b \sum_{\alpha=1}^n \psi_\alpha(t)\,[f_\alpha(t,x_0,u_m)-f_\alpha(t,x_m,u_m)]\,dt . \]

But \(\psi_0[I(x_m)-I(x_0)] \geq 0\), while the remaining terms on the right-hand side tend to zero as \(m \to \infty\), since \(x_m(t)\to x_0(t)\) uniformly, and

\[ \int_a^b \psi_\alpha(t) \left[\frac{dx_{\alpha m}}{dt}-\frac{dx_{\alpha 0}^*}{dt}\right]dt = -\int_a^b \frac{d\psi_\alpha(t)}{dt}(x_{\alpha m}-x_{0m})\,dt . \]

Corollary. Suppose the conditions of Theorem 1 are satisfied and

\[ \mathcal H(\psi(t),t,x_0(t),u_0(t))> \mathcal H(\psi(t),t,x_0(t),u) \]

for almost all \(t\) in \((a,b)\) and \(u\ne u_0(t)\), \(u\in U\). Then \(u_n(t)\to u_0(t)\) in measure.

1. Let a functional be given by

\[ I(x)=\int_a^b f(t,x(t),\dot x(t))\,dt,\qquad x(a)=x_0,\quad x(b)=x_1, \tag{3} \]

and let the smooth vector function \(x=x_0(t)\) satisfy the Euler equations for this functional.

Theorem 2. Suppose that along the curve \(x=x_0(t)\)

\[ \int_a^b \sum_{i,j=1}^n \left[ f_{x_i x_j}(t,x_0(t),\dot x_0(t))h_i h_j + f_{x_i \dot x_j} h_i \dot h_j + f_{\dot x_i x_j}\dot h_i h_j \right]dt \geq c\int_a^b \sum_{i=1}^n h_i^2\,dt \tag{A} \]

\[ (c>0;\quad h_i(a)=h_i(b)=0;\quad i=1,\ldots,n) \]

(the strengthened Legendre and Jacobi conditions are satisfied), and the Weierstrass function is positive (the strict maximum principle holds). Then, whatever \(M\) may be, \(I(x_0)<I(x)\) for all \(x\) sufficiently close to \(x_0\) in the metric \(C(a,b)\) and such that \(x(a)=x_0,\ x(b)=x_1,\ |\dot x(t)|<M\).

Remark 1. It can be shown that condition (A) is analogous to the condition of positivity of the second variation in the topology considered in Theorem 1. Since the theorem allows one to restrict oneself to a local investigation, the situation here is analogous to the classical one: a sufficient condition for a minimum at a point where the derivative is zero is the condition that the second derivative be positive.*

Remark 2. We note that for \(M=\infty\) the curve \(x=x_0(t)\) may fail to be optimal.

1. In the case where the maximum principle is non-strict, the Legendre and Jacobi conditions are insufficient for an extremum.

Example. Let

\[ I(x)=\int_{-1}^{1} \dot x^{\,2}\,[t^2+(\dot x-1)^2+\alpha x]\,dt, \qquad x(-1)=x(1)=0\quad(\alpha>0). \tag{4} \]

It is easy to see that the curve \(x(t)\equiv 0\) satisfies the Euler equation and the Legendre and Jacobi conditions, but is not optimal, since if we put

\[ x_1(t)= \begin{cases} -\varepsilon(t+1)(1-\varepsilon)^{-1}, & -1\leq t\leq -\varepsilon,\\ t, & -\varepsilon\leq t\leq 0,\\ 0, & 0\leq t\leq 1, \end{cases} \]

* For other reasons, Theorem 2 was obtained by V. F. Krotov (⁴).

then \(I(x_1)<0\), if \(\varepsilon\) is sufficiently small. Note that in this case the function \(H\) is equal to \(u^2[t^2+(u-1)^2]\), and uniqueness of the maximum point is violated only for \(t=0\).

By means of the variational methods proposed in (3), one can obtain necessary conditions in the case of a non-strict maximum principle. In this case those nonnegativity conditions for the second variation which are obtained by considering controls close in the sense of Theorem 1 differ from the conditions arising when uniformly close controls are considered. Moreover, for a certain structure of the set of points \((t,u)\) at which the maximum of \(H\) is attained, it may turn out that the necessary conditions obtained with the aid of the second variation cannot be converted into sufficient ones. Below we consider a class of problems for which such a conversion is possible.

Theorem 3. Let the functional (3) be given and let \(n=1\). Let \(M_+\) \((M_-)\) be the set of those \(t\) for which there exists at least one \(u(t)>\dot x_0(t)\) \((u(t)<\dot x_0(t))\) such that

\[ H(\psi(t),t,x_0(t),u(t))=\sup_{u\in U} H(\psi(t),t,x_0(t),u), \]

and suppose that \(\rho(M_+,M_-)>0\). Assume that along the curve \(x=x_0(t)\) the Euler equation and the strengthened Legendre condition are fulfilled, the Weierstrass function is nonnegative, and for any \(t_1\in M_+\cup M_-\)

\[ \int_a^{t_1}2\omega(h,\dot h)\,dt - \frac{f_x(t_1,x_0(t_1),\dot x_0(t_1))-f_x(t_1,x_0(t_1),u(t_1))} {\dot x_0(t_1)-u(t_1)} h^2(t_1) \ge c_0\int_a^{t_1}\dot h^2\,dt, \tag{5} \]

\[ \int_{t_1}^{b}2\omega(g,\dot g)\,dt + \frac{f_x(t_1,x_0(t_1),\dot x_0(t_1))-f_x(t_1,x_0(t_1),u(t_1))} {\dot x_0(t_1)-u(t_1)} g^2(t_1) \ge c_0\int_{t_1}^{b}\dot g^2\,dt, \]

and if \(t_1\in M_+\ (M_-)\), \(t_2\in M_-\ (M_+)\) and \((t_1,t_2)\cap(M_+\cup M_-)=0\), then

\[ \int_{t_1}^{t_2}2\omega(z,\dot z)\,dt + \frac{f_x(t_1,x_0(t_1),\dot x_0(t_1))-f_x(t_1,x_0(t_1),u(t_1))} {\dot x_0(t_1)-u(t_1)} z^2(t_1) - \]

\[ - \frac{f_x(t_2,x_0(t_2),\dot x_0(t_2))-f_x(t_2,x_0(t_2),u(t_2))} {\dot x_0(t_2)-u(t_2)} z^2(t_2) \ge c_0\int_{t_1}^{t_2}(z^2+\dot z^2)\,dt . \tag{6} \]

Here \(c_0>0\), \(h(a)=0\), \(g(b)=0\), \(z(t_1)z(t_2)\ge0\),

\[ 2\omega(h,\dot h)\equiv f_{xx}(t,x_0,\dot x_0)h^2+ 2f_{x\dot x}(t,x_0,\dot x_0)h\dot h+ f_{\dot x\dot x}(t,x_0,\dot x_0)\dot h^2 . \]

Then, whatever \(M\) may be, \(I(x_0)<I(x)\) for all \(x(t)\) sufficiently close to \(x_0(t)\) in the metric \(C(a,b)\) and such that \(x(a)=x_0\), \(x(b)=x_1\), \(|\dot x(t)|<M\).

If in conditions (5), (6) one sets \(c_0=0\), these conditions become necessary for the optimality of \(x_0(t)\).

Let us note that in the example given above these conditions have the form:

\[ \int_{-1}^{0}2(1+t^2)\dot h^2\,dt-\alpha h^2(0)\ge0, \qquad \int_{0}^{1}2(1+t^2)\dot g^2\,dt+\alpha g^2(0)\ge0. \]

Obviously, the first of these conditions cannot be satisfied for all \(h(t)\) with \(h(-1)=0\), if \(\alpha\) is sufficiently large.

1. In conclusion we give a simple derivation of L. S. Pontryagin’s maximum principle. As we have learned, the same approach to obtaining the maximum principle was used independently of us by R. V. Gamkrelidze. However, his method of proof differs somewhat from ours.

Let the general optimal-control problem be considered (see Sec. 1). Let \(u_0(t)\) and \(u_1(t)\) be arbitrary admissible controls, and let \(\alpha(t)\) be a measurable function, \(a \le t \le b\), \(0 \le \alpha(t) \le 1\). If one constructs a control \(u(t)\) by alternating small intervals on which \(u=u_0(t)\) with intervals on which \(u=u_1(t)\), the ratio of the lengths of the former intervals to the lengths of the latter being close to \(\alpha(t):[1-\alpha(t)]\), then the trajectory corresponding to \(u(t)\) will be close to the solution of the equation

\[ \frac{dx(t)}{dt} = \alpha(t) f(t,t(t),u_0(t)) + [1-\alpha(t)] f(t,x(t),u_1(t)), \tag{7} \]

and the value of the functional \(I(x)\) will be close to

\[ J(\alpha) = \int_a^b \{\alpha(t) f_0(t,x(t),u_0(t)) + [1-\alpha(t)] f_0(t,x(t),u_1(t))\}\,dt . \tag{8} \]

Suppose that \(\alpha(t)\) is such that there exists a solution of (7) satisfying the conditions \(x(a)=x_0,\ x(b)=x_1\). Generally speaking, it can be shown that in this case there exists a sequence of admissible controls of the above indicated form. Consequently, if \(u=u_0(t)\) is an optimal control, and \(u_1(t)\) is arbitrary, then \(I(x_0)\le J(\alpha)\). Choose \(u_1(t)\) so that, for an arbitrarily small neighborhood of any point of the interval \((a,b)\), the closure of the range of values of \(u_1(t)\) coincides with \(U\).

We shall regard \(\alpha(t)\) as a control. Write the Euler equation (see (3)), taking into account that \(\alpha(t)\equiv 1\) is the optimal control:

\[ -\psi_0 \left\{ \int_a^b \left[ \bar\alpha\,[f_0(t,x_0,u_0)-f_0(t,x_0,u_1)] + \frac{\partial f(t,x_0,u_0)}{\partial x}\,\bar x \right]dt \right\} + v(\bar\alpha) + c\bar x(1) - \]

\[ - \int_a^b \left\{ \bar x(\tau) - \int_0^t \left[ \bar\alpha(\tau)\bigl(f(\tau,x_0(\tau),u_0(\tau))-f(\tau,x_0(\tau),u_1(\tau))\bigr) + \frac{\partial f(\tau,x_0(\tau),u_0(\tau))}{\partial x}\,\bar x(\tau) \right]d\tau \right\} \,d\psi(t) =0, \]

where \(\psi_0\le 0\) is a constant, and \(c\) is a constant vector. It is easy to see, taking into account the arbitrariness of \(\bar x(t)\), that \(\psi(t)\) is an absolutely continuous vector-function satisfying the system

\[ -\frac{d\psi}{dt} = \left( \frac{\partial f(t,x_0(t),u_0(t))}{\partial x} \right)^{*} \psi(t) + \psi_0 \frac{\partial f_0(t,x_0(t),u_0(t))}{\partial x}, \qquad \psi(b)=c . \]

Putting \(\bar x(t)\equiv 0\), we have

\[ -\psi_0 \int_a^b \bar\alpha\,[f_0(t,x_0,u_0)-f_0(t,x_0,u_1)]\,dt + v(\bar\alpha) - \]

\[ - \int_a^b \bigl(\psi(t),\,f(t,x_0,u_0)-f(t,x_0,u_1)\bigr)\,\bar\alpha\,dt =0. \]

Since \(\bar\alpha\le 0\) is an arbitrary bounded function and \(v(\bar\alpha)\ge 0\), it follows that

\[ \psi_0[f_0(t,x_0,u_0)-f_0(t,x_0,u_1)] + (\psi(t),f(t,x_0,u_0)-f(t,x_0,u_1)) \ge 0 . \]

This is precisely the maximum principle.

Moscow State University
named after M. V. Lomonosov

Institute of Chemical Physics
Academy of Sciences of the USSR

Received
11 VI 1964

CITED LITERATURE

1. G. A. Bliss, Lectures on the Calculus of Variations, IL, 1950.
2. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, Mathematical Theory of Optimal Processes, Moscow, 1961.
3. A. Ya. Dubovitskii, A. A. Milyutin, DAN, 160, No. 1 (1965).
4. V. F. Krotov, Dissertation, Moscow, 1964.