UDC 519.3:62-50

MATHEMATICS

L. NEUSTADT

A GENERAL THEORY OF VARIATIONAL PROBLEMS WITH APPLICATIONS TO OPTIMAL CONTROL

(Presented by Academician L. S. Pontryagin on 4 IV 1966)

In the present note a very general variational problem is formulated and the corresponding necessary condition for extremality is given, containing as special cases the known first-order necessary conditions of the calculus of variations, as well as Pontryagin’s maximum principle and its various generalizations.

Let \(B\) and \(Q\) be two sets in a locally convex linear topological space \(J\). It is assumed that \(0 \in B \cap Q\) and that the topology in \(J\) induces the usual Euclidean topology on every finite-dimensional subspace of \(J\). Let \(\hat{\varphi}\) be a continuous mapping of some neighborhood \(N\) of zero in \(J\) into \(R^m\) (\(m\)-dimensional Euclidean space), with \(\hat{\varphi}(0)=0\), and let \(Y=\{x: x \in N,\ \hat{\varphi}(x)=0\}\). We shall say that zero in \(J\) is a \((Q,B,\hat{\varphi})\)-extremal if there exists a neighborhood \(N^*\) of zero in \(J\) such that \(N^* \cap B \cap Q \cap Y=\{0\}\).

To obtain meaningful necessary conditions for extremality, it is necessary to assume that \(B\), \(Q\), and \(\hat{\varphi}\) possess certain properties which, in a known sense, may be characterized as the existence of “convex first-order approximations.” Precisely, these properties may be formulated as follows. It is assumed that there exists a continuous linear mapping \(\hat{l}\) from \(J\) into \(R^m\) such that

\[ \frac{\hat{\varphi}(\varepsilon y)}{\varepsilon}\longrightarrow \hat{l}(x)\quad \text{as } y\to x,\ \varepsilon\to 0 \tag{1} \]

for every \(x \in J\).

It is also assumed that there exists a convex cone \(Z \subset J\) with vertex at 0, possessing the following property: for each ray \(\rho \subset Z\) there exist a cone \(Z_\rho\) and a neighborhood \(N_\rho\) of zero in \(J\) such that: a) the vertex of \(Z_\rho\) is at 0; b) \(Z_\rho \subset Z\); c) \(Z_\rho\) has a nonempty open core and \(\rho\) is an interior ray of \(Z_\rho\); d) \(Z_\rho \cap N_\rho \subset B\). In this case we say that \(Z\) is an inner cone for \(B\) at 0.

Next we shall assume the existence of a convex subset \(K\) of \(J\), containing 0 and possessing the following property: if \(S\) is an arbitrary simplex from \(K\) with vertices \(x_1,\ldots,x_\nu\), and \(\widetilde{N}\) is an arbitrary neighborhood of zero in \(J\), then there exists a number \(\varepsilon_0>0\) such that for every \(\varepsilon\), \(0<\varepsilon<\varepsilon_0\), there is a continuous mapping \(\zeta_\varepsilon\) from \(P^\nu=\{\beta=(\beta_1,\ldots,\beta_\nu): \beta_i\ge 0,\ i=1,\ldots,\nu,\ \sum_{i=1}^{\nu}\beta_i=1\}\) into \(J\) (this mapping may depend both on \(\varepsilon\) and on \(S\) and \(\widetilde{N}\)), satisfying the condition

\[ \zeta_\varepsilon(\beta)\in \left\{\varepsilon\left(\sum_{i=1}^{\nu}\beta_i x_i+\widetilde{N}\right)\right\}\cap Q \quad \text{for every } \beta\in P^\nu . \]

In this case we shall say that \(K\) is a convex first-order approximation for \(Q\) at zero.

Let \(\Pi=\{x: x\in J,\ \hat l(x)=0\}\), \(Z'=Z\cap \Pi\). Our last assumption is that \(Z'\ne \Pi,\ Z'\ne\{0\}\).

As is indicated below, most optimal-control problems can be reduced to the scheme described.

Necessary conditions for optimality are given by the following two theorems.

Theorem 1. If \(0\) is a \((Q,B,\hat\varphi)\)-extremal and \(Q,B\), and \(\hat\varphi\) satisfy the conditions listed above, then \(K\) and \(Z'\) are separable, i.e., there exists a linear continuous nonzero functional \(l^*\) on \(J\) such that
\[ l^*(x)\le 0\le l^*(y) \]
for all \(x\in K,\ y\in Z'\).

Theorem 2. If \(l^*\) is a linear functional on \(J\) such that \(l^*(y)\ge 0\) for \(y\in Z'\), then
\[ l^*(x)=\bar l(x)+a\cdot \hat l(x) \]
for any \(x\in J\), where \(a\) is a vector in \(R^m\) and \(\bar l(y)\ge 0\) for any \(y\in Z\).

Let \(\varphi_0\) be a real function defined in some neighborhood \(N_0\) of zero in \(J\), and suppose that there exists a continuous convex functional \(l_0\) from \(J\) to \(R^1\) such that \(l_0(x)<0\) for some \(x\in J\) and
\[ \varphi_0(\varepsilon y)/\varepsilon \underset{\substack{y\to x\\ \varepsilon\to 0+}}{\longrightarrow} l_0(x) \quad \text{for any } x\in J. \tag{2} \]

If
\[ B=\{x:x\in N_0,\ \varphi_0(x)<0\}\cup\{0\} \]
and
\[ Z=\{x:x\in J,\ l_0(x)<0\}\cup\{0\}, \]
then \(Z\) is an interior cone for \(B\) at \(0\).

We now indicate how the formulated results are to be applied to optimal-control problems. Let \(J_0\) be the space of continuous mappings of the compact interval \(I\) into \(R^n\), with the usual metric of uniform convergence. Let \(J=J_0\times A\), where \(A\) is a convex set in \(R^k\), and \(U\) an arbitrary set in \(R^r\). Denote by \(\Omega\) the set of all essentially bounded measurable functions from \(I\) into \(U\). Let \(f(x,u,t)\) be a function from \(R^n\times U\times I\) into \(R^n\), of class \(C^1\) with respect to \(x\) and measurable with respect to \((u,t)\), and suppose that \(f(x,u,t)\), \(f_x(x,u,t)\) are bounded on \(X\times I\) for any function \(u\in\Omega\) and compact set \(X\subset R^n\).

Further, let \(Q_0\) be the set of all absolutely continuous functions \(x(t)\in J_0\) satisfying the equation
\[ \dot x(t)=f(x(t),u(t),t) \tag{3} \]
for a suitable choice of \(u(t)\in\Omega\). Let \(\tilde\varphi\) be a continuous function from an open set \(W\subset J\) into \(R^m\), and let \(\varphi_0,\varphi_1,\ldots,\varphi_\mu\) be functionals defined on \(W\) (not necessarily linear). Consider the following optimal problem: find an element \(z\in W\cap(Q_0\times A)\) satisfying the conditions \(\tilde\varphi(z)=0,\ \varphi_i(z)\le 0,\ i=1,\ldots,\mu\), and minimizing the functional \(\varphi_0(z)\). Let \(z^*=(x^*,\alpha^*)\), \(x^*\in Q_0,\ \alpha^*\in A\), be a solution of the problem, and put
\[ \varphi(z)=\tilde\varphi(z+z^*),\quad \varphi_i=\varphi_i(z+z^*),\quad \varphi_0(z)=\varphi_0(z+z^*)-\varphi_0(z^*). \]
Assume that conditions (1) and (2) are satisfied for the functions \(\hat\varphi,\varphi_i,\ i=0,1,\ldots,\mu\), where \(l\) is a linear continuous mapping onto all of \(R^m\), and the functionals \(l_i\) are convex and continuous. Discarding, if necessary, some of the \(\varphi_i\), assume that \(\varphi_i(0)=0\). If we put
\[ Q=(Q_0-x^*)\times(A-\alpha^*),\quad B=\{z:z\in W,\ \varphi_i(z)<0,\ i=0,1,\ldots,\mu\}\cup\{0\}, \]
then we obtain that \(0\) is a \((Q,B,\hat\varphi)\)-extremal and that the set
\[ Z=\{x:x\in J,\ l_i(x)<0,\ i=0,1,\ldots,\mu\}\cup\{0\} \]
is an interior cone for \(B\) at \(0\). Using the main result of [2], it is easy to show that \(Q\) admits at zero a convex first-order approximation.

If the “differentials” \(l_0,l_1,\ldots,l_\mu\) are linear and \(\bar l\) is a linear functional on \(J_\mu\) such that \(\bar l(y)\ge 0\) for any \(y\in Z\), then
\[ \bar l=\sum_{i=0}^{\mu}\alpha_i l_i,\qquad \alpha_i\le 0. \]
Using Theorems 1 and 2, we conclude that in the case under consideration

there exist real numbers \(\alpha_i,\ i=0,1,\ldots,\mu\), and a vector \(\hat a\in R^m\) (not all equal to zero), such that

\[ \hat a\cdot l(x)+\sum_{i=0}^{\mu}\alpha_i l_i(x)\leq 0,\qquad x\in K,\qquad \alpha_i\leq 0,\qquad i=0,1,\ldots,\mu . \tag{4} \]

It is easy to show that (4) contains Pontryagin’s maximum principle as a special case.

If \(\mu>0\), \(\varphi_i=\sup_{t\in I} g(x(t))\), \(i=1,\ldots,\mu\), where \(g\) is a function of class \(C^1\) from \(R^n\) to \(R^1\), then the continuous convex functionals

\[ l_i=\sup_{t\in T}[\operatorname{grad} g(x^*(t))]\cdot x(t),\qquad T=\{t:t\in I,\ g(x(t))=0\}, \tag{5} \]

\(i=1,\ldots,\mu\), satisfy condition (2), and Theorems 1 and 2 give necessary conditions for optimal problems with constrained phase coordinates, analogous to the conditions in \((^1)\), Chap. VI and \((^{3,4})\). It can also be shown that formula (4) is valid in this case as well and gives new necessary conditions for optimality. If (5) is also valid for \(i=0\), we obtain necessary conditions for minimax problems. Similar considerations make it possible to obtain necessary conditions for discrete optimal processes \((^5)\) and for optimal processes with impulsive control.

In conclusion, let us note that optimal processes from a similar point of view were recently considered by A. Ya. Dubovitskii and A. A. Milyutin \((^7)\), and our results in many respects supplement the results contained in \((^7)\).

University of Southern California
Los Angeles, USA

Received
10 III 1966

REFERENCES

\(^1\) L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, The Mathematical Theory of Optimal Processes, Moscow, 1961.
\(^2\) R. V. Gamkrelidze, J. Soc. Ind. and Appl. Math. on Control, Ser. A, 3, Philadelphia, 106 (1965).
\(^3\) J. W. Warga, Trans. Am. Math. Soc., 112, 432 (1964).
\(^4\) J. W. Warga, Michigan Math. J., 12, 289 (1965).
\(^5\) B. W. Jordan, E. Polak, J. Soc. Ind. and Appl. Math. on Control, Ser. A, 2, Philadelphia, 332 (1964).
\(^6\) L. W. Neustadt, ibid., 3, 317 (1965).
\(^7\) A. Ya. Dubovitskii, A. A. Milyutin, Zhurn. vychislit. matem. i matem. fiz., 5, No. 3, 395—453.