Mathematics

Yu. V. EGOROV

ON SOME PROBLEMS IN THE THEORY OF OPTIMAL CONTROL

(Presented by Academician L. S. Pontryagin, March 15, 1962)

In this note some problems are considered in the theory of optimal control of processes that are described by partial differential equations. Questions of this kind have been studied in \((^{1-5})\). Various possible approaches to their solution were discussed at O. A. Oleinik’s seminar at Moscow University. In the present paper, for a number of problems, questions of the existence of an optimal control and its uniqueness are resolved, and methods are given for finding it in practice.

1. Let us call a measurable function \(p(t)\), defined on the interval \(0 \leq t \leq T\), a control if \(|p(t)| \leq 1\) (almost everywhere).

We shall call a control \(p(t)\) \(I\)-optimal if the solution of the equation

\[ \frac{\partial u(t,x)}{\partial t}=\frac{\partial^{2}u(t,x)}{\partial x^{2}} \tag{1} \]

in the domain \(Q=\{0 \leq t \leq T,\ 0 \leq x \leq 1\}\), with boundary conditions

\[ u(0,x)=0,\qquad \frac{\partial u(t,0)}{\partial x}=0,\qquad \frac{\partial u(t,1)}{\partial x}=\alpha[p(t)-u(t,1)] \quad (\alpha>0\ \mathrm{const}) \tag{2} \]

minimizes the functional

\[ I(p)=\int_{0}^{1}[u(T,x)-u_{0}(x)]^{2}\,dx, \tag{3} \]

where \(u_{0}(x)\) is a given function from \(L_{2}(0,1)\).

Theorem 1. An \(I\)-optimal control always exists. Moreover, the last of the boundary conditions (2) is satisfied in the following sense:

\[ \lim_{\varepsilon\to 0}\int_{0}^{T} \left[ \frac{\partial u(t,1-\varepsilon)}{\partial x} +\alpha u(t,1-\varepsilon)-\alpha p(t) \right]\varphi(t)\,dt=0 \]

for every \(\varphi(t)\in C_{0}^{(\infty)}(0,T)\).

Proof. From a minimizing sequence of controls \(\{p_n(t)\}\) one can choose a subsequence, weakly convergent in \(L_2(0,T)\), \(p_{n_k}(t)\to p(t)\), in such a way that the corresponding sequence \(u_{n_k}(t,x)\) of solutions of problem (1)—(2) converges for \(0 \leq x < 1,\ 0 \leq t \leq T\), together with derivatives with respect to \(t\) and \(x\) of arbitrary order. Using S. N. Bernstein’s estimates \((^6)\), it is easy to show that this convergence is uniform in the domain \(\{0 \leq x \leq 1-\varepsilon,\ 0 \leq t \leq T\}\) \((\varepsilon>0)\). It is not hard to see that \(p(t)\) is an \(I\)-optimal control and \(u(t,x)=\lim u_{n_k}(t,x)\) is the corresponding solution of problem (1)—(2).

But this \(I\)-optimal control is, generally speaking, not unique. Let \(f(t)\) be an infinitely differentiable function equal to zero for

\(t \leqslant 0\) and \(t \geqslant 1\), with \(f(t) \geqslant 0\), \(f(t) \not\equiv 0\), and \(|f^{n}(t)| \leqslant A^n \Gamma(3/2 n)\) (see (7)). If an \(I\)-optimal control \(p(t)\) is such that \(|p(t)| < 1-\delta\) \((\delta>0)\) for \(t \in [t_1,t_2]\) \((0 \leqslant t_1 < t_2 \leqslant T)\), then the control

\[ p_1(t)=p(t)+\gamma \sum_{n=0}^{\infty}\frac{d^n}{dt^n}\left(\frac{t-t_1}{t_2-t_1}\right)\left[\frac{1}{\alpha(2n-1)!}+\frac{1}{(2n)!}\right] \tag{4} \]

for sufficiently small \(\gamma\) is also \(I\)-optimal.

Let \(\{\lambda_k\}\) be the sequence of positive roots of the equation \(\lambda \operatorname{tg}\lambda=\alpha\). Multiply equation (1) by \(e^{\lambda_k^2 t}\cos \lambda_k x\) and integrate over \(Q\). Taking (2) into account, we obtain:

\[ \int_0^1 u(T,x)\cos\lambda_k x\,dx = \alpha \cos\lambda_k \int_0^T p(t)e^{\lambda_k^2(t-T)}\,dt \qquad (k=1,2,\ldots). \tag{5} \]

The system \(\{\cos \lambda_k x\}\) is complete in \(L_2(0,1)\), whereas the system \(\{e^{\lambda_k^2(t-T)}\}\) is incomplete on any interval (see, for example, \((^8)\)). From (5) it is clear that, in order that \(u(T,x)=0\), it is necessary and sufficient that

\[ \int_0^T p(t)e^{\lambda_k^2(t-T)}\,dt=0 \]

\((k=1,2,\ldots)\). Such \(p(t)\) have an even number of sign changes on \((0,T)\). The condition \(|p(t)|\equiv 1\) also does not guarantee uniqueness. If an \(I\)-optimal control \(p(t)\) is such that \(|p(t)|\equiv 1\), then it will be nonunique if and only if there exists a control \(p_1(t)\) such that

\[ p(t)p_1(t)\geqslant 0,\qquad p_1(t)\not\equiv 0 \quad\text{and}\quad \int_0^T p_1(t)e^{\lambda_k^2(t-T)}\,dt=0 \qquad (k=1,2,\ldots). \]

Consequently, if an \(I\)-optimal control \(p(t)\) is piecewise constant and \(|p(t)|\equiv 1\), then it is unique.

This problem was considered in \((^{4,5})\). We note that the assertion of paper \((^4)\) that, in the case where \(u_0(x)=\mathrm{const}\), every \(I\)-optimal control \(p(t)\) is such that \(|p(t)|\equiv 1\), is inaccurate. There exist controls \(p(t)\) for which \(u(t,x)\) becomes a constant in finite time—it suffices to take

\[ p(t)=\gamma\int_0^t f(\xi)\,d\xi + \gamma \sum_{n=1}^{\infty} f^{(n-1)}(t) \left[\frac{1}{\alpha(2n-1)!}+\frac{1}{(2n)!}\right], \]

where \(f(t)\) is the same as in (4), and \(\gamma\) is such that \(|p(t)|<1\). To this control there corresponds the solution of problem (1)—(2):

\[ u(t,x)=\gamma\int_0^t f(\xi)\,d\xi + \gamma \sum_{n=1}^{\infty}\frac{f^{(n-1)}(t)x^{2n}}{(2n)!}, \]

and

\[ u(1,x)=\gamma\int_0^1 f(\xi)\,d\xi=\mathrm{const}, \]

although \(|p(t)|\not\equiv 1\). In paper \((^5)\) the assertion concerning the uniqueness of the \(I\)-optimal control is incorrect (as proved above).

Let us also note that Theorem 1 remains valid if the functional (3) is replaced by a functional of the form

\[ \max_{0\leqslant x\leqslant 1}|u(T,x)-u_0(x)|,\quad \|u(T,x)-u_0(x)\|_{L_p(0,1)}, \]

\[ \|u(T,x)-u_0(x)\|_{W_p^{(l)}(0,1)} \quad\text{(for notation see \((^{10})\)),}\quad \|u(T,x)-u_0(x)\|_{C^{(k)}[0,1]} \]

\((k,l=1,2,\ldots;\ p\geqslant 1)\). In spaces with a uniformly convex sphere, the values \(u(T,x)\) for an \(I\)-optimal control are determined uniquely.

1. Consider the problem in which the functional (3) is replaced by the functional

\[ I_\gamma(p)=\int_0^1 [u(T,x)-u_0(x)]^2\,dx+\gamma\int_0^T p^2(t)\,dt \quad (\gamma>0). \]

Here the \(I_\gamma\)-optimal control \(p^*(t)\) exists and is unique. The practical determination of this control can be reduced to the problem of finding the extremum of a function of many variables, if at the \(n\)-th step one seeks the control \(p_n^*(t)\) that is \(I_\gamma\)-optimal in the class of step functions \(p_n(t)\), equal to the constant \(p_{nk}\) for

\[ t\in\left(\frac{k-1}{n}T,\frac{k}{n}T\right)\quad (k=1,2,\ldots,n). \]

From (5) we have

\[ u(T,x)=\sum_{k=1}^{\infty} a_k \int_0^T p(t)e^{\lambda_k^2(t-T)}\,dt\,\cos\lambda_k x . \]

Consequently, \(I_\gamma(p_n)=F(p_{n1},\ldots,p_{nn})\) is a function of \(n\) variables.

If \(p_n^*\) is its point of minimum in the domain \(|p_{nk}|\leq 1\), \((k=1,\ldots,n)\) (it is not difficult to see that such a point is unique), then

\[ I_\gamma(p_n)\geq I_\gamma(p_n^*)\geq I_\gamma(p^*). \]

From the inequality

\[ \gamma\|p_n^*-p^*\|_{L_2(0,T)}^2 \leq 2I_\gamma(p^*)+2I_\gamma(p_n^*)-4I_\gamma\left(\frac{p^*+p_n^*}{2}\right) \]

it follows that \(\|p_n^*-p^*\|_{L_2(0,T)}\to 0\) \((n\to\infty)\).

1. In the domain \(Q\) consider the solution of equation (1) with the boundary conditions

\[ \frac{\partial u(t,0)}{\partial x}=\beta[u(t,0)-f_0(t)],\qquad \frac{\partial u(t,1)}{\partial x}=\alpha[f_1(t)-u(t,1)],\qquad u(0,x)=p(x) \]

(\(\alpha\) and \(\beta\) are positive constants).

The problem consists in finding a measurable function \(p(x)\), \(|p(x)|\leq 1\), for which the functional (3) assumes its minimum value. This problem always has a unique solution. Its determination can also be reduced to the problem of finding the minimum of a function of many variables, if one seeks controls optimal in the class of step functions.

Other variants of this problem are possible. The functional (3) may be replaced by the functional

\[ \int_M [u(T,x)-u_0(x)]^2\,dx \quad (M\subset [0,1],\ \operatorname{mes} M>0) \]

or even by the functional

\[ \sum_{k=1}^{\infty} [u(T,x_k)-u_k]^2, \]

where \(x_k\in[0,1]\), \(x_k\ne x_l\) \((k\ne l)\), and the set \(\{x_k\}\) has a limit point in \((0,1)\). The uniqueness of the solution of these problems is a consequence of the analyticity of \(u(t,x)\) with respect to \(x\) and, for example, of the results of paper (9).

1. Theorem 2. Let \(u_0(x)\in L_2(0,1)\), and let \(S\) be the set of functions \(v(x)\) in \(L_2(0,1)\) for which

\[ \int_0^1 [u_0(x)-v(x)]^2\,dx\leq \delta^2. \]

Suppose that to some control \(p_1(t)\) there corresponds a solution \(u_1(t,x)\) of problem (1)—(2), and \(u_1(t_1,x)\in S\). Then there exists a control \(p^*(t)\), to which there corresponds a solution \(u^*(t,x)\) of problem (1)—(2) such that \(u^*(T,x)\in S\) and the time \(T\) is minimal. This control is unique and \(|p^*(t)|=1\) (almost everywhere in \((0,T)\)).

The existence of the optimal control \(p^*(t)\) is proved essentially in the same way as in Theorem 1 (this also applies to the following Theorems 3 and 4).

Let us show that \(|p^*(t)|=1\) (almost everywhere in \((0,T)\)). To each control \(p(t)\) on the interval \((0,T)\) there corresponds a vector \(x=(x_1,\ldots,x_k,\ldots)\in l_2\), where

\[ x_k=\alpha\cos\lambda_k\left(\int_0^1 \cos^2\lambda_k \xi\,d\xi\right)^{-1/2} \int_0^T p(\xi)e^{\lambda_k^2(\xi-T)}\,d\xi . \]

The set \(X\) of all such...

of vectors is convex and has one common point \(x^*\) with the ball \(\|x-x_0\|_{l_2}\leqslant \delta\), where
\[ x^0=(x_1^0,x_2^0,\ldots,x_k^0,\ldots),\qquad x_k^0=\left(\int_0^1 \cos^2\lambda_k\xi\,d\xi\right)^{-1/2} \int_0^1 u_0(\xi)\cos\lambda_k\xi\,d\xi . \]

There exists a nonzero vector \(f\in l_2\) for which
\[ \sum_{k=1}^{\infty} f_k(x_k^*-x_k)\geqslant 0, \]
\(x\in X\) (see (11)). From this it is not difficult to derive, analogously to how this is done in (12), that \(|p^*(t)|=1\) for almost all \(t\in(0,T)\). If \(p_1^*(t)\) and \(p_2^*(t)\) are optimal controls, then \(\frac12[p_1^*(t)+p_2^*(t)]\) is also such a control, but from
\[ |p_1^*(t)|=|p_2^*(t)|=\frac12\,|p_1^*(t)+p_2^*(t)|=1 \]
it follows that \(p_1^*(t)=p_2^*(t)\).

Theorem 3. Let \(u_0(x)\) be a function on \((0,1)\) such that, for some control \(p_1(t)\), the solution of problem (1)—(2) coincides with \(u_0(x)\) at \(t=t_1\), and
\[ \left|\int_0^1 u_0(x)\cos\lambda_k x\,dx\right| \leqslant C e^{-c_0\lambda_k^{1+2\varepsilon}} \qquad (C>0,\ c_0>0,\ \varepsilon>0). \]
Then there exists a control \(p^*(t)\) for which the solution of problem (1)—(2) coincides with \(u_0(x)\) at the time \(T\), and the time \(T\) is minimal. This control is unique and \(|p^*(t)|=1\) (almost everywhere on \((0,T)\)).

Let \(B\) be the Banach space of vectors \(b=(b_1,\ldots,b_n,\ldots)\) such that
\[ b_n e^{c_0\lambda_n^{1+\varepsilon}}\to 0 \quad\text{as } n\to\infty \]
(\(c_0>0\) and \(\varepsilon>0\) fixed), with norm
\[ \|b\|_B=\sup_n |b_n e^{c_0\lambda_n^{1+\varepsilon}}|. \]
Consider the set \(A\) of sequences
\[ a=(a_1,a_2,\ldots,a_n,\ldots) \]
such that
\[ a_n=\int_0^T p(\xi)e^{\lambda_n^2(\xi-T)}\,d\xi \quad (|p(t)|\leqslant 1) \]
and \(a\in B\).

It is not difficult to see that \(A\) is a convex closed set in \(B\). It can be shown, using the construction of the work (13), that \(A\) contains the ball \(\|b\|_B<\rho\). Therefore (see (11)) there exists a linear functional \(f\in B^*\) for which
\[ (f,a^*)\geqslant (f,a) \]
for \(a\in A\), where \(a^*\) is the vector corresponding to \(p^*(t)\). Hence it follows that \(|p^*(t)|=1\) (almost everywhere on \((0,T)\)).

Theorem 4. Let \(u_0(x)\) be a function on \((0,1)\) such that there exists \(p_1(t)\in W_2^{(1)}(0,t_1)\) for which the solution of problem (1)—(2) coincides with \(u_0(x)\) at \(t=t_1\) and
\[ \|p_1\|_{W_2^{(1)}(0,t_1)}\leqslant a_0 . \]
Then there exists a function \(p^*(t)\) for which the solution of problem (1)—(2) coincides with \(u_0(x)\) at \(t=T\),
\[ \|p^*\|_{W_2^{(1)}(0,T)}\leqslant a_0, \]
and the time \(T\) is minimal. This function \(p^*(t)\) is unique and
\[ \|p^*\|_{W_2^{(1)}(0,T)}=a_0. \]

In the formulation of Theorem 4 the space \(W_2^{(1)}\) may be replaced by \(L_q\) (\(q>2\)) or \(W_p^{(l)}\) (\(p>1,\ l=1,2,\ldots\)).

The author expresses gratitude to O. A. Oleinik for her attention to this work.

Moscow State University
named after M. V. Lomonosov

Received
14 III 1962

References

1. E. F. Mishchenko, L. S. Pontryagin, Izv. AN SSSR, Ser. Mat., 25, No. 4 (1961).
2. I. V. Girsanov, DAN, 136, No. 4 (1961).
3. A. G. Butkovskii, A. Ya. Lerner, DAN, 134, No. 4 (1960).
4. A. G. Butkovskii, Avtomatika i telemekh., 22, No. 1 (1961).
5. A. G. Butkovskii, Avtomatika i telemekh., 22, No. 12 (1961).
6. S. N. Bernstein, Collected Works, 3, Publishing House of the Academy of Sciences of the USSR, 1960.
7. S. Mandelbrojt, Quasi-analytic Classes of Functions, Moscow, 1937, p. 24.
8. N. I. Akhiezer, Lectures on the Theory of Approximation, 1947, p. 52.
9. J. Lions, B. Malgrange, Math. Scand., 8, No. 2 (1960).
10. S. L. Sobolev, Some Applications of Functional Analysis in Mathematical Physics, Leningrad, 1950.
11. N. Bourbaki, Topological Vector Spaces, IL, 1959, pp. 94–95.
12. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, The Mathematical Theory of Optimal Processes, 1961.
13. B. S. Mityagin, DAN, 138, No. 2 (1961).