MATHEMATICS

Corresponding Member of the Academy of Sciences of the USSR A. N. TIKHONOV

ON METHODS OF REGULARIZATION OF OPTIMAL CONTROL PROBLEMS

Let a system of equations be given

\[ dx/dt=f(t,x,u),\quad (x=x_1,\ldots,x_n),\quad (u=u_1,\ldots,u_m),\quad t_0\leq t\leq T, \]

with control functions \(u(t)\) from some complete functional class \(U\) and with initial conditions

\[ x(t_0)=u_0 \]

and a continuous nonnegative functional \(F[x]\), defined on functions \(x(t)\) given for \(t_0\leq t\leq T\) \((^{1,2})\) (problem (1)).

Assume that in the class \(U\) there exists an optimal control, i.e., that there exists a function \(\bar u^{(0)}(t)\) such that \(x(t,\bar u^{(0)})\) realizes the minimum of the functional \(F[x]\)

\[ \inf_{u\in U} F[x(u)] = F_0. \]

Let us consider the problem of approximately determining \(\bar u^{(0)}(t)\).

For the approximate determination of the optimizing control, the method of minimizing the functional \(F\) is widely used; in it, by means of some algorithm, a sequence of functions \(u_n(t)\) is computed such that

\[ F_n = F[x(u_n)] \underset{n\to\infty}{\longrightarrow} F_0; \]

in this case the function \(u_n(t)\), for which the value \(F_n\) is sufficiently close to \(F_0\), is treated as an approximation to \(\bar u^{(0)}(t)\).

In what follows we shall assume that we possess a method allowing us to construct minimizing sequences for the problems under consideration, and that \(U\) contains no isolated elements.

It is not difficult to see that the hypothesis that \(u_n(t)\) approximates \(\bar u^{(0)}(t)\) is false. Whatever the accuracy \(\varepsilon\), it is not difficult to find a control \(\bar u(t)\) such that

\[ F[x(\bar u)] \leq F_0+\varepsilon, \]

and such that the difference \(\bar u^{(0)}(t)-\bar u(t)\) can assume arbitrarily large values, admissible by virtue of the membership of \(\bar u(t)\) in the class \(U\).

Choose \(\bar u(t)\) coinciding with \(\bar u^{(0)}(t)\) everywhere except on a small interval \((t_1-\eta,\ t_1+\eta)\) about some point \(t_1\), on which the difference \(\bar u(t)-\bar u^{(0)}(t)\) is made to exceed some fixed number \(M_0\), admissible by the class \(U\). Obviously, for any accuracy \(\delta\) the quantity \(\eta\) can be chosen so that the difference \(|x(t)-\bar x^{(0)}(t)|\leq \delta\). Choosing \(\eta\), and hence also \(\delta\), sufficiently small, we shall have that \(\bar F\leq F+\varepsilon\).

Variational problems for which there exist minimizing sequences of functions that do not converge uniformly to the extremal solution will be called ill-posed variational problems. Thus, the optimal control problem posed above will be an ill-posed variational problem.

The purpose of the present article is to construct regularizing algorithms for finding an optimal control, i.e., such algorithms that the minimizing sequences constructed with their aid will converge to \(\bar u^{(0)}(t)\).

Consider the smoothing functional

\[ G^\alpha[u]=F[x(u)]+\alpha\Omega[u], \]

where \(\Omega[u]\) is a regularizing functional; we choose, for example, in the form

\[ \Omega[u]=\int_{t_0}^{T}\sum_{i=1}^{m}\{k_1(t)(u_i')^2+k_0(t)(u_i)^2\}\,dt,\qquad k_1(t)>0,\ k_0(t)>0. \]

The functional \(G^\alpha[u]\) is nonnegative, and therefore its lower bound \(G_0^\alpha\) exists. Consider some decreasing sequence of numbers \(\alpha_k\to 0\) and some controls \(\hat u^{\alpha_k}(t)\) for which

\[ G^{\alpha_k}[\hat u^{\alpha_k}(t)]\leqslant G_0^{\alpha_k}+\alpha_k c, \]

where \(c\) is a constant independent of \(\alpha\).

Theorem 1. If there exists a unique optimal control \(u^{(0)}(t)\) of problem (1), which is a smooth function, then the sequence of functions \(\hat u^{\alpha_k}(t)\) satisfying the conditions

\[ G^{\alpha_k}[\hat u^{\alpha_k}]\leqslant G_0^{\alpha_k}+\alpha_k c, \]

converges uniformly to \(\bar u^{(0)}(t)\).

It is obvious that

\[ G_0^{\alpha_k}\leqslant F[x(\bar u^{(0)})]+\alpha_k\Omega[\bar u^{(0)}]=F_0+\alpha_k c_0\quad (c_0=\Omega[\bar u^{(0)}]). \]

It follows from this that

\[ G_0^{\alpha_k}[\hat u^{\alpha_k}] =F[x(\hat u^{\alpha_k})]+\alpha_k\Omega[\hat u^{\alpha_k}] \leqslant G_0^{\alpha_k}+\alpha_k c \leqslant F_0+\alpha_k(c_0+c) \]

and that

\[ \alpha_k\Omega[\hat u^{\alpha_k}] \leqslant F_0-F[x(\hat u^{\alpha_k})]+\alpha_k(c_0+c) \leqslant \alpha_k(c_0+c), \]

since

\[ F_0-F[x(\hat u^{\alpha_k})]\leqslant 0. \]

Thus,

\[ \Omega[\hat u^\alpha]\leqslant c_0+c, \]

and the totality of functions \(\{\hat u^{\alpha_k}(t)\}\) forms a compact family. Let a subsequence \(\hat u^{\alpha_k}(t)\) converge uniformly to a function \(\bar u(t)\). It is obvious that

\[ \lim_{k\to\infty} F[x(\hat u^{\alpha_k})]+\alpha_k\Omega[\hat u^{\alpha_k}] =F[x(\bar u)]=F_0 \]

and that, by virtue of the uniqueness of the optimal control, \(u(t)=\bar u^{(0)}(t)\).

Remark 1. If there exists at least one optimal control belonging to the class \(W_2'\), then a convergent subsequence of \(\hat u^{\alpha_k}\) will converge to one of the optimal controls.

Remark 2. Theorem 1 holds if \(U\) contains a subset \(\bar U\), allowing a new metric \(\rho_1(\bar u_1,\bar u_2)\), majorized with respect to the metric \(C\), and such that \(\bar S_c(\bar u)=\{\bar u:\rho(\bar u,O)\leqslant c\}\) (\(O\) is some fixed element in \(\bar U\)) is compact in \(U\). In this case, putting

\[ \Omega[\bar u]=\rho_1^2(\bar u,O), \]

we obtain convergence of the minimizing sequence if there exists an optimal control \(\bar u^{(0)}\in\bar U\) (3).

Denote by $\overline U$ the set of elements $\overline u \in U$ for which $\Omega[\overline u]$ is defined, and suppose that $\overline U$ is a convex complete set in the Hilbert norm $(\Omega[\overline u])^{1/2}$.

Theorem 2. If $\overline U$ is the set of elements of $U$ on which $\Omega[u]$ is defined, convex and complete, then there exists at least one function $u^\alpha(t) \in \overline U$ realizing the minimum of the functional

\[ G^\alpha[\overline u]=F[x(\overline u)]+\alpha\Omega[\overline u]\qquad(\overline u\in\overline U). \]

Indeed, let the sequence of minimizing functions $u_n^\alpha$ converge uniformly to the function $\overline u(t)\in U$. We shall show that $\overline u\in\overline U$ and that

\[ \Omega[u_n^\alpha(t)-\overline u(t)]\to 0\qquad(n\to\infty). \]

For this it is enough to prove that the sequence $u_n^\alpha(t)$ is fundamental:

\[ \Omega[u_n^\alpha-u_m^\alpha]\to 0\qquad(n,m\to\infty). \]

If this is not so, then there exists an $\varepsilon_0$ such that, for an infinite sequence of indices,

\[ \Omega[u_n^\alpha-u_{n+p_n}^\alpha]\geqslant\varepsilon_0. \]

Put $\zeta_n=u_n^\alpha-u_{n+p_n}^\alpha$ and $\xi_n=\frac12(u_n^\alpha+u_{n+p_n}^\alpha)$, so that $\xi_n=u_n^\alpha-\frac12\zeta_n=u_{n+p_n}^\alpha+\frac12\zeta_n$.

Further, since $u_n^\alpha\Rightarrow \overline u$, we also have $\xi_n\Rightarrow \overline u$, and, by the continuity of the functional $F[x]$,

\[ F[x(\xi_n)]-F[x(u_n^\alpha)]=\eta_n'\to0;\qquad F[x(\xi_n)]-F[x(u_{n+p_n}^\alpha)]=\eta_n''\to0. \]

Obviously, for the functions $\xi_n$ we shall have

\[ G^\alpha[\xi_n]\geqslant G_0^\alpha\geqslant G^\alpha[u_n^\alpha]-\varepsilon_1 \qquad\text{for } n\geqslant n(\varepsilon_1), \]

i.e.

\[ F[x(\xi_n)]+\alpha\{\Omega[u_n^\alpha]-\Omega[u_n^\alpha,\xi_n]+\tfrac14\Omega[\zeta_n]\}\geqslant \]

\[ \geqslant F[x(u_n^\alpha)]+\alpha\Omega[u_n^\alpha]-\varepsilon_1 \]

or

\[ \alpha\{-\Omega[u_n^\alpha,\xi_n]+\tfrac14\Omega[\zeta_n]\}\geqslant F[x(u_n^\alpha)]-F[x(\xi_n)]-\varepsilon_1\geqslant \]

\[ \geqslant-\varepsilon_1-|\eta_n'|. \]

Similarly, using the representation $\xi_n=u_{n+p_n}^\alpha+\frac12\zeta_n$, we obtain

\[ \alpha\{\Omega[u_{n+p_n}^\alpha,\xi_n]+\tfrac14\Omega[\zeta_n]\}\geqslant -\varepsilon_1-|\eta_n''|, \]

whence

\[ \alpha\{-\Omega[u_n^\alpha-u_{n+p_n}^\alpha,\xi_n]+\tfrac12\Omega[\zeta_n]\}\geqslant -2\varepsilon_1-|\eta_n'|-|\eta_n''| \]

or

\[ \Omega[\zeta_n]\leqslant \frac{2}{\alpha}\,[2\varepsilon_1+|\eta_n'|+|\eta_n''|], \]

which, for a suitable choice of $\varepsilon_1$, contradicts the assumption

\[ \Omega[\zeta_n]\geqslant\varepsilon_0\qquad(\varepsilon_1<\tfrac12\alpha\varepsilon_0) \]

for an infinite sequence of indices $n$.

Following the basic idea of the proof of Theorem 1, it is not difficult to see that the functions $u^\alpha(t)$ realizing the minimum of $G^\alpha[u]$ are such that $u^\alpha(t)\Rightarrow \overline u^{(0)}$, if $\overline u^{(0)}(t)$ is the unique optimal control.

The question of stability of the problem with respect to small perturbations of $f$ and $F$ is investigated analogously.

Received
13 III 1965

REFERENCES

1. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, The Mathematical Theory of Optimal Processes, Moscow, 1961.
2. R. Bellman, Dynamic Programming, Moscow, 1960.
3. A. N. Tikhonov, DAN, 151, No. 3 (1963); 161, No. 5 (1965).