UDC 519.9

MATHEMATICS

Yu. N. Kiselev

ASYMPTOTIC SOLUTION OF THE PROBLEM OF OPTIMAL TIME PERFORMANCE FOR CONTROL SYSTEMS CLOSE TO LINEAR ONES

(Presented by Academician L. S. Pontryagin, December 11, 1967)

A controlled object is considered whose motion in the phase space \(X^n\) of variables \(x^1,\ldots,x^n\) is described by the differential equation

\[ \dot{x}=A(t)x+B(t)u+\varepsilon c(t,x) \tag{1} \]

with small parameter \(\varepsilon\), where \(x=(x^1,\ldots,x^n)\), \(u=(u^1,\ldots,u^r)\), the matrices \(A(t)\), \(B(t)\) are defined and continuous for \(t\geq t_0\), and the functions \(c(t,x)\), \(\partial c(t,x)/\partial x^j\), \(j=1,\ldots,n\), are defined and continuous on the direct product \([t_0,\infty)\times X^n\).

The main problem is to carry out, optimally in the sense of time performance, the transfer according to law (1) from the point \(X_*(\varepsilon)=x_0+\varepsilon x_1+o(\varepsilon)\in X^n\) at \(t=t_0\) to the origin of the coordinate system of the space \(X^n\). We shall assume here that \(x_0\ne 0\), and that the class \(D_m\) of admissible controls consists of measurable (for fixed \(\varepsilon\)) functions \(u(t,\varepsilon)\), whose values belong to the unit cube \(U=\{u:\ |u^\rho|\leq 1,\ \rho=1,\ldots,r\}\). In this note an asymptotic solution of the main problem is given.

Theorem 1. If in the class \(D_m\) there exists at least one control \(u(t,\varepsilon)\), \(t_0\leq t\leq t(\varepsilon)\), \(0<\varepsilon\leq\varepsilon_0\), which transfers the phase point moving according to law (1) from the position \(X_*(\varepsilon)\) to the origin of the coordinate system of the space \(X^n\), and moreover \(t(\varepsilon)\leq t_1\), \(0<\varepsilon\leq\varepsilon_0\), where \(t_1\) is a number independent of \(\varepsilon\), then in the class \(D_m\) there also exists an optimal control \(u_{\mathrm{opt}}(t,\varepsilon)\), \(t_0\leq t\leq t_{\mathrm{opt}}(\varepsilon)\), carrying out the indicated transfer, for all sufficiently small \(\varepsilon\).

The necessary condition of optimality \((^1)\) for the main problem can be written in a compact analytic form as three relations

\[ \dot{x}=A(t)x+B(t)u+\varepsilon c(t,x), \tag{2} \]

\[ \dot{\psi}=-\psi[A(t)+\varepsilon\,\partial c(t,x)/\partial x], \tag{3} \]

\[ \psi(t,\varepsilon)B(t)u(t,\varepsilon)=\max_{u\in U}\psi(t,\varepsilon)B(t)u; \tag{4} \]

where \(\psi\) is an auxiliary \(n\)-row. We prescribe for the system (2)—(4) the initial conditions

\[ x(t_0,\varepsilon)=X_*(\varepsilon),\qquad \psi(t_0,\varepsilon)=\Psi^*(\varepsilon), \tag{5} \]

where \(\Psi^*(\varepsilon)=\psi^0+\varepsilon\psi^1+o(\varepsilon)\); \(\psi^0,\psi^1\) are constant \(n\)-rows; \(\psi^0\ne 0\). The problem (2)—(5) is studied under fulfillment of the following three conditions:

I. The matrix \(A(t)\) belongs to the class \(C[t_0,t_1]\), and the matrix \(B(t)\) to the class \(C^1[t_0,t_1]\).

II. Denote by \(\hat{\psi}(t)\), \(t_0\leq t\leq t_1\), the solution of the equation \(\dot{\psi}=-\psi A(t)\) with the initial condition \(\hat{\psi}(t_0)=\psi^0\). Each of the functions \(a_\rho(t)=\hat{\psi}(t)b_\rho(t)\), \(\rho=1,\ldots,r\), where \(b_\rho(t)\) is a column of the matrix \(B(t)\), has on the interval \(t_0\leq t\leq t_1\) only a finite number of zeros \(\tau_{\rho s}\), \(s=1,\ldots,s_\rho\), \(t_0<\tau_{\rho 1}<\cdots<\tau_{\rho s_\rho}<t_1\), and these zeros are simple, i.e. \(\dot{a}_\rho(\tau_{\rho s})\ne 0\).

III. The functions \(c(t,x)\), \(\partial c(t,x)/\partial x^j\), \(j=1,\ldots,n\), are defined and continuous on the direct product \([t_0,t_1]\times X^n\).

Denote by \(x(t)\), \(\psi(t)\), \(\hat u(t)\), \(t_0\leq t\leq t_1\), the solution of the linear problem obtained from (2)—(5) for \(\varepsilon=0\). This solution exists and is determined uniquely (up to the values of the control \(u(t)\) at the switching points).

Theorem 2. If conditions I, II, III are satisfied, there exists a number \(\bar\varepsilon>0\) such that for every \(\varepsilon\in(0,\bar\varepsilon]\) problem (2)—(5) has on the interval \(t_0\leq t\leq t_1\) a unique solution \(x(t,\varepsilon)\), \(\psi(t,\varepsilon)\), \(u(t,\varepsilon)\), which we shall call the extremal triple. Here, as a function of \(t\), \(x(t,\varepsilon)\) is continuous and piecewise differentiable, the function \(\psi(t,\varepsilon)\) is continuously differentiable, the function \(u(t,\varepsilon)\) is piecewise constant and takes values only at the vertices of the cube \(U\). The control \(u(t,\varepsilon)\) is determined uniquely up to values at the switching points; it can be obtained from the control \(\hat u(t)\) by shifting the switching points \(\tau_{\rho s}\) to the points \(\tau_{\rho s}+\varepsilon d_{\rho s}(\varepsilon)\), where the quantities \(d_{\rho s}(\varepsilon)\) are bounded \((s=1,\ldots,s_\rho;\ \rho=1,\ldots,r)\).

Introduce the following notation: \(\Psi(t)=(\psi_j^i(t))\) is the fundamental matrix of the equation \(\dot\psi=-\psi A(t)\) with initial conditions \(\psi_j^i(t_0)=\delta_j^i\); \(\Phi(t)=(\varphi_j^i(t))\) is the fundamental matrix of the equation \(\dot x=A(t)x\) with initial conditions \(\varphi_j^i(t_0)=\delta_j^i\), \(i,j=1,\ldots,n\);

\[ \delta\psi(t)= \left[ \psi^1-\int_{t_0}^{t}\hat\psi(\tau) \frac{\partial c(\tau,\hat x(\tau))}{\partial x}\, \Psi^{-1}(\tau)\,d\tau \right]\Psi(t); \tag{6} \]

\[ \delta u^\rho(t,\varepsilon)= \begin{cases} -2\hat u^\rho(t), & t\in I_\rho(\varepsilon),\\ 0, & t\in [t_0,t_1]\setminus I_\rho(\varepsilon), \end{cases} \qquad (\rho=1,\ldots,r), \tag{7} \]

\[ \delta x(t)=\Phi(t)\left\{ x_1+\int_{t_0}^{t}\Phi^{-1}(\tau)c(\tau,\hat x(\tau))\,d\tau -\sum_{\rho,s:\,\tau_{\rho s}\leq t} \left[\Phi^{-1}(\theta)b_\rho(\theta)d_{\rho s} \bigl(\hat u^\rho(\theta+0)-\hat u^\rho(\theta-0)\bigr)\right]_{\theta=\tau_{\rho s}} \right\}, \tag{8} \]

\[ d_{\rho s}= -\left. \frac{\delta\psi(\theta)b_\rho(\theta)} {\dot\alpha_\rho(\theta)} \right|_{\theta=\tau_{\rho s}}, \]

where \(I_\rho(\varepsilon)=\displaystyle\bigcup_{s=1}^{s_\rho} I_{\rho s}(\varepsilon)\); \(I_{\rho s}(\varepsilon)\) are semi-intervals with endpoints \(\tau_{\rho s}\) and \(\tau_{\rho s}+\varepsilon d_{\rho s}\), closed on the left and open on the right; the control \(\hat u(t)=(\hat u^1(t),\ldots,\hat u^r(t))\) is regarded as right-continuous at discontinuity points. We shall call the asymptotic solution of problem (2)—(5) on the interval \(t_0\leq t\leq t_1\) the triple of functions

\[ \bar x(t,\varepsilon)=\hat x(t)+\varepsilon\delta x(t),\quad \bar\psi(t,\varepsilon)=\hat\psi(t)+\varepsilon\delta\psi(t),\quad \bar u(t,\varepsilon)=u(t)+\delta u(t,\varepsilon), \]

where the functions \(\delta x(t)\), \(\delta\psi(t)\) are defined by formulas (8), (6), and \(\delta u(t,\varepsilon)\) is the \(r\)-column with coordinates (7). Let a vector function \(\varphi(t)=(\varphi^1(t),\ldots,\varphi^k(t))\) be defined on some set \(J\) of the real line; set

\[ \|\varphi\|_{C_k[J]}=\sup_{t\in J}\max_{1\leq i\leq k}|\varphi^i(t)|,\qquad \|\varphi\|_{L_1^k[J]}=\int_J \max_{1\leq i\leq k}|\varphi^i(t)|\,dt. \]

Theorem 3. Let \(x,\psi,u\) be the exact solution of problem (2)—(5); let \(\bar x,\bar\psi,\bar u\) be the asymptotic solution of problem (2)—(5) on the interval \(t_0\leq t\leq t_1\). If conditions I, II, III are satisfied, the following relations hold:

\[ \|x-\bar x\|_{C_n\{[t_0,t_1]\setminus I(\varepsilon)\}}=o(\varepsilon), \qquad \|x-\bar x\|_{L_1^n\{I(\varepsilon)\}}=o(\varepsilon), \]

\[ \|\psi-\bar\psi\|_{C_n[t_0,t_1]}=o(\varepsilon), \qquad \|u-\bar u\|_{L_1^r[t_0,t_1]}=o(\varepsilon), \quad \text{where } I(\varepsilon)=\bigcup_{\rho=1}^{r} I_\rho(\varepsilon). \]

Suppose that the basic problem for \(\varepsilon=0\) is solvable, i.e., in the class \(D_m\) there exists an optimal control \(\hat u(t)\), \(t_0 \leqslant t \leqslant T\), transferring the object moving according to the linear law

\[ \dot x=A(t)x+B(t)u, \tag{9} \]

from the position \(x_0\) at \(t=t_0\) to the origin of the coordinate space \(X^n\) at \(t=T\). Let \(\hat x(t)\) be the corresponding optimal trajectory. By virtue of the maximum principle \((^1)\), there exists a nonzero solution \(\hat\psi(t)\) of the equation \(\dot\psi=-\psi A(t)\) such that almost everywhere on \([t_0,T]\) the relation
\[ \hat\psi(t)B(t)\hat u(t)=\max_{u\in U}\hat\psi(t)B(t)u \]
holds. Put \(\hat\psi(t_0)=\psi^0\). We may assume that the vector \(\psi^0\) has unit Euclidean length. If the nondegeneracy condition is fulfilled (for any \(n\)-row \(p\ne0\), each coordinate of the \(r\)-row \(p\Phi^{-1}(t)B(t)\) can vanish for \(t\geqslant t_0\) only on a set of measure zero), the optimal control for the linear systems (9) is unique up to the values it assumes on a set of measure zero, and the maximum principle is not only a necessary but also a sufficient condition for optimality. Thus, for linear nondegenerate systems of the form (9), the solution of the minimum-time problem of transfer from the position \(x_0\) to the origin reduces to finding such an initial value \(\psi^0\) of the variable \(\psi\) that the trajectory corresponding to it by virtue of the maximum principle, issuing at \(t=t_0\) from the point \(x_0\), reaches the origin. The synthesis problem for the optimal control (finding the indicated vector \(\psi^0\)) for equation (9) was first solved in \((^3)\); see also \((^{2,4})\).

Proceeding to the solution of the basic problem, we assume that the synthesis problem for equation (9) and the initial point \(x_0\) has been solved. Let the functions \(x(t)\), \(\psi(t)\), \(u(t)\), \(t_0 \leqslant t \leqslant t_1\), form an extremal triple (at the same time optimal) giving the solution of the basic problem for \(\varepsilon=0\) on the interval \(t_0 \leqslant t \leqslant T\), \(t_0<T<t_1\), and satisfying condition II*, which differs from condition II by the requirement that all zeros of the functions \(\alpha_\rho(t)\) on the interval \([t_0,t_1]\) belong to the interval \((t_0,T)\). Let \(p\) be a nonzero \(n\)-row. Denote by \(\psi(t,p)\) the solution of the equation \(\dot\psi=-\psi A(t)\) with the initial condition \(\psi(t_0,p)=p\), and by \(u(t,p)\) the control corresponding to the solution \(\psi(t,p)\) by virtue of the maximum principle. Consider the mapping

\[ \xi_T(p)=-\int_{t_0}^{T}\Phi^{-1}(\tau)B(\tau)u(\tau,p)\,d\tau,\qquad T\geqslant t_0,\quad p\ne0. \]

Let \(\Sigma_T\) be the image of the unit sphere under the mapping \(\xi_T(p)\). The mapping \(\xi_T(p)\) is continuous, and \(\Sigma_T\) is a closed bounded convex surface; see \((^2)\). Under conditions I, II* the mapping \(\xi_T(p)\) is continuously differentiable in some neighborhood of the point \(p=\psi^0\). The derivatives \(\partial \xi_T^i(p)/\partial p_j\) have been computed; moreover, it turns out that the rank of the functional matrix

\[ \partial \xi_T(p)/\partial p=\bigl(\partial \xi_T^i(p)/\partial p_j\bigr)_{i,j=1}^n \tag{10} \]

of the mapping \(\xi_T(p)\) does not exceed \(n-1\).

Introduce the following condition:

IV. The rank of the matrix (10) for \(p=\psi^0\) is equal to \(n-1\).

Problem 1. Determine the \(n\)-row \(\psi^1\) and the real number \(T_1\) in such a way that the asymptotic solution of problem (2)—(4) with initial conditions \(x(t_0,\varepsilon)=X_*(\varepsilon)\), \(\psi(t_0,\varepsilon)=\psi^0+\varepsilon\psi^1\) satisfies the terminal condition \(\bar x(T+\varepsilon T_1,\varepsilon)=o(\varepsilon)\).

Theorem 4. If conditions I, II, III, IV are fulfilled, Problem 1 is solvable. The set of its solutions is described as follows: the number \(T_1\) is determined uniquely, and the sought \(n\)-row depends on a free parameter \(\lambda\)*

and has the form $\psi_\lambda^1=\lambda\psi^0+\psi^1$, where $(\psi^1,\varphi^0)=0$, $-\infty<\lambda<\infty$, and the vector $\psi^1$ is determined uniquely. We do not write out the formulas for the corrections $T_1$, $\psi^1$ because of the limited size of the note. The trajectory and the control in the corresponding asymptotic solution do not depend on $\lambda$.

Problem 2. Prove the existence of such an $n$-tuple $\psi^1(\varepsilon)$ and such a scalar $T_1(\varepsilon)$, where $\psi^1(\varepsilon)$, $T_1(\varepsilon)$ are bounded functions of the parameter $\varepsilon$, that the solution of problem (2)—(4) with initial conditions $x(t_0,\varepsilon)=X_*(\varepsilon)$, $\psi(t_0,\varepsilon)=\psi^0+\varepsilon\psi^1(\varepsilon)$ satisfies the terminal condition $x(t+\varepsilon T_1(\varepsilon),\varepsilon)=0$ for sufficiently small $\varepsilon$.

Let $\delta$ be such a positive number that $|\psi^1(\varepsilon)|$, $|T_1(\varepsilon)|<\delta$. Put:
\[ \Pi=\{\chi=(\chi_1,\ldots,\chi_n):|\chi-\psi^1|<\delta\},\quad I=\{t:|T+\varepsilon T_1-t|<\varepsilon\delta\}, \]
where $\psi^1$, $T_1$ are defined in Theorem 4; $x(t,\varepsilon)-\bar x(t,\varepsilon)=\varepsilon R(t,\psi^1,\varepsilon)$.

Finally, let us introduce the following condition:

V. There exists a number $\varepsilon'>0$ such that for $0<\varepsilon\leq\varepsilon'$, $t\in I$, $\chi\in\Pi$ the function $R(t,\chi,\varepsilon)$ is continuous in the aggregate of the variables $(t,\chi)$ and satisfies the Lipschitz condition in the variable $\chi$ uniformly with respect to $t$ and $\varepsilon$.

Theorem 5. If conditions I, II*, III, IV and condition V are fulfilled, in which the Lipschitz constant is sufficiently small, then Problem 2, and hence also the main problem, is solvable for sufficiently small $\varepsilon$; moreover
\[ T_1(\varepsilon)=T_1+T_2(\varepsilon), \]
\[ \psi_\lambda^1(\varepsilon)=\lambda\psi^0+\psi^1+\psi^2(\varepsilon),\quad -\infty<\lambda<\infty, \]
where $\psi^1$, $T_1$ are defined in Theorem 4, $(\psi^2(\varepsilon),\psi^0)=0$, and the functions $T_2(\varepsilon)$, $\psi^2(\varepsilon)$ are determined uniquely, with $T_2(\varepsilon)$, $\psi^2(\varepsilon)\to0$ as $\varepsilon\to0$. If the linear system (9) is nondegenerate, then
\[ t_{\mathrm{opt}}(\varepsilon)=T+\varepsilon T_1+o(\varepsilon). \]

Definition. We shall call a control $u(t,\varepsilon)$, $t_0\leq t\leq t(\varepsilon)$, from a certain class $D$ of admissible controls asymptotically optimal in the class $D$ if: 1) the solution $x(t,\varepsilon)$ of equation (1) with the initial condition $x(t_0,\varepsilon)=X_*(\varepsilon)$ under $u=u(t,\varepsilon)$ satisfies the terminal condition $x(t(\varepsilon),\varepsilon)=o(\varepsilon)$; 2) $|t_{\mathrm{opt}}(\varepsilon)-t(\varepsilon)|=o(\varepsilon)$.

Theorem 6. In the class of piecewise-constant controls, the main problem has an asymptotically optimal control, which may be taken to be an extremal control.

Thus, for the class under study of control systems close to linear ones, the maximum principle is a sufficient condition for “asymptotic optimality.” Therefore one may speak of the problem of “asymptotic synthesis” for equation (1); a solution of this problem is given by Theorems 4, 5. Using this solution and the result of Theorem 3, one can find an asymptotic representation of the extremal process that is asymptotically optimal.

In conclusion the author expresses gratitude to V. G. Boltyanskii for posing the problem and for his attention to the work.

Mathematical Institute named after V. A. Steklov
Academy of Sciences of the USSR

Received
30 XI 1967

CITED LITERATURE

1. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, The Mathematical Theory of Optimal Processes, Moscow, 1961.
2. V. G. Boltyanskii, Mathematical Methods of Optimal Control, Moscow, 1966.
3. L. W. Neustadt, J. Math. Anal. and Appl., 1, No. 3—4, 484 (1960).
4. T. G. Babunashvili, DAN, 155, No. 2, 295 (1964).