UDC 517.91

MATHEMATICS

Yu. P. KRIVENKOV

LINEAR DYNAMIC PROGRAMMING WITH UNBOUNDED CONTROL

(Presented by Academician L. S. Pontryagin on 24 VI 1965)

In the study of optimal processes \((^{1,5})\) from the class of problems of linear dynamic programming \((^{2,6})\), it becomes necessary to consider problems in which the domain of admissible controls is unbounded. The latter requires the consideration of variational problems in the class of discontinuous functions of the phase coordinates and in the class of unbounded nonclassical functions describing the control.

In the present work, within the framework of the theory of generalized functions \((^3)\), sufficiency is proved and, on the basis of a new approach to problems of the calculus of variations \((^4)\), the necessity of the maximum principle is proved for the class of problems under consideration.

Let us consider the space \(K_n^l\) of test functions \(\varphi(t)\), i.e., the space of \(n\)-dimensional vector-functions consisting of finite (vanishing outside the interval \((T_1^*, T_2^*)\), where \(-\infty < T_1^* < T_2^* < \infty\)), \(l\)-times continuously differentiable functions.

We shall call a generalized function \(f(t)\) any linear continuous functional \((f(t), \varphi(t))\) defined on \(K_n^l\), and denote the class of such generalized functions by \(F_n^l\) or \(F^l\). If for \(f(t) \in F^l\) there exist an integer \(k\) \((k \le l)\) and, generally speaking, a summable, and in the present work integrable, function \(F(t)\) such that for any \(\varphi(t) \in K^l\)

\[ (f(t), \varphi(t)) = (-1)^k(F(t), \varphi^{(k)}(t)), \]

then the smallest \(k\) will be called the order of \(f(t)\), and we shall denote \(f(t) \in F^{l,k}\). We shall say that \(f \in F^{l,k}\) is nonnegative if, for every nonnegative \(\varphi \in K^l\), the values of the functional \((f,\varphi)\) are nonnegative. The concept of positivity of \(f \in F^{l,k}\) is defined correspondingly.

As the class of admissible controls \(U\) we take the class of generalized functions \(u(t)\) whose support belongs to the interval \([T_1,T_2]\), where \((T_1^* < T_1 < T_2 < T_2^*)\), and with values belonging to an \(r\)-dimensional, generally speaking unbounded, convex polyhedron \(U(x)\), defined by the inequalities:

\[ S(u,x) = Pu - Qx - R \le 0; \tag{1} \]

\[ G(u) = Mu - N \le 0; \tag{2} \]

\[ O(u) = -Eu \le 0, \tag{3} \]

where \(x(t) \in F_n^l\); \(P, Q, R, M, N\) are certain constants; \(E\) is the identity matrix, whose rows we denote by \(P_i, Q_i, R_i, M_i, N_i\).

We require that for some \(u(t) \in U\) the function \(x(t)\): 1) on the interval \([T_1,T_2]\) satisfy the equation

\[ dx/dt = Ax + Bu + A_0, \tag{4} \]

where \(A, B, A_0\) are certain constant matrices and a vector of corresponding

orders; 2) for \(t=T_1\) it lay on the linear manifold:

\[ x=x^1=e_0^1+\sum_1^{k_1}\vartheta_i^1 e_i^1, \tag{5} \]

and for \(t=T_2\), on the linear manifold

\[ x=x^2=e_0^2+\sum_1^{k_2}\vartheta_i^2 e_i^2, \tag{6} \]

where \(e_0^1, e_i^1\) and \(e_0^2, e_i^2\) are systems of linearly independent vectors \((k_1,k_2\leq n)\).

This formulation of the problem means that for \(u(t)\in F^{l}\), where \(l\geq 0\), and \(x(t)\in F^{n,l}\), where \(l\geq 1\), there exist such numbers \(\vartheta_i^1\) and \(\vartheta_j^2\) \((i\leq k_1,\ j\leq k_2)\) for which, for any \(\varphi\in k^{n,l}\) \((l\geq 1)\) satisfying the condition \(\varphi_i(t)\equiv 0\) on the interval \([T_1^*,T_1]\) when \(x_i^1=0\) and on the interval \([T_2,T_2^*]\) when \(x_i^2=0\), the following holds:

\[ (x_1\varphi' + A^*\varphi) + (u,B^*\varphi) + (\bar A_0,\varphi) + x^1\varphi(T_1)-x^2\varphi(T_2)=0, \tag{7} \]

where \(A^*\) and \(B^*\) are the matrices transposed to \(A\) and \(B\); the vector-function \(\bar A_0\) is equal to \(A\) for \(t\in [T_1,T_2]\) and is equal to zero at the remaining points; the coordinates: \(x_i(t)\equiv 0\) for \(t\in [T_1^*,T_1)\), if \(x_i^1\ne 0\), and for \(t\in (T_2,T_2^*]\), if \(x_i^2\ne 0\).

Statement of the problem. It is required to find a control \(u(t)\in U\) such that: 1) the corresponding solution \(x(t)\) of equation (4) satisfies the boundary conditions (5) and (6); 2) the functional \((u(t),\rho(t))\), in which \(\rho(t)\in K_r^l\), takes the maximum value among all \(u(t)\in U\).

Theorem 1. If \(u(t)\in U\), then \(u(t)\in F^{l,1}\) and \(x(t)\in F^{n,l}\), where \(l\geq 1\).

Variation of generalized functions. We shall say that a function \(g(\tau)\) belongs to the class \(G\) if \(g(\tau)\) is absolutely continuous and monotone on the interval \([0,1]\) and satisfies the conditions

\[ g(0)=T_1^*,\qquad g(1)=T_2^*. \]

With the aid of \(g(\tau)\in G\), represent an arbitrary \(f(t)\in F^{l,k}\) \((l\geq k)\) in the form \(f(t)=f^*(\tau(t))\), where \(\tau\) is determined by the value of \(t\) from the inverse relation \(t=g(\tau)\), i.e.

\[ (f(t),\varphi(t))=(-1)^k(F(t),\varphi^{(k)}(t))=(-1)^k(F^*(\tau(t)),\varphi^{(k)}(t))= \]

\[ =(-1)^k\int_0^1 F^*(\tau)\varphi^{(k)}(g(\tau))g'(\tau)\,d\tau=(f^*(\tau(t)),\varphi(t)), \]

where \(F^*(\tau)\) is uniquely determined everywhere where \(g'(\tau)\ne 0\), and is arbitrary when \(g'(\tau)=0\).

Let us vary \(f(t)\); for this purpose consider \(f(t,\varepsilon)=f^*(\tau_\varepsilon(t))\), where \(\tau_\varepsilon(t)\) is constructed with the aid of \(g(t,\varepsilon)=g(\tau)+\varepsilon g^*(\tau)\in G\) \((\varepsilon>0)\) in the form \(t=g(\tau_\varepsilon,\varepsilon)\). We obtain the expression

\[ (f(t,\varepsilon),\varphi(t))=(f^*(\tau_\varepsilon(t)),\varphi(t))=(-1)^k(F(t,\varepsilon),\varphi^{(k)}(t))= \]

\[ =(-1)^k(F^*(\tau_\varepsilon(t)),\varphi^{(k)}(t))=(-1)^k\int_0^1 F^*(\tau)\varphi^{(k)}(g(\tau,\varepsilon))g'(\tau,\varepsilon)\,d\tau, \]

which exists under the assumption of integrability of \(F^*(\tau_\varepsilon(t))\).

Theorem 2. For any \(f(t)\in F^{l,k}\) \((l\geq k+2)\) and any \(g(\tau)\in G\), and for arbitrary \(f^*(\tau)\) at points where \(g'(\tau)=0\), but such that \(f(t,\varepsilon)\in F^{l,k}\), there exists a function \(\delta f(t)\in F^{l,k+1}\) such that

\[ f(t,\varepsilon)-f(t)=\varepsilon\,\delta f(t)+o(\varepsilon). \]

According to this theorem we obtain the variation of the optimal control \(u_0(t)\):

\[ u(t)-u_0(t)=\varepsilon\,\delta u(t)+o(\varepsilon), \]

where \(\delta u(t)\in F^{3,2}\).

The variation of the phase coordinates \(\delta x(t)\) is obtained as the solution of the equation:

\[ (\delta x,\varphi' + A^*\varphi) + (\delta u,B^*\varphi) + \delta x^1\varphi(T_1) - \delta x^2\varphi(T_2)=0, \tag{8} \]

in which \(\delta x^1\) and \(\delta x^2\) are determined by expressions (5) and (6), where it is assumed that \(e_0^1=e_0^2=0\); \(\varphi(t)\in K_n^3\), and moreover \(\varphi_i(t)\equiv 0\) on the interval \([T_1^*,T_1]\), if \(\delta x_i^1=0\), and on the interval \([T_2,T_2^*]\), if \(\delta x_i^2=0\). The values of the coordinates \(\delta x_i(t)\) are equal to zero for \(t\in[T_1^*,T_1)\), if \(\delta x_i^1\ne 0\), and for \(t\in(T_2,T_2^*]\), if \(\delta x_i^2\ne 0\). In this case \(\delta x(t)\in F^{3,1}\).

Considering the sets of variations generated by each constraint of the inequality types (1), (2), (3), we note that the arguments given in [5] make it possible to assume the absence of a fixed boundary for the set of phase coordinates.

Considering also the set of variations arising from the set of variations and constraints of equality type, and determining the general form of the functionals generated by each constraint, we obtain the generalized Euler equation

\[ \sum_{j\in I_1}(P_j\delta u,\lambda_{j,1}) +\sum_{j\in I_2}(M_j\delta u,\lambda_{j,2}) +\sum_{j\in I_3}(\delta u_j,\lambda_{j,3}) + \]

\[ +\sum_{j\in I_1}(Q_j\delta x,\lambda_{j,4}) -\alpha(\delta u,\rho) +l(\delta x-\bar{x})=0, \]

in which the integrals

\[ (P_j\delta u,\lambda_{j,1}) = -\int_{T_1^*}^{T_2^*} P_j\delta w\,d\lambda'_{j,1}; \qquad (M_j\delta u,\lambda_{j,2}) = -\int_{T_1^*}^{T_2^*} M_j\delta w\,d\lambda'_{j,2}; \]

\[ (\delta u_j,\lambda_{j,3}) = \int_{T_1^*}^{T_2^*}\delta w_j\,d\lambda'_{j,3}; \qquad (Q_j\delta x,\lambda_{j,4}) = -\int_{T_1^*}^{T_2^*} Q_j\,dz\,d\lambda_{j,4}; \]

exist, since \(\delta w(t)\) and \(\delta z(t)\) are certain integrable functions whose first and, respectively, second derivatives are equal to \(\delta u(t)\in F^{3,2}\) and \(\delta x(t)\in F^{3,1}\); \(\alpha>0\) is a certain constant; \(\bar{x}(t)\) is a solution of equation (8), and \(l\) is an arbitrary linear functional defined on \(F^{3,1}\). The nonnegative functions

\[ \lambda_{j,1}(t),\lambda_{j,2}(t),\lambda_{j,3}(t),\lambda_{j,4}(t), \tag{9} \]

for which \(\lambda'_{j,1}(t)\), \(\lambda'_{j,2}(t)\), \(\lambda'_{j,3}(t)\), \(\lambda_{j,4}(t)\) are absolutely continuous, differ from zero only, respectively, on the sets \(\Lambda_j^1,\Lambda_j^2,\Lambda_j^3,\Lambda_j^4\) of those \(t\in[T_1,T_2]\), where \(S_j(x_0,u_0)=0\) for \(\Lambda_j^1\) \((j\in I_1)\); \(G_j(u_0)=0\) for \(\Lambda_j^2\) \((j\in I_2)\), \(u_{0,j}(t)=0\) for \(\Lambda_j^3\) \((j\in I_3)\).

Theorem 3. For the optimality of the solution \(u(t),x(t)\), it is necessary that functions (9) exist with the properties indicated above, such that the following hold:

\[ \sum_{j\in I_1}(P_j\delta u,\lambda_{j,1}) +\sum_{j\in I_2}(M_j\delta u,\lambda_{j,2}) +\sum_{j\in I_3}(\delta u_j,\lambda_{j,3})=0, \]

\[ \sum_{j\in I_1}(Q_j\delta x,\lambda_{j,4})=0. \]

For each \(u(t),x(t)\), consider the vector function \(\psi(t)\), which on \([T_1,T_2]\) is a solution of the equation

\[ \frac{d\psi}{dt}=-A^*\psi+\sum_{j\in I_1}Q^*\lambda_{j,4}. \]

and satisfying the boundary conditions

\[ \delta x^1 \psi(T_1)=0,\qquad \delta x^2 \psi(T_2)=0. \]

By virtue of the assumed absolute continuity of \(\lambda_{j,4}(t)\) on \([T_1,T_2]\), the function \(\psi(t)\) will be absolutely continuous together with its first derivative.

Adding to the values of the vector \(\psi(t)\) the coordinate \(\psi_0(t)\), equal to \(-1\), we obtain the vector

\[ \overline{\psi}(t)=(\psi_0,\psi_1,\ldots,\psi_n). \]

Let \(\overline{B}^{*}=\overline{B}^{*}(t)\) denote the matrix transposed to \(\overline{B}(n+1,r)\), which is obtained from \(B(n\times r)\) by adding the upper row \(\rho_1(t),\ldots,\rho_r(t)\).

Theorem 4. If for the solution \(u(t), x(t)\) of the variational problem under consideration there exists a vector-function \(\psi(t)\) satisfying the conditions specified above, then the attainment of the maximum by the expression \((u(t), \overline{B}^{*}\overline{\psi}(t))\) is a necessary and sufficient condition for the optimality of this solution.

Moscow Institute of Physics and Technology

Received
27 V 1965

REFERENCES

\(^{1}\) L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, Mathematical Theory of Optimal Processes, Moscow, 1961.
\(^{2}\) R. Bellman, Dynamic Programming, IL, 1960.
\(^{3}\) I. M. Gelfand, G. E. Shilov, Generalized Functions, vol. 1, Moscow, 1958–1959.
\(^{4}\) A. Ya. Dubovitskii, A. A. Milyutin, DAN, 149, No. 4 (1963).
\(^{5}\) Yu. P. Krivenkov, DAN, 156, No. 6 (1964).
\(^{6}\) Yu. P. Krivenkov, DAN, 156, No. 4 (1964).