Academician L. S. PONTRYAGIN

ON SOME DIFFERENTIAL GAMES

1. Formulation of the problem

It is assumed that the state of an object is determined by a point \(z=(z^1,\ldots,z^n)\) of the vector \(n\)-dimensional space \(R\), and its behavior is determined by a system of ordinary differential equations:

\[ dz/dt=Z(z,u,v)=X(z,u)+Y(z,v), \tag{1} \]

whose right-hand sides are analytic functions; \(u\) and \(v\) are control parameters. Here \(u\) is a point of an analytic \(p\)-dimensional manifold \(P\), and \(v\) is a point of an analytic \(q\)-dimensional manifold \(Q\). In the space \(R\) an analytic manifold \(M\) of some dimension is given. The game is considered finished when the point \(z\) reaches the manifold \(M\). The problem consists in determining, at each instant of time, the behavior of the parameter \(u\) that leads to completion of the game in the shortest time, knowing the state \(z\) of the object at that instant of time and the value of the parameter \(v\) at the same instant of time. It should be noted that at some points \(z\) one must use not only the values of the parameter \(v\) themselves, but also a certain number of its derivatives with respect to time. The parameter \(v\) is assumed to be a piecewise-analytic function of time \(t\).

2. Main result

As in the theory of optimal processes \((^1)\), alongside the contravariant vector \(z\) we introduce a covariant vector \(\psi=(\psi_1,\ldots,\psi_n)\) and define the function \(H\) by setting:

\[ H(z,\psi,u,v)=\psi Z=\sum_{i=1}^n \psi_i Z^i(z,u,v). \tag{2} \]

For fixed values of the vectors \(z\) and \(\psi\), let us find the maximum \(M(z,\psi)\) of the function \(\psi X(z,u)\) and the minimum \(m(z,\psi)\) of the function \(\psi Y(z,v)\). We form the system of ordinary differential equations:

\[ dz^i/ds^1=\partial H/\partial \psi_i;\qquad d\psi_i/ds^1=-\partial H/\partial z^i, \tag{3} \]

taking \(s^1\) as the independent variable, and supplement this system with the finite relations

\[ \begin{aligned} \psi X(z,u)&=M(z,\psi),\\ \psi Y(z,v)&=m(z,\psi). \end{aligned} \tag{4} \]

We shall assume that the system (3), (4) is solvable in the following sense. Let \(z_0\) be an arbitrary point of \(M\), and let \(\psi^0\) be a unit covariant vector defining a hyperplane tangent to \(M\) at the point \(z_0\). We shall assume that the system (3), (4) has a unique solution

\[ z=z(s^1);\qquad \psi=\psi(s^1);\qquad u=u(s^1);\qquad v=v(s^1), \tag{5} \]

defined for all values \(s^1\le 0\) and satisfying the initial conditions \(z(0)=z_0\); \(\psi(0)=\psi^0\). The solution (5) depends on the initial pair \((z_0,\psi^0)=x\), and the totality of all such pairs forms an analytic manifold \(N\) of dimension \(n-1\), in which we introduce local coordi-

coordinates \(s^2,\ldots,s^n\). Forming all possible pairs of the form \((s^1,x)\), where \(s^1\) is a negative number and \(x\) is a point of the manifold \(N\), we obtain a manifold \(S\) of dimension \(n\), whose points we shall denote by
\[ s=(s^1,x)=(s^1,s^2,\ldots,s^n). \]
Taking into account the dependence of the solution (5) on the initial conditions, we can write:
\[ z=z(s)=\omega(s);\qquad \psi=\psi(s);\qquad u=\mathbf{u}(s);\qquad v=\mathbf{v}(s). \tag{6} \]

The function \(\omega\) gives an analytic mapping of the manifold \(S\) into the space \(R\). In the case when the mapping \(\omega\) is one-to-one and has a functional determinant nowhere vanishing, the problem was solved by Bellman. However, in very simple cases of interest the mapping \(\omega\) is not one-to-one. The present paper is devoted to overcoming this difficulty under certain simple assumptions.

Among all points \(s=(s^1,x)\) that pass into one and the same point \(z\) under the mapping \(\omega\), choose that for which the number \(s^1\) has the greatest value. We shall say of this point \(s\) that it belongs to the upper layer, and shall denote it by \(\omega^{-1}(z)\).

Theorem. Let \(\hat z\) be some point of \(R\), and let
\[ s_0=(s_0^1,x_0)=\omega^{-1}(\hat z). \]
Then, starting from the state \(\hat z\) of the object, the game can always be ended in a time not exceeding the number \(|s_0^1|\).

This theorem is, of course, not true for an arbitrary game (1). Here it will be proved only under certain very restrictive assumptions.

Let \(z(t)\) be a solution of equation (1), starting at \(\hat z\) and ending on \(M\). Put
\[ \omega^{-1}(z(t))=(s^1(t),x(t)). \]
Suppose that
\[ \frac{ds^1(t)}{dt}\geq 1. \]
Then the game ends in a time not exceeding the number \(|s_0^1|\). Consequently, it is sufficient for us to construct the control \(u(t)\) as the control \(v(t)\) becomes known, and to construct it in such a way that the inequality
\[ \frac{ds^1}{dt}\geq 1 \]
is satisfied at all times.

The controls
\[ \tilde u(t)=\mathbf{u}(s_0^1+t,x_0),\qquad \tilde v(t)=\mathbf{v}(s_0^1+t,x_0) \]
are called extremal. They correspond to the extremal motion of the object
\[ z=\omega(s_0^1+t,x_0), \]
for which \(ds^1/dt\equiv 1\).

Below we give some indications of the method of proof for a non-extremal control \(v(t)\), as well as a formulation of the conditions under which the theorem is proved.

3. Recording equation (1) in the variables \(s^1,s^2,\ldots,s^n\)

Put:
\[ H(s,u,v)=H(z(s),\psi(s),u,v);\qquad H(s)=H(s,\mathbf{u}(s),\mathbf{v}(s)); \tag{7} \]
\[ \delta H=\delta H(s,u,v)=H(s,u,v)-H(s). \tag{8} \]
From condition (4) it follows that:
\[ H(s,u,\mathbf{v}(s))\leq H(s);\qquad H(s,\mathbf{u}(s),v)\geq H(s). \tag{9} \]
In the case where the points \(u\) and \(v\) are respectively close to the points \(\mathbf{u}(s)\) and \(\mathbf{v}(s)\), one can give meaning to the quantities
\[ \delta u=u-\mathbf{u}(s);\qquad \delta v=v-\mathbf{v}(s); \tag{10} \]
we shall regard them as vectors whose coordinates are computed in local coordinates of the manifolds \(P\) and \(Q\). Expanding the quantity \(\delta H\) in a series in the coordinates of the vectors (10), we obtain
\[ \delta H=-f_s(\delta u)+g_s(\delta v)+\cdots, \tag{11} \]
where \(f_s\) and \(g_s\) are nonnegative quadratic forms depending on \(s\), and terms of order higher than the second have been omitted.

It is easily proved that

\[ H(s)=\psi(s)\,\partial \omega(s)/\partial s^1;\qquad \partial H(s)/\partial s^1=0;\qquad \psi(s)\,\partial \omega(s)/\partial s^i=0, \tag{12} \]

\[ i=2,\ldots,n. \]

In what follows we shall assume that the following is satisfied.

Condition 1. At each point \(s\) the vectors \(\partial \omega(s)/\partial s^2,\ldots,\partial \omega(s)/\partial s^n\) are linearly independent.

It follows from this that the functional determinant \(D(s)\) of the mapping \(\omega\) satisfies the condition \(D(s)=d(s)H(s)\), where \(d(s)\) does not vanish.

Let \(\hat z=\omega(s_0)\), and let \(s^1,\ldots,s^n\) be local coordinates in a neighborhood of the point \(s_0\). In order to write system (1) near the point \(\hat z\) in the variables \(s^1,\ldots,s^n\), it is enough to solve the vector equation

\[ Z(\omega(s),u,v)=\sum_{i=1}^{n}\frac{\partial \omega(s)}{\partial s^i}\frac{ds^i}{dt} \tag{13} \]

with respect to the quantities \(ds^i/dt\). Multiplying relation (13) by \(\psi(s)\) and dividing the result by \(H(s)\), we obtain, by virtue of (12),

\[ ds^1/dt=1+\delta H/H(s). \tag{14} \]

If \(H(s_0)>0\), then \(D(s_0)=d(s_0)H(s_0)\ne0\), and therefore equation (13) can be solved; in particular, relation (14) is valid. In this case, whatever the control \(v(t)\), we define the control \(u(t)\) by the relation \(u(t)=u(s(t))\), and then, by virtue of (9), relation (14) gives \(ds^1/dt\ge 1\). This corresponds to Bellman’s solution.

If \(H(s_0)=0\), then we shall assume that the following is satisfied.

Condition 2. For \(H(s_0)=0\) we have \(\operatorname{grad}H(s_0)\ne0\).

Then there exists a vector \(r(s)=(r^1(s),r^2(s),\ldots,r^n(s))\), analytic in a neighborhood of the point \(s_0\), such that \(r^1(s)\equiv1\) and

\[ \sum_{i=1}^{n}\frac{\partial \omega(s)}{\partial s^i}r^i(s)=0 \quad \text{when } H(s)=0. \tag{15} \]

From (13), in addition to (14), one can derive that

\[ \frac{ds^i}{dt}=\frac{\delta H}{H(s)}\,r^i(s)+R^i(s,u,v), \qquad \text{where } R^i(s,u(s),v(s))=0, \tag{16} \]

where \(R^i(s,u,v)\) is an analytic function.

The derivative of a certain function \(\varphi(s)\) by virtue of system (14), (16) is equal to

\[ \frac{d\varphi(s)}{dt} = \frac{\delta H}{H(s)}\varphi_r(s) + \frac{\partial \varphi(s)}{\partial s^1} + \varphi_R(s,u,v), \tag{17} \]

where \(\varphi_r(s)=\operatorname{grad}\varphi(s)\cdot r(s)\); \(\varphi_R(s,u(s),v(s))=0\).

4. Condition for solvability of the system (14), (16). Let

\[ s_0=(s_0^1,x_0)=\omega^{-1}(\hat z),\qquad H(s_0)\ge0. \]

Regarding the control \(v(t)\), \(t\ge0\), as arbitrarily prescribed and non-extremal, we shall seek such a \(u(t)\), \(t\ge0\), that the solution \(s(t)\) of system (14), (16) with initial condition \(s(0)=s_0\) satisfies the inequalities

\[ ds^1(t)/dt>1;\qquad H(s(t))>0\quad \text{for } t>0. \tag{18} \]

Since \(v(t)\ne \tilde v(t)\), two cases are possible:

\[ v(0)\ne v(s_0), \tag{19} \]

\[ v(t)=\tilde v(t)+\tilde b t^m+O(t^{m+1}), \tag{20} \]

where \(\tilde b\ne0\) is some \(q\)-dimensional vector, and \(m\) is a natural number.

In case (19) we impose two additional conditions.

Condition 3. \(\delta H(s_0,u(s_0),v(0))>0\).

Put \(H^0(s)=H(s),\ H^1(s)=H_r^0(s),\ldots,\ H^{i+1}(s)=H_r^i(s)\) (see (17)).

Condition 4. For each point \(s_0\) of the upper layer there exists a nonnegative integer \(k\) such that
\[ H^0(s_0)=0,\ldots,\ H^{k-1}(s_0)=0,\quad H^k(s_0)>0. \]

Putting \(u(t)=u(s(t))\), under these conditions we can find a solution \(s(t)\) of the system (14), (16) satisfying conditions (18).

In the case (20) we impose two further additional conditions.

Condition 5. The quadratic forms \(f_{s_0}\) and \(g_{s_0}\) (see (11)) are nondegenerate.

The quadratic forms with matrices inverse to the matrices of the forms \(f_{s_0}\) and \(g_{s_0}\) will be denoted by \(\hat f_{s_0}, \hat g_{s_0}\). They are applicable to covariant vectors. Expand the function \(H_R(s,u,v)\) (cf. (17)) in a Taylor series in \(\delta u\) and \(\delta v\):
\[ H_R(s,u,v)=\lambda(s)\delta u+\mu(s)\delta v+\ldots \]
Here \(\lambda(s)\) and \(\mu(s)\) are covariant vectors.

Condition 6. \(\hat f_{s_0}(\lambda(s_0))>\hat g_{s_0}(\mu(s_0))\).

In the case (20) there exists a control \(u(t)\) (generally speaking, not coinciding with \(u(s(t))\)) such that the solution \(s(t)\) of the system (14), (16) satisfies conditions (18).

If, for \(s^1\) close to \(s_0^1\), the identities
\[ \lambda(s^1,x_0)\equiv0;\quad \mu(s^1,x_0)\equiv0;\quad \frac{\partial}{\partial s^1}H_r(s^1,x_0)\equiv0, \tag{21} \]
hold, then condition 6 is not fulfilled; but then, for \(k=2\) (see condition 4), for the control (20) one may take the control \(u(t)=u(s(t))\), and the solution \(s(t)\) will satisfy conditions (18).

5. Example. Let \(a\) and \(b\) be two objects whose geometric positions are determined by vectors \(\xi\) and \(\eta\) in a Euclidean space \(E\) of arbitrary dimension. Their motions are given by the equations:
\[ \ddot{\xi}+\alpha\dot{\xi}=\rho u,\qquad \ddot{\eta}+\beta\dot{\eta}=\sigma v. \]
Here \(\alpha,\beta,\rho,\sigma\) are positive numbers; \(u\) and \(v\) are control vectors from \(E\), of modulus equal to 1. The game consists in the pursuit of object \(b\) by object \(a\). The theory set out above is applicable to it if the inequalities
\[ \rho>\sigma,\qquad \rho/\alpha>\sigma/\beta \]
are fulfilled.

In computing this example the following general proposition is used.

Let \(z\) be an arbitrary point of \(R\) not belonging to \(M\). Suppose that the totality of all such negative numbers \(s^1\) for which \(\omega(s^1,x)=z\) is determined from the equation \(F(s^1,z)=0\), and that, for negative \(s^1\), this equation is incompatible with the equations
\[ \frac{\partial}{\partial s^1}F(s^1,z)=0;\quad \frac{\partial}{\partial z^i}F(s^1,z)=0,\quad i=1,\ldots,n. \]
Then, if condition 1 is fulfilled, for negative \(s^1\) the relation
\[ H(s)=a(s)\frac{\partial}{\partial s^1}F(s^1,z), \]
holds, where \(z=\omega(s)\), \(s^1\) is a root of the equation \(F(s^1,z)=0\), and \(a(s)\) does not vanish.

This proposition makes it possible to verify condition 4; namely, if \(a(s)>0\), condition 4 is equivalent to the condition
\[ \frac{\partial}{\partial s^1}F(s_0^1,\hat z)=0,\ldots,\quad \frac{\partial^k}{(\partial s^1)^k}F(s_0^1,\hat z)=0,\quad \frac{\partial^{k+1}}{(\partial s^1)^{k+1}}F(s_0^1,\hat z)>0. \]

Received
17 III 1964

References

1. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, E. F. Mishchenko, Mathematical Theory of Optimal Processes, Moscow, 1962.