B. N. PSHENICHNYI

THE STRUCTURE OF DIFFERENTIAL GAMES

(Presented by Academician L. S. Pontryagin, 14 V 1968)

1. In the present note we consider the general structure of a differential game in which the payoff is the time at which the game ends. The approach to differential games developed below is based on a generalization of the ideas of L. S. Pontryagin, set forth for linear games in the work (¹).

Let the differential game be described by the system of differential equations

\[ \dot z = f(z,u,v), \tag{1} \]

where \(z \in E^n\), \(u \in E^r\), \(v \in E^s\), and \(u\) and \(v\) range over compact sets \(U\) and \(V\). In addition, a terminal set \(M\) is given. The game consists in the fact that the opponent controlling \(u\) (in what follows, opponent \(U\)) seeks to bring the phase point \(z\) to the set \(M\) in minimal time, while opponent \(V\) seeks to prevent this. At each moment of time opponent \(U\) knows only local information about the object, i.e. the value of the phase coordinates \(z\) at the current moment of time. Thus each of his strategies is some function \(u(z)\). This is the usual formulation of the problem.

Unfortunately, it leads to a number of great difficulties, and the first of them is that, as a rule, the strategy \(u(z)\) must be taken discontinuous, after which the question of the existence of a solution of system (1) becomes completely unclear, since usually theorems on the existence of solutions of nonlinear differential equations with discontinuities can be proved only under certain restrictions on the character of the discontinuities. To avoid this difficulty, which in a certain sense is not essential here, we shall assume that opponent \(V\), at the initial moment of time, communicates to opponent \(U\) his control on some nonzero interval of time \(\sigma_1\), where (essentially) the quantity \(\sigma_1 > 0\) is chosen by opponent \(V\) at his own discretion. On the basis of this information opponent \(U\) constructs his control on the same interval of time. After the time \(\sigma_1\) has elapsed, opponent \(V\) again communicates the time interval \(\sigma_2\) and his control on it, and so on. Thus, at each moment of time opponent \(U\) knows not the entire future behavior of opponent \(V\), but only his control on a small interval of time. The strategies described will be called \(\sigma\)-strategies.

We shall say that the game starting from the point \(z_0\) can be completed in time \(T\), if to each \(\sigma\)-strategy of opponent \(V\), opponent \(U\) can assign his own \(\sigma\)-strategy in such a way that the trajectory \(z(t)\) of system (1) corresponding to these controls reaches the set \(M\) no later than in time \(T\).

Throughout the entire work we shall assume that the right-hand side of (1) is continuously differentiable with respect to \(z\) and continuous with respect to \(u\) and \(v\). In addition, for any \(z\) and \(v\) the set \(f(z,U,v)\) is convex and

\[ \left|z f(z,u,v)\right| \leq c(1+|z|^2). \]

From the results of the work (²) it follows that, under these conditions, the set of trajectories of system (1) starting from a fixed point \(z_0\) under

fixed control \(v(t)\) and all possible admissible controls \(u(t)\), compact in the metric of the space of continuous functions.

1. Definition 1. The operator \(T_\sigma\), \(\sigma \geq 0\), assigns to each set \(X \subset E^n\) the set \(T_\sigma(X)\) of points \(z \in E^n\) such that for every measurable control \(v(t)\), \(v(t) \in V\), there exists a measurable control \(u(t)\), \(u(t) \in U\), for which the solution of system (1) with the initial condition \(z(0)=z\) and \(u=u(t)\), \(v=v(t)\), reaches the set \(X\) no later than at time \(\sigma\).

Let us list the properties of the operator just introduced, which are almost obvious consequences of its definition.

Property 1.
a) \(T_\sigma(X) \subset T_{\sigma'}(X)\) for \(\sigma' \geq \sigma\);

b) \(T_\sigma(X) \subset T_\sigma(X')\) for \(X' \supset X\);

c) \(T_0(X)=X,\quad T_\sigma(X) \supset X\).

Property 2. \(T_{\sigma_1}T_{\sigma_2}(X) \subset T_{\sigma_1+\sigma_2}(X)\).

Property 3. If \(X\) is closed, then \(T_\sigma(X)\) is also closed, and from \(z \in T_\sigma(X)\) with \(\sigma>\sigma_0\) it follows that \(z \in T_{\sigma_0}(X)\).

Property 4. If the closed sets \(X_i\), \(i=1,\ldots\), are nested in one another, i.e. \(X_{i+1}\subset X_i\), then

\[ \bigcap_{i=1}^{\infty} T_\sigma(X_i) = T_\sigma\left(\bigcap_{i=1}^{\infty} X_i\right). \]

Property 5. For arbitrary sets \(X_\alpha\),

\[ \bigcap_{\alpha} T_\sigma(X_\alpha) \supset T_\sigma\left(\bigcap_{\alpha} X_\alpha\right). \]

1. Definition 2. By a rational partition \(\omega\) we shall mean an arbitrary finite sequence of rational numbers \(\tau_i\), \(\tau_i \leq \tau_{i+1}\), \(i=0,1,\ldots,m\), \(\tau_0=0\).

Put \(|\omega|=\tau_m\). We shall say that a rational partition \(\omega'\) is finer than the partition \(\omega\) (denoted \(\omega'<\omega\)) if \(|\omega'|\leq|\omega|\) and all numbers \(\tau_i\), \(\tau_i\leq|\omega'|\), coincide with some of the numbers \(\tau'_j\) defining the partition \(\omega'\).

Let, for a given \(\omega\), \(\delta_i=\tau_i-\tau_{i-1}\), \(i=1,\ldots,n\). Define

\[ T_\omega(X)=\bigl(T_{\delta_m}T_{\delta_{m-1}}\cdots T_{\delta_1}\bigr)(X). \]

Lemma 1. If \(\omega'<\omega\), then \(T_{\omega'}(X)\subset T_\omega(X)\).

The proof of the lemma follows from Property 2 of the operator \(T_\sigma(X)\) and the definition of the relation \(\omega'<\omega\).

Definition 3.

\[ \widetilde T_t(X)=\bigcap_{|\omega|>t} T_\omega(X). \]

It is obvious that \(\widetilde T_t(X)\subset \widetilde T_{t'}(X')\) if \(t'\geq t\) and \(X'\supseteq X\).

Lemma 2. \(\widetilde T_{t_1+t_2}(X)=\widetilde T_{t_1}\widetilde T_{t_2}(X)\).

Proof. We shall denote by \(\omega_\sigma\) those rational partitions \(|\omega_\sigma|>t_1+t_2\) for which there exist rational partitions \(\omega^1\) and \(\omega^2\) satisfying the condition

\[ T_{\omega^1}T_{\omega^2}(X)=T_{\omega_\sigma}(X),\quad |\omega^1|>t_1,\quad |\omega^2|>t_2. \tag{2} \]

If \(|\omega|>t_1+t_2\), \(\omega=\{\tau_0,\tau_1,\ldots,\tau_m\}\), then there exists a rational number \(\tau\) such that \(\tau>t_2\), \(\tau_m-\tau>t_1\). Consider the partition \(\bar\omega=\{\tau_0,\tau_1,\ldots,\tau_i,\tau,\tau_{i+1},\ldots,\tau_m\}\). Obviously, \(\bar\omega<\omega\), and

\[ T_{\omega^1}T_{\omega^2}(X)=T_{\bar\omega}(X)\subset T_\omega(X), \tag{3} \]

where \(\omega^1=\{0,\tau_{i+1}-\tau,\ldots,\tau_m-\tau\}\), \(\omega^2=\{\tau_0,\tau_1,\ldots,\tau_i,\tau\}\). Thus, for every partition \(\omega\) there is a finer partition \(\omega_\sigma\). On the other hand, to any two partitions \(\omega^1\) and \(\omega^2\), \(|\omega^1|>t_1\), \(|\omega^2|>t_2\), one can assign a partition \(\omega_\sigma\) so that (2) will hold.

Taking into account only what has been said, we obtain

\[ \widetilde T_{t_1+t_2}(X)= \bigcap_{|\omega|>t_1+t_2} T_\omega(X)= \bigcap_{|\omega\sigma|>t_1+t_2} T_{\omega\sigma}(X)= \bigcap_{|\omega^1|>t_1}\bigcap_{|\omega^2|>t_2}T_{\omega^1}T_{\omega^2}(X)\supset \]

\[ \supset \bigcap_{|\omega^1|>t_1}T_{\omega^1}\left(\bigcap_{|\omega^2|>t_2}T_{\omega^2}(X)\right) =\widetilde T_{t_1}\widetilde T_{t_2}(X), \tag{4} \]

where, in the derivation, property 5 of the operator \(T_\sigma\) was used. Since each partition is determined by a set of rational numbers, the number of partitions \(\omega,\ |\omega|>t_2\), is countable. It turns out that one can construct such a sequence \(\omega_k,\ |\omega_k|>t_2,\ k=1,\ldots,\) that \(\omega_{k+1}<\omega_k\) and

\[ \widetilde T_{t_2}(X)=\bigcap_{k=1}^{\infty}T_{\omega_k}(X), \]

so that \(\widetilde T_{t_2}(X)\) is formed as the intersection of sets nested in one another.

Now

\[ \widetilde T_{t_1}\widetilde T_{t_2}(X)= \bigcap_{|\omega^1|>t_1}T_{\omega^1}\left(\bigcap_{|\omega^2|>t_2}T_{\omega^2}(X)\right) = \bigcap_{|\omega^1|>t_1}T_{\omega^1}\left(\bigcap_{k=1}^{\infty}T_{\omega_k}(X)\right)= \]

\[ = \bigcap_{|\omega^1|>t_1}\bigcap_{k=1}^{\infty}T_{\omega^1}T_{\omega_k}(X) \supset \bigcap_{|\omega\sigma|>t_1+t_2}T_{\omega\sigma}(X) =\widetilde T_{t_1+t_2}(X). \tag{5} \]

Comparison of (4) and (5) completes the proof of the lemma.

4. Theorem. Let the differential game be described by system (1), and let a closed terminal set \(M\) be given. Let

\[ t(z_0)=\min_{z_i\in T_t(M)} t. \]

If \(z_0\in \widetilde T_t(M),\ t\ge 0\), then \(t(z_0)=+\infty\).

Then, in order that the game can be ended from the point \(z_0\) in time \(t_0\), it is necessary and sufficient that \(t_0\) be not less than \(t(z_0)\).

We indicate the main ideas of the proof. If \(t_0\ge t(z_0)\), then \(z_0\in \widetilde T_{t_0}(M)\). If the control \(v(t)\) is known on the interval \([0,\sigma]\), then from the relation

\[ z_0\in \widetilde T_{t_0}(M)=T_\sigma \widetilde T_{t_0-\sigma}(M)\subset T_\sigma \widetilde T_{t_0-\sigma}(M) \]

it follows that one can choose a control \(u(t)\) such that \(z(\delta)\in \widetilde T_{t_0-\sigma}(M)\) for some \(\delta,\ 0<\delta\le \sigma\). By continuing this process, the trajectory of system (1), under any \(\sigma\)-strategy of the opponent \(V\), is brought to the set \(M\) no later than in time \(t_0\), since \(T_0(M)=M\).

If \(t_0<t(z_0)\), then \(z_0\notin \widetilde T_{t_0}(M)\), and there exists a partition \(\omega\) such that \(z_0\notin T_\omega(M),\ |\omega|>t_0\). Therefore

\[ z_0\notin T_{\delta_m}T_{\delta_{m-1}}\ldots T_{\delta_1}(M), \]

\[ \delta_m+\delta_{m-1}+\ldots+\delta_1=|\omega|>t_0. \]

Using now the definition of the operator \(T_\sigma\), we are convinced that \(V\) has such a \(\sigma\)-strategy, determined by the partition and by the control of the adversary \(U\), that, no matter how the adversary \(U\) acts, the trajectory of system (1) will not reach \(M\) before time \(t_0\).

Institute of Cybernetics
Academy of Sciences of the Ukrainian SSR

Received
7 V 1968

REFERENCES

1. L. S. Pontryagin, DAN, 175, No. 4 (1967).
2. G. S. Goodman, Mathematical Theory of Control, 1967, p. 222.