MATHEMATICS

I. V. GIRSANOV

MINIMAX PROBLEMS IN THE THEORY OF DIFFUSION PROCESSES

(Presented by Academician A. N. Kolmogorov, 3 IX 1960)

1. The following game is considered: let \(U\) be a domain in the \((n+1)\)-dimensional space of the variables \((x_1, x_2, \ldots, x_n, t)\). In \(U\) a family
\[ X(m^1,m^2)=\{x_t(\omega,m^1,m^2)\} \]
of diffusion processes is given, which are characterized by the matrix of local diffusion
\[ A(t,x)=\|a_{ij}(t,x)\|, \]
by the velocity field
\[ b(t,x)=\{b_1(t,x),\ldots,b_n(t,x)\}, \]
and by the lifetime, defined by the additive functional
\[ \int_1^t h(s,x_s)\,ds,\quad h\leq 0\ (^{1}). \]

Suppose that
\[ b(t,x)=b^0(t,x)+m^1(t,x)-m^2(t,x), \tag{1} \]
where \(m^i\) belongs to some set \(M^i\) of vector fields in \(U\). If \(\Phi[x(\cdot)]\) is a functional on continuous functions of a real variable with values in \(U\), and
\[ u(0,x,m^1,m^2)=\mathbf E_{x,0}[\Phi[x_\bullet(\omega,m^1,m^2)]] \tag{2} \]
is the mathematical expectation of the value of \(\Phi\) on the trajectory \(X(m^1,m^2)\) issuing at \(t=0\) from the point \(x\), then, taking \(M^i\) as the set of strategies of the \(i\)-th player, and \(u(0,x,m^1,m^2)\) as the payoff function, we obtain a certain game \(G(M^1,M^2,u)\) in normal form.

Let \(\varphi(t,x)\) be a bounded function on the boundary \(\gamma\) of the domain \(U\); \(g(t,x)\) a bounded function on \(U\); \(\tau_\gamma\) the moment of the first exit of the trajectory \(x_t(\omega,m^1,m^2)\) onto the set \(\gamma\). Put
\[ \Phi[x(\cdot)]=\varphi(\tau_\gamma,x_{\tau_\gamma})+ \int_0^{\tau_\gamma} g(s,x_s)\,ds \tag{3} \]
for those trajectories which reach the boundary of \(U\).

It is known \((^{2})\) that, under certain smoothness restrictions on \(\gamma\), \(A\), and \(b\), the function
\[ u(t,x,m^1,m^2)=\mathbf E_{x,t}[\Phi[x_\bullet(\omega,m^1,m^2)]] \]
satisfies the equation
\[ \frac{\partial u}{\partial t}+Lu+(m^1-m^2,\nabla u)=g \tag{4} \]
with the condition \(u|_\gamma=\varphi\).

Here and below
\[ \nabla u=\left\{\frac{\partial u}{\partial x_1},\ldots,\frac{\partial u}{\partial x_n}\right\},\quad (a,b)=\sum_{i=1}^n a_i b_i,\quad Lu=a_{ij}\frac{\partial^2 u}{\partial x_i\partial x_j}+b_i^0\frac{\partial u}{\partial x_i}+hu. \]

Equation (4) makes it possible to prove the existence of a value and to give a description of the optimal strategies of the players, if the sets \(M^i\) are specified by the conditions

\[ M^i=\left\{m^i(t,x): \sum_{k,l=1}^{n} b^i_{k,l}(t,x)m^i_k(t,x)m^i_l(t,x)\leqslant 1\right\}, \tag{5} \]

where \(b^i_{k,l}(t,x)\) are nondegenerate quadratic forms,

\[ \sum_{k,l=1}^{n} b^i_{k,l}\xi_k\xi_l>\alpha \sum_{k=1}^{n}\xi_k^2, \]

\(\alpha>0\).

2. Theorem 1. Let \(A(t,x)\), \(b^0(t,x)\), \(h(t,x)\), and \(U\) be such that equation (4) has a fundamental solution satisfying the homogeneous boundary condition for any smooth vector fields \(m^1, m^2\) satisfying (5); and let \(u(t,x,m^1,m^2)\) depend continuously on \(m^i\) in the topology of bounded convergence almost everywhere. Suppose that the equation

\[ \frac{\partial u}{\partial t}+Lu+ \left(B^1_{k,l}(t,x)\frac{\partial u}{\partial x_k}\frac{\partial u}{\partial x_l}\right)^{1/2} - \left(B^2_{k,l}(t,x)\frac{\partial u}{\partial x_k}\frac{\partial u}{\partial x_l}\right)^{1/2} =g, \tag{6} \]

where \(\|B^i_{k,l}(t,x)\|=\|b^i_{k,l}(t,x)\|^{-1}\), has a continuous generalized solution \(\bar u(t,x)\) in \(\bar V\), taking the value \(\varphi\) on \(\gamma\). Then \(\bar u(0,x)\) is the value of the game, and the vector field

\[ \bar m^i(t,x)= \begin{cases} (-1)^{i+1}\dfrac{B^i(t,x)\nabla u}{\bigl(B^i(t,x)\nabla u,\nabla u\bigr)^{1/2}}, & \text{if } \nabla u \text{ is defined},\\[1.2em] 0, & \text{if } \nabla u \text{ is not defined}, \end{cases} \tag{7} \]

is a pure optimal strategy of the \(i\)-th player \((i=1,2)\).

Proof. We shall prove that the point \(\bar m^1,\bar m^2\) is a saddle point of the game \(G(M^1,M^2,u)\), i.e., that

\[ u(0,x,m^1,\bar m^2)\leqslant u(0,x,\bar m^1,\bar m^2)\leqslant u(0,x,\bar m^1,m^2). \tag{8} \]

To this end, first note that it is sufficient to verify (8) for smooth vector fields \(m^i\), since the value of the game is continuous with respect to \(m^i\) in the sense of bounded convergence.

Let \(m^1,m^2\) be two arbitrary strategies; set

\[ u(t,x,m^1,m^2)=\bar u(t,x)+\delta u(t,x). \]

After substituting into (4), we obtain from (4), (6), and (7)

\[ \frac{\partial}{\partial t}\delta u+L\delta u+(m^1-m^2,\delta\nabla u) =-\bigl[(m^1-\bar m^1,\nabla u)-(m^2-\bar m^2,\nabla u)\bigr], \tag{9} \]

with \(\delta u|_{\gamma}=0\).

A simple geometric argument shows that the linear functions
\((m-\bar m^i,\nabla\bar u)=\Lambda^i(m)\) on the ellipsoids (5) are nonnegative and are equal to zero only when \(m=\bar m^i\). Using now the representation of the solution of (9) through the right-hand side with the help of the fundamental solution, we obtain (8), and with it the assertion of Theorem 1.

It should be noted that equation (6), which connects the value of the game and the optimal strategies, can be derived from heuristic considerations under the assumption of smoothness of \(u\) by the usual methods of dynamic programming \((^3)\). The existence of a classical solution of equation (6) for a nondegenerate matrix \(A\) was proved by Friedman \((^4)\) for a cylindrical domain under rather stringent restrictions on the coefficients \(L\) and small \(B_{k,l}\) satisfying a Hölder condition. The proof is based on estimates for the derivatives of the fundamental solution and the Schauder fixed-point principle. Using more precise estimates for the fundamental solution \((^5)\) and the Leray–Schauder theorem on the index of a solution, it is possible

to weaken the smoothness requirements on the coefficients of \(L\) to Hölder conditions and to replace the smallness requirement on \(B_{kl}^{i}\) by a requirement that they be bounded.

1. Let us consider some special cases.

If the coefficients of equation (6), the right-hand side, and the boundary values do not depend on \(t\), and the domain \(U\) is the cylinder \(U_0 \times [0,\infty)\), then we obtain the elliptic equation

\[ Lu+\left(B_{k,l}^{1}\frac{\partial u}{\partial x_k}\frac{\partial u}{\partial x_l}\right)^{1/2} -\left(B_{k,l}^{2}\frac{\partial u}{\partial x_k}\frac{\partial u}{\partial x_l}\right)^{1/2}=g, \tag{10} \]

\[ u|_{\gamma_0}=\varphi \quad (\gamma_0 \text{ is the boundary of } U_0). \]

The case \(g=0\) means that the payoff function is the solution of the Dirichlet problem; for \(\varphi=0,\ g=-1\), the payoff function will be the mean value of the time spent in the domain \(U\).

For \(n=1,\ g=0\), and \(U=R^1\times[0,T]\), equation (6) takes the form

\[ \frac{\partial u}{\partial t} +a(t,x)\frac{\partial^2u}{\partial x^2} +b(t,x)\frac{\partial u}{\partial x} +B(t,x)\left|\frac{\partial u}{\partial x}\right|=0, \qquad u(T,x)=\varphi(x), \]

where \(B(t,x)=B_{11}^{1}(t,x)-B_{11}^{2}(t,x)\).

The strategy of the first player at time \(t\) at the point \(x\) will consist in moving with the greatest possible speed toward that local maximum of the function \(\bar u(t,x)\) on the way to which there is no local minimum; the strategy of the second player is the opposite.

If one of the \(M^i\) consists of a single point, then we obtain an extremum problem.

Simple examples show that the presence of indeterminacy points \((\nabla u=0)\) is connected with the essence of the problem; in particular, such points necessarily exist if in equation (6) \(g=0\), \(\gamma_0\) is a bounded closed surface, and \(\varphi\) has more than one maximum on \(\gamma_0\). It is possible to prove the following proposition:

Let \(u\) be a solution of equation (10), and suppose that one of the following conditions is satisfied:

\(1^\circ.\ h=0,\quad g\ne0.\)

\(2^\circ.\ u\ge0,\ h\le0,\ g>0\) or \(u\le0,\ h\le0,\ g<0.\)

\(3^\circ.\ g=0,\quad \gamma_0\) is a bounded closed surface, \(\varphi(x)\) has one maximum and one minimum.

Then the set \(C=\{x:\nabla u(x)=0\}\) has Lebesgue measure zero. If the solution \(u\) is classical, then the dimension of \(C\) does not exceed \((n-1)\).

1. Theorem 1 can be extended to the case of \(M^i\) defined by more general conditions of the form

\[ M^i=\{m^i(t,x):\ F^i[t,x,m_1^i(t,x),\ldots,m_i^i(t,x)]\le1\}, \]

where

\[ S^i(t,x)=\{y:\ F^i[t,x,y_1,\ldots,y_n]\le1\} \]

is a convex body; in particular, to the case when each of the players controls not all the coordinates, but only part of them. Analogous results can also be obtained in the case of degeneracy of the matrix \(A\), when equation (4) has no fundamental solution and Theorem 1 cannot be applied to it. In this case one must assume that the admissible \(m^i\) have the form \(Ar\), where \(r\) is a vector field in \(U\); the methods of proof differ from those set forth above.

The case in which not only the velocity fields but also the diffusion coefficients are subjected to variations leads to equations containing nonlinear terms of the form

\[ \left(\sum_{i,j,k,l=1}^{n} a_{ijkl}\, \frac{\partial^2 u}{\partial x_i\partial x_j}\, \frac{\partial^2 u}{\partial x_k\partial x_l}\right)^{1/2}. \]

An interesting feature of the class of games considered above is that their solutions are given by pure strategies, which are determined from the differential equation relating them to the value of the game. This circumstance is due to the fact that the processes under consideration have continuous trajectories, the payoff function is given by the mathematical expectation of an additive functional, and the constraints on \(m^i\) are local in character.

In conclusion, the author expresses his gratitude to I. V. Romanovskii and R. L. Dobrushin, who drew his attention to the problems presented above.

REFERENCES

1. E. B. Dynkin, Foundations of the Theory of Markov Processes, Moscow, 1959.
2. R. Z. Khas’minskii, Theory of Probability and Its Applications, 4, 3, 332 (1959).
3. R. Bellman, Dynamic Programming, IL, 1960.
4. A. Friedman, J. Math. and Mech., 7, 5, 793 (1958).
5. V. Pogorzelski, Matem. sbornik, 47, 4, 397 (1959).