Reports of the Academy of Sciences of the USSR

1. Vol. 145, No. 5

MATHEMATICS

Academician A. N. KOLMOGOROV, E. F. MISHCHENKO, and Academician L. S. PONTRYAGIN

ON A PROBABILISTIC PROBLEM OF OPTIMAL CONTROL

Let \(p(\sigma, x, \tau, y)\) be the probability density of a Markov process in \(n\)-dimensional Euclidean space \(R_n\) \((n \geqslant 3)\), subject to the Kolmogorov equation \({}^{(1)}\)

\[ \frac{\partial p}{\partial \sigma} + \sum_{i,j=1}^{n} a^{ij}(\sigma,x)\frac{\partial^2 p}{\partial x^i \partial x^j} + \sum_{i=1}^{n} b^i(\sigma,x)\frac{\partial p}{\partial x^i} =0. \tag{1} \]

Let a second point \(z\) move in the same space \(R_n\) deterministically according to the law \(z=z(t)\). Together with \(z\), its neighborhood moves, bounded by the closed surface \(\Sigma_t = z(t)+\varepsilon \Sigma\), similar with a small similarity coefficient \(\varepsilon\) to the fixed surface \(\Sigma\) (in what follows, for simplicity, we shall take \(\Sigma\) to be the sphere of unit radius). It is required to determine the probability \(\psi(\sigma,x,\tau)\) that a random point, whose transition density is subject to equation (1), will intersect the surface \(\Sigma_t\) in the time interval \(\sigma \leqslant t \leqslant \tau\).

This problem was solved by E. F. Mishchenko and L. S. Pontryagin in \({}^{(2)}\) in connection with the needs of optimal control. However, the approximate formula for the probability \(\psi\) obtained in that work turned out to be cumbersome and of little use for further application.

A. N. Kolmogorov, having become acquainted with \({}^{(2)}\), proposed, on probabilistic grounds, another, considerably simpler expression for the approximation of E. F. Mishchenko and L. S. Pontryagin. However, he did not give a proof.

In the present note we give Kolmogorov’s formula and its proof, proposed by E. F. Mishchenko and L. S. Pontryagin. This proof is based on the constructions given in \({}^{(2)}\).

It is known (cf. \({}^{(2)}\)) that the desired probability \(\psi(\sigma,x,\tau)\) is a solution of equation (1) and satisfies the conditions

\[ \psi(\tau,x,\tau)=0,\qquad \psi(\sigma,x,\tau)\big|_{\Sigma_\sigma}=1. \tag{2} \]

A. N. Kolmogorov proposed the following formula for the principal part \(K(\sigma,x,\tau,\varepsilon)\) of the probability \(\psi\):

\[ K(\sigma,x,\tau,\varepsilon) = \varepsilon^{\,n-2} \int_{\sigma}^{\tau} p(\sigma,x,s,z(s))\,\beta(s)\,ds, \tag{3} \]

where

\[ \beta(s) = \int_{A_s\Sigma} \frac{\partial w(s,\xi)}{\partial n}\,d\Sigma; \tag{4} \]

\(A_s\) is the linear transformation \(\xi=A_s\bar{\xi}\) which brings the differential form

\[ \sum a^{ij}(s,z(s))\frac{\partial^2}{\partial \xi^i \partial \xi^j} \]

to the form

\[ \sum_{k=1}^{n}\frac{\partial^2}{\partial \xi^{k^2}}, \]

and \(w(s,\xi)\) is a harmonic function satisfying the conditions:

\[ w(s,\xi)=1 \quad \text{for } \xi\in A_s\Sigma; \qquad w(s,\xi)\to 0 \quad \text{as } |\xi|\to\infty. \]

It is verified directly that the function \(K(\sigma,x,\tau,\varepsilon)\), defined by formula (3), satisfies equation (1) outside the point \(z(\sigma)\).

We shall prove that on a certain specially chosen small ellipsoid with center at the point \(z(\sigma)\), the function \(K(\sigma,x,\tau,\varepsilon)\) and the function \(\Psi(\sigma,x,\tau,\varepsilon)\), constructed in [2] and constituting the principal part of the probability \(\psi(\sigma,x,\tau)\), differ “inessentially,” i.e., coincide up to \(o(\varepsilon)\) for \(\tau-\sigma \geqslant \varepsilon\) and differ only by \(O(1)\) for \(\tau-\sigma<\varepsilon\). Hence, by virtue of Lemma 3 of [2], it follows that
\[ K(\sigma,x,\tau,\varepsilon)=\Psi(\sigma,x,\tau,\varepsilon). \]

For the proof, introduce in the space \((z,t)\) new coordinates by the formulas \(z=\xi+z(t)\), \(\sigma\leqslant t\leqslant s\), so that \(x=\xi+z(\sigma)\), \(y=\eta+z(s)\). Next put \(\xi=A_\sigma \xi\). Under this change of coordinates, the function \(K(\sigma,x,\tau,\varepsilon)\) becomes a function \(Q(\sigma,\xi,\tau,\varepsilon)\), while the function \(\Psi(\sigma,x,\tau,\varepsilon)\) becomes a function \(\Phi(\sigma,\xi,\tau,\varepsilon)\). Obviously,
\[ Q(\sigma,\xi,\tau,\varepsilon) = \varepsilon^{\,n-2}\int_\sigma^\tau q(\sigma,\xi,s,0)\,\beta(s)\,ds, \tag{5} \]
where
\[ q(\sigma,\xi,s,\eta) = p\bigl(\sigma,A_\sigma^{-1}\xi+z(\sigma),s,A_s^{-1}\eta+z(s)\bigr). \tag{6} \]
The function \(q(\sigma,x,\tau,\eta)\) is the fundamental solution of the parabolic equation obtained from equation (1) upon passing to the coordinates \(\xi\).

In [2] it is shown that the function \(\Phi(\sigma,\xi,\tau,\varepsilon)\), for \(|\xi|=\varepsilon\), differs only “inessentially” from the quantity \(\alpha(\sigma)\), which arises as follows. We shall solve the Dirichlet problem for the equation \(\Delta w=0\) under the conditions
\[ w(\sigma,\xi)\big|_{H_\sigma}=1,\qquad w(\sigma,\xi)\to 0 \quad \text{as } |\xi|\to\infty . \]
Here \(H_\sigma\) is the ellipsoid obtained from the sphere \(\varepsilon\Sigma\) by the transformation \(A_\sigma\). As is known, the function \(w(\sigma,\xi)\) can be represented in the form
\[ w(\sigma,\xi)=\frac{\varepsilon^{\,n-2}\alpha(\sigma)}{r^{\,n-2}(\xi)}+\Pi(\sigma,\xi,\varepsilon), \tag{7} \]
where \(\Pi(\sigma,\xi,\varepsilon)\) is the double-layer potential produced by the ellipsoid \(H_\sigma\) at the point \(\xi\).

It is not difficult to establish the relation between \(\alpha(\sigma)\) and \(\beta(\sigma)\), which occurs in formula (4). Indeed, if one takes into account that the integral of the normal derivative over the surface \(H_\sigma\) of the double-layer potential \(\Pi(\sigma,\xi,\varepsilon)\) is equal to zero, then, differentiating the right- and left-hand sides of relation (7) along the normal to \(H_\sigma\) and then taking the integral over \(H_\sigma\), we find that
\[ \beta(\sigma)=\frac{4\pi^{n/2}}{\Gamma(n/2-1)}\,\alpha(\sigma), \tag{8} \]
where \(\Gamma\) is the gamma function.

Let us show that the Kolmogorov function \(Q(\sigma,\xi,\tau,\varepsilon)\), defined by formula (5), also differs only “inessentially” from \(\alpha(\sigma)\) when \(|\xi|=\varepsilon\).

Using the arguments and estimates of [2], one can first of all show that
\[ Q(\sigma,\xi,\tau,\varepsilon)\big|_{|\xi|=\varepsilon} = \varepsilon^{\,n-2}\int_\sigma^\tau \gamma(\sigma,\xi,s,0)\big|_{|\xi|=\varepsilon}\,\beta(s)\,ds + o(1), \tag{9} \]
where \(\gamma(\sigma,\xi,s,\eta)\) is the Green function of the heat equation:
\[ \gamma(\sigma,\xi,s,\eta) = \frac{1}{[4\pi(s-\sigma)]^{n/2}} e^{-|\xi-\eta|^2/4(s-\sigma)}. \tag{10} \]

We compute the value of the integral \(\varepsilon^{n-2}\displaystyle\int_\sigma^\tau \gamma(\sigma,\xi,s,0)\beta(s)\,ds\) for \(|\xi|=\varepsilon\).

We have

\[ \begin{aligned} \varepsilon^{n-2}\int_\sigma^\tau \gamma(\sigma,\xi,s,0)\big|_{|\xi|=\varepsilon}\beta(s)\,ds &= \varepsilon^{n-2}\int_\sigma^\tau \gamma(\sigma,\xi,s,0)\big|_{|\xi|=\varepsilon} [\beta(\sigma)\beta(s)-\beta(\sigma)]\,ds \\ &= \frac{\beta(\sigma)\varepsilon^{n-2}}{(4\pi)^{n/2}} \int_\sigma^\tau \frac{1}{(s-\sigma)^{n/2}}e^{-\varepsilon^2/4(s-\sigma)}\,ds \\ &\quad+ \varepsilon^{n-2}\int_\sigma^\tau \gamma(\sigma,\xi,s,0)\big|_{|\xi|=\varepsilon} [\beta(s)-\beta(\sigma)]\,ds . \end{aligned} \tag{11} \]

Put \(s-\sigma=\varepsilon^2 t\). Then

\[ \frac{\beta(\sigma)}{(4\pi)^{n/2}} \int_\sigma^\tau \frac{1}{(s-\sigma)^{n/2}}e^{-\varepsilon^2/4(s-\sigma)}\,ds = \frac{\beta(\sigma)}{(4\pi)^{n/2}} \int_0^\infty \frac{1}{t^{n/2}}e^{-1/4t}\,dt +\omega(\varepsilon,\sigma,\tau), \]

where \(\omega(\varepsilon,\sigma,\tau)\) is bounded for \(\tau-\sigma\leqslant\varepsilon\) and has magnitude of order \(o(1)\) for \(\tau-\sigma>\varepsilon\). Making the substitution \(x=1/4t\), we obtain

\[ \frac{\beta(\sigma)}{(4\pi)^{n/2}} \int_0^\infty \frac{1}{t^{n/2}}e^{-1/4t}\,dt = \frac{\beta(\sigma)}{4\pi^{n/2}} \Gamma\left(\frac n2-1\right) =\alpha(\sigma). \tag{12} \]

Further, it is easy to verify that

\[ \varepsilon^{n-2}\int_\sigma^\tau \gamma(\sigma,\xi,s,0)\big|_{|\xi|=\varepsilon} [\beta(s)-\beta(\sigma)]\,ds =o(1). \tag{13} \]

V. A. Steklov Mathematical Institute
Academy of Sciences of the USSR

Received
19 V 1962

CITED LITERATURE

1. A. N. Kolmogoroff, Math. Ann., 104, 415 (1931).
2. E. F. Mishchenko, L. S. Pontryagin, Izv. AN SSSR, ser. matem., 25, 477 (1961).