UDC 517.948+518.12

MATHEMATICS

S. I. AL’BER, Ya. I. AL’BER

THE METHOD OF DIFFERENTIAL DESCENT FOR SOLVING MULTIDIMENSIONAL VARIATIONAL PROBLEMS

(Presented by Academician L. S. Pontryagin on 2 III 1966)

In this article trajectories of differential descent in an infinite-dimensional Hilbert or Banach manifold are investigated. The results obtained are applied to the solution of nonlinear systems and to the minimization of variational functionals.

1. Local theorems. Let a functional \(u(x)\) of class \(C^2(G)\) be given in a domain \(G\) of a Hilbert space \(H\), and suppose it is required to find points of minimum of the functional inside the domain. Consider the trajectories of the differential descent equation

\[ dx / dt = -u \operatorname{grad} u / |\operatorname{grad} u|^2 . \tag{1} \]

Theorem 1. For any solution \(x(t)\) of equation (1) the identity holds

\[ u(x(\tau)) = u(x(t)) e^{-(\tau-t)} . \tag{2} \]

Proof. We have

\[ du / dt = (\operatorname{grad} u, dx / dt) = \]

\[ = -u / |\operatorname{grad} u|^2(\operatorname{grad} u, \operatorname{grad} u) = -u, \]

whence (2) follows.

Theorem 2. If at all points of the ball \(K_{\rho,x_0}: |x-x_0| < \rho\), belonging to \(G\), the inequalities

\[ 0 \leq u(x) - c \leq k|\operatorname{grad} u|^m \quad (m > 1,\ k = \text{const},\ c = \text{const}), \tag{3} \]

\[ u(x_0) - c \leq \varkappa \rho^{m/(m-1)}, \quad \varkappa = [(m-1)/2mk^{1/m}]^{m/(m-1)}, \tag{4} \]

are satisfied, then, first, the functional \(u(x)\) has in \(K_{\rho,x_0}\) a point of minimum \(\xi\), with \(u(\xi)=c\); second, equation (1) has a solution \(x(t)\) for \(0 \leq t < T\), for which \(x(0)=x_0\) and \(\lim_{t\to T} x(t)=\xi\),

\[ |x(t_2)-x(t_1)| \leq \tfrac12 \rho \left(e^{-\beta s_1} - e^{-\beta s_2}\right) \]

for \(0 \leq t_1 \leq t_2 < T\). In particular,

\[ |x(t)-\xi| \leq \tfrac12 \rho e^{-\beta s}. \]

Here \(s = \ln (u_0-c)/(u_0 e^{-t}-c)\), \(\beta=(m-1)/m\), \(T=\ln u_0/c\).

Let now a nonlinear operator \(F: G \to \hat H\) of class \(C^2(G)\) be defined in the domain \(G\), mapping the domain into the Hilbert space \(\hat H\). It is required to find solutions of the equation

\[ F(x)=0 \tag{5} \]

inside the domain \(G\). Consider the functional

\[ u(x)=|F(x)|^2 \tag{6} \]

and the differential descent equation composed for it

\[ dx/dt = -2|F(x)|^2 \operatorname{grad} |F(x)|^2 / |\operatorname{grad} |F(x)|^2|^2 . \tag{7} \]

Theorem 3. For any solution \(x(t)\) of equation (7) the identity is valid

\[ |F(x(\tau))| = F(x(t)) e^{-(\tau-t)} . \tag{8} \]

* In (⁷, ⁸), under close assumptions, the existence of a minimum of the functional and the convergence of gradient trajectories in a sufficiently small neighborhood of an isolated point of minimum were proved.

If, in the ball \(K_{\rho,x_0}\subset G\), the inequalities

\[ |F(x)|\leq k|\operatorname{grad}|F||^m \quad \text{for } |F(x)|\ne 0,\ m>1, \tag{9} \]

\[ |F(x_0)|\leq \varkappa \rho^{m/(m-1)},\quad \varkappa=\bigl[(m-1)/2mk^{1/m}\bigr]^{m/(m-1)}, \tag{10} \]

are satisfied, then equation (5) has a solution \(\xi\) in \(K_{\rho,x_0}\). There exists the limit of the trajectory \(x(t)\) \((0\leq t<\infty)\) of equation (7) passing through the point \(x_0\),

\[ \lim_{t\to\infty} x(t)=\xi, \]

and

\[ |x(\tau)-x(t)|\leq \frac12\rho\bigl(e^{-\beta t}-e^{-\beta \tau}\bigr) \quad \text{for } \beta=(m-1)/m,\ \tau>t. \tag{11} \]

In particular,

\[ |x(t)-\xi|\leq \frac12\rho e^{-\beta t}. \tag{12} \]

Remark. For simplicity we restrict ourselves to the case when \(G\) is a domain in a Hilbert space. It should be noted, however, that all the propositions and proofs obtained here carry over directly to the case of an infinite-dimensional Hilbert or Banach manifold \((^{1,2})\). To accelerate convergence it is sometimes convenient to consider the more general equation

\[ dx/dt=-[w(\cdot)]^{-1}\operatorname{grad} u, \tag{13} \]

in which \(w(\cdot)\) is a positive definite operator, generally speaking nonlinear.

Theorem 4. For any solution \(x(t)\) of the differential equation (13), the identity

\[ \int_t^\tau \left( w(\cdot)\frac{dx}{dt},\frac{dx}{dt} \right)\,dt = u(x(t))-u(x(\tau)), \tag{14} \]

holds; i.e., the trajectory \(x(t)\) is a curve of strict descent for the functional \(u(x)\).

In the particular case the theorem is valid for the functional \(u(x)=|F(x)|^2\).

2. Simple roots. Let us denote the Jacobi operator of the mapping \(F(x)\) at the point \(x\) by \(D_x=F'_x(x)\) and consider the positive self-adjoint operator \(A_x=D_xD_x^*\), where \(D_x^*\) is the operator adjoint to \(D_x\).

Definition 1. A solution \(\xi\) of the equation \(F(x)=0\) is called a nondegenerate simple root if zero does not belong to the spectrum of the operator \(A_\xi\).

Theorem 5. If \(\xi\) is a nondegenerate simple root of the equation \(F(x)=0\), then there exists a circular neighborhood \(K_{\rho,\xi}\) of the root in which all trajectories of equation (7) stabilize at \(\xi\), i.e., \(\lim_{t\to\infty}x(t)=\xi\), and inequalities (11) and (12) are satisfied.

Proof. We have

\[ |\operatorname{grad}|F||^2 = \bigl(D_xD_x^*(F/|F|),\,F/|F|\bigr). \tag{15} \]

From the definition of a nondegenerate root it follows that there exists a neighborhood \(K_{R,\xi}\) in which \(\xi\) is the unique solution and

\[ |\operatorname{grad}|F||^2\geq d>0,\quad d=\mathrm{const}. \tag{16} \]

Next, there exists a number \(\rho<R/2\) such that, for any \(x\in K_{\rho,\xi}\), in the ball \(K_{\rho,x_0}\) inequality (10) is satisfied for \(m=2\), and the proof follows from Theorem 3. In inequalities (11) and (12) one may put \(\beta=1\).

For the linear equation

\[ Ax=b \tag{17} \]

inequalities (3) and (4) are satisfied in the whole space for \(u(x)=|Ax-b|^2,\ c=0\). Therefore all trajectories of equation (1) stabilize at the unique solution \(\xi\).

3. Manifold of solutions

Suppose that the equation \(F(x)=0\) has an \(s\)-dimensional smooth manifold of solutions \(V^s\) of class \(C^2\).

Definition 2. The manifold of solutions \(V^s\) of equation (5) is called nondegenerate if, at every point \(\xi\in V^s\), zero is an isolated eigenvalue of finite multiplicity \(s\) of the operator
\[ A_\xi = D_\xi D_\xi^* . \]

Construct the normal bundle \({}^{(3,4)}\) \(B_\rho=(B_\rho,V^s,p,K_\rho,O_\infty)\) of the manifold \(V^s\) in the Hilbert space \(H\). Since for any compact domain \(u\) on the manifold \(V^s\) there exists a number \(\rho_0\) such that the layers of the bundle \(K_{\rho_0,\xi}\) at different points \(\xi\) of the domain \(u\) do not intersect, in the cylindrical neighborhood \(B_{\rho_0,\xi_0}=\rho^{-1}(u_{\rho_0,\xi_0})\), where \(u_{\rho_0,\xi_0}\) is the set of points \(\xi\in V^s\) for which \(|\xi-\xi_0|<\rho_0\), and \(\rho_0\) is sufficiently small, one can introduce normal coordinates \(x=(\xi,\eta)\). Here \((\xi)\in u_{\rho_0,\xi_0}\), \((\eta)\in K_\rho\). Carrying out the necessary estimates of \(\|\operatorname{grad}|F(x)|\|\) in the new coordinates, we obtain the theorem:

Theorem 6. If the equation \(F(x)=0\) has a nondegenerate manifold of solutions \(V^s\) and \(\xi_0\in V^s\), then there exists a cylindrical neighborhood \(B_{\rho,\xi_0}\) of the root \(\xi_0\), in which all trajectories of equation (7) are stabilized to the manifold of solutions and the inequalities (11) and (12) are satisfied.

Remark. The theorem on stabilization of the trajectories of equation (1) is proved analogously in the case when the functional \(u(x)\) has a nondegenerate manifold of minimum points.

4. Multiple roots of nonlinear systems

Let \(\xi\) be a solution of the finite-dimensional system \(F(x)=0\), and suppose the vector-function \(F(x)\) belongs to the class \(C^{k+1}(G)\). Expand \(F(x)\) in a neighborhood of \(\xi\) by Taylor’s formula:
\[ F(x)=D_\xi^{1}\eta^{[1]}+\ldots+D_\xi^{k-1}\eta^{[k-1]}+D_\xi^{k}\eta^{[k]} . \tag{18} \]

Definition 3. A solution \(\xi\) of the system \(F(x)=0\) is called a nondegenerate multiple root of multiplicity \(k\) if, in the expansion (18), the matrices \(D_\xi^{1},\ldots,D_\xi^{k-1}\) are equal to zero, while the resultant of the matrix \(D_\xi^{k}\) is different from zero. The root is called strongly nondegenerate if the resultant of the system of homogeneous polynomials
\[ \frac{\partial}{\partial x_i}\left|D_\xi^{k}\eta^{[k]}\right|^2 \qquad (i=1,\ldots,n) \]
is not equal to zero.

Theorem 7. If \(\xi\) is a strongly nondegenerate multiple root of multiplicity \(k\) of the system \(F(x)=0\), then there exists a circular neighborhood \(K_{\rho,\xi}\) of the root in which all trajectories of system (7) are stabilized to \(\xi\):
\[ \lim_{t\to+\infty} x(t)=\xi, \]
and the inequalities (11) and (12) are satisfied with coefficient \(\beta=1/k\).

5. Multidimensional variational problems

Let \(G\) be a bounded domain of the \(n\)-dimensional Euclidean space \(R\) with smooth boundary \(\partial G\). Denote
\[ \operatorname{grad}^{s} u(x)=\underbrace{(\operatorname{grad}\otimes\ldots\otimes\operatorname{grad})}_{s}u(x). \]

The jet of order \(k\) of the function \(u(x)\) will mean the vector-function
\[ \operatorname{jet}^{k}u(x)=\bigl(\operatorname{grad}^{0}u(x),\operatorname{grad}^{1}u(x),\ldots,\operatorname{grad}^{k}u(x)\bigr), \]
i.e. the vector-function with components \(\{D^{j}u(x)\}\), where \(j\) ranges over the set of multiindices satisfying the condition \(|j|\le k\). Suppose that the function \(F(x,\operatorname{jet}^{k}u)\) is defined and belongs to the class \(C^{2k}\), and for \(x\in G\), \(|\operatorname{jet}^{k}u|<+\infty\), and that the variational functional
\[ I(u)=\int_G F(x,\operatorname{jet}^{k}u)\,dx \tag{19} \]
is defined on functions \(u\in w_2^{(k)}(G)\) for which
\[ \operatorname{jet}^{k-1}u\big|_{\partial G}=\operatorname{jet}^{k-1}u_0\big|_{\partial G}. \tag{20} \]

where \(u_0\) is a fixed function in the domain \(G\), determining the boundary conditions \((^5,^6)\). The Euler–Lagrange equation for problem (19)—(20) has the form

\[ -\operatorname{grad} I(u) \equiv \sum_{|j|\le k}(-1)^{|j|+1}D^jF_{D^j u}=0 . \tag{21} \]

Consider the boundary-value problem for the corresponding equation of parabolic type

\[ \frac{\partial u}{\partial t} = \sum_{|j|\le k}(-1)^{|j|+1}D^jF_{D^j u} \tag{22} \]

under condition (20), with the initial condition \(u|_{t=0}=u_0\) and additional compatibility conditions.

Theorem 8. For the solution of problem (22), the identity \((^6)\) holds:

\[ \int_t^\tau dt\int_G\left(\frac{\partial u}{\partial t}\right)^2 dx = I(u(x,t))-I(u(x,\tau)) \qquad (\tau>t), \tag{23} \]

i.e., the parabolic trajectory is a curve of steepest descent for the variational functional (19)—(20).

Theorem 9. If equation (22) has a solution \(u(x,t)\in C^{2k,1}(G\times[0,t])\), then the function \(v(x,s)=u(x,t(s))\),

\[ t=\int_0^s \frac{I(v)\,ds}{\|\operatorname{grad} I(v)\|^2}, \]

is a solution of the equation

\[ \frac{\partial v}{\partial s} = \frac{I(v)}{\|\operatorname{grad} I(v)\|^2} \sum_{|j|\le k}(-1)^{|j|+1}D^jF_{D^j v}, \tag{24} \]

and for it the identity holds

\[ |Iv(x,\sigma)|=|I(v(x,s))|e^{-(\sigma-s)}. \tag{25} \]

The case \(k=1\) is considered in \((^6)\).

An approximate solution of nonlinear systems and variational problems can be obtained by approximately solving the differential descent equation.

Scientific Research Institute
of Applied Mathematics and Cybernetics

Scientific Research Radiophysics Institute
at Gorky State University
named after N. I. Lobachevsky

Received
2 III 1966

REFERENCES CITED

\(^1\) L. A. Lyusternik, V. I. Sobolev, Elements of Functional Analysis, “Nauka,” 1965.
\(^2\) S. Lang, Introduction to Differentiable Manifolds, N. Y., 1962.
\(^3\) L. S. Pontryagin, Proceedings of the Mathematical Institute named after V. A. Steklov, Academy of Sciences of the USSR, 45 (1955).
\(^4\) R. Thom, in: Fiber Spaces. Moscow, 1958, p. 293.
\(^5\) S. Smale, Ann. Math., 80, No. 2, 382 (1964).
\(^6\) S. I. Al’ber, Dokl. Akad. Nauk, 156, No. 4, 727 (1964).
\(^7\) T. Lezanski, Math. Ann., 152, No. 4, 271 (1963).
\(^8\) B. T. Polyak, Journal of Computational Mathematics and Mathematical Physics, 3, No. 4, 643 (1963).