MATHEMATICS

M. A. KRASNOSEL′SKII and Ya. B. RUTITSKII

ON SOME APPROXIMATE METHODS FOR SOLVING NONLINEAR OPERATOR EQUATIONS BASED ON LINEARIZATION

(Presented by Academician I. N. Vekua, 13 VII 1961)

Let \(P(x)\) be a nonlinear operator acting from a Banach space \(E\) into a Banach space \(E_1\) and having, on some open ball \(\Omega \subset E\), a Fréchet derivative \(P'(x)\). In this case the most natural approximate method for solving the equation

\[ P(x)=0 \tag{1} \]

is the Newton–Kantorovich method (see, for example, \((^1)\), where a sufficiently complete bibliography is given), which, as is known, consists in constructing successive approximations \(x_n\), each of which is a solution of the linearized equation

\[ P'(x_{n-1})(x-x_{n-1})+P(x_{n-1})=0. \tag{2} \]

Under certain conditions on the derivative \(P'(x)\) and on the initial approximation \(x_0\), the successive approximations \(x_n\) converge to a solution \(x^*\) of equation (1).

However, it is usually impossible to solve the linear equation (2) exactly, and one has to solve it approximately. This means that in fact one must apply methods of successive approximations different from the Newton–Kantorovich method.

1. Suppose that we seek an approximate solution of the linear equation

\[ Bx=b \tag{3} \]

by some fixed method. This method may be one or several steps of the method of steepest descent, or of the method of minimal residuals, or of the method of ordinary iterations, etc. In all these cases the transition from the initial approximation \(x_0\) to the “better” approximation \(x_1\) is determined by some nonlinear operator

\[ x_1=V(x_0;\,B,b). \tag{4} \]

The properties of the operator \(V\) are determined by the chosen method of approximate solution of equation (3).

If the method (4) is applied successively for the approximate solution of the linearized equation (2), then we obtain an iterative process described by the formulas

\[ x_n=V\bigl(x_{n-1};\,P'(x_{n-1}),\,P'(x_{n-1})x_{n-1}-P(x_{n-1})\bigr). \tag{5} \]

Thus, for example, if the operator \(P(x)\) acts in a Hilbert space and has, for all considered values of \(x\), a self-adjoint positive definite derivative \(P'(x)\), and if one step of the method of steepest descent is chosen as the method (5), then formulas (5) take the form

\[ x_n=x_{n-1} - \frac{\|P(x_{n-1})\|^2} {\bigl(P'(x_{n-1})P(x_{n-1}),\,P(x_{n-1})\bigr)} \,P(x_{n-1}). \tag{6} \]

This method (and some of its modifications) has been studied by various authors (see, for example, \((^{2,3})\)), proceeding from gradient considerations.

For solving equation (3) with a positive definite operator \(B\) acting in Hilbert space, one may apply the \(\alpha\)-methods proposed by M. A. Krasnosel’skii and S. G. Krein \((^{4})\) and described by the formulas

\[ x_n=x_{n-1}- \frac{(B^\alpha\Delta_{n-1},\Delta_{n-1})} {(B^{\alpha+1}\Delta_{n-1},\Delta_{n-1})}\Delta_{n-1}, \tag{7} \]

where \(\Delta_{n-1}=Bx_{n-1}-b\). For \(\alpha=0\) these formulas give the method of steepest descent; for \(\alpha=1\) they give the method of minimal residuals. For \(\alpha=-1\) the method in the form (7) is unrealizable.

Let the equation now to be solved be

\[ B_1x=b_1, \tag{8} \]

where \(B_1\) is an invertible, non-self-adjoint operator. Then this equation is equivalent to equation (3), in which \(B=B_1^{*}B_1\), \(b=B_1^{*}b_1\). If in this case, for equation (3), we write the formulas of the \(-1\)-method, we obtain the following method for the approximate solution of equation (8):

\[ x_n=x_{n-1}- \frac{\|B_1x_{n-1}-b_1\|^2} {\|B_2^{*}(B_1x_{n-1}-b_1)\|^2} B_1^{*}(B_1x_{n-1}-b_1). \tag{9} \]

This method was considered by V. M. Fridman \((^{5})\).

If now we write the method (5), in which the operator \(V(x_0;B_1,b_1)\) is one step according to the formulas (9), then we obtain the recurrent formulas

\[ x_n=x_{n-1}- \frac{\|P(x_{n-1})\|^2} {\|P'^{*}(x_{n-1})P(x_{n-1})\|^2} P'^{*}(x_{n-1})P(x_{n-1}). \tag{10} \]

This method has also been studied by various authors, proceeding from gradient considerations.

2. We now consider some general theorems on the convergence of the methods (5).

In what follows it is assumed everywhere that the Fréchet derivative \(P'(x)\) of the operator \(P(x)\) satisfies, on the set \(\Omega\), the Hölder condition

\[ \|P'(x_1)-P'(x_2)\|\le k\|x_1-x_2\|^\alpha \qquad (0<\alpha<1). \]

With respect to the operator \(V\) we shall assume that for all \(x_0\in\Omega\) (or at least for all \(x_0\) from some ball \(S_0\), which will be indicated below) the following condition is fulfilled: the approximate solution

\[ V(x_0)=V(x_0;P'(x_0),P'(x_0)x_0-P(x_0)) \]

and the exact solution \(\tilde{x}\) of the linear equation

\[ P'(x_0)x=P'(x_0)x_0-P(x_0) \]

are related by

\[ \|V(x_0)-\tilde{x}\|\le q\|x_0-\tilde{x}\|, \tag{11} \]

where \(0<q<1\).

Theorem 1. Suppose that for the initial approximation \(x_0\) the following conditions are satisfied:

1) The operator \(P'(x_0)\) has an inverse \(\Gamma_0=[P'(x_0)]^{-1}\), and \(\|\Gamma_0\|\le B_0\).

2) \(\|\Gamma_0P(x_0)\|\le\eta_0.\)

3)

\[ h_0\equiv B_0k\eta_0^\alpha \le \frac{\beta(q)} {(1+q)^\alpha}, \tag{12} \]

where \(\beta(q)\) is the root of the equation

\[ \frac{1}{(1-\beta)^{1+\alpha}} \left(q+\frac{\beta(1+q)^\alpha}{1+\alpha}\right)=1. \tag{13} \]

4) The ball \(S_0\)

\[ \|x-x_0\|\leq \frac{1+q}{1-\gamma}\eta_0, \]

where \(\gamma=(1-\beta(q))^{1/\alpha}\), is contained in \(\Omega\).

Then in the ball \(S_0\) there exists a solution \(x^*\) of equation (1), to which the successive approximations (5) converge.

If in condition (12) a strict inequality holds, then

\[ \|x_n-x^*\|\leq (q+\varepsilon_n)\|x_{n-1}-x^*\|, \]

where \(\varepsilon_n\to 0\).

If the operator \(\Gamma(x)\), inverse to \(P'(x)\), exists not only at the point \(x_0\), but also in some neighborhood \(S\) of it and

\[ \|\Gamma(x)\|\leq B \qquad (x\in S), \]

then the condition of Theorem 1 can be weakened. Let

\[ h_0= Bk\eta_0^\alpha,\qquad d_0=\frac{1+q^{1+\alpha}}{1+\alpha}h_0+q. \]

Define the sequence of numbers \(\eta_n,d_n,h_n\) by the recurrence formulas

\[ \eta_n=d_{n-1}\eta_{n-1},\qquad h_n=Bk\eta_n^\alpha,\qquad d_n=\frac{1+q^{1+\alpha}}{1+\alpha}h_n+q. \]

If \(d_0<1\), then the series

\[ 1+d_0+d_0d_1+d_0d_1d_2+\cdots \]

converges. Denote its sum by \(D\).

Theorem 2. Suppose

\[ h_0=Bk\eta_0^\alpha<\frac{1-q}{1+q^{1+\alpha}}(1+\alpha). \tag{14} \]

Suppose the ball \(S_0\)

\[ \|x-x_0\|\leq (1+q)D\eta_0 \]

is contained in \(S\).

Then in \(S_0\) there exists a solution \(x^*\) of equation (1), to which the successive approximations (5) converge.

It is not difficult to see that condition (14) is weaker than condition (12).

If the solution of equation (1) at each step is reduced to the solution of the linear equation

\[ P'(x_0)(x-x_{n-1})+P(x_{n-1})=0, \]

then we arrive at the modified Newton—Kantorovich method. Application of the operator \(V\) in this case leads to the following method of approximate solution of equation (1):

\[ x_n=V\bigl(x_{n-1};\,P'(x_0),\,P'(x_0)x_{n-1}-P(x_{n-1})\bigr). \tag{15} \]

For method (15) the following assertion can be formulated:

Theorem 3. Suppose in the conditions of Theorem 1

\[ h_0\equiv B_0k\eta_0^\alpha< \frac{(1-q)^{1+\alpha}}{1+q}\left(\frac{\alpha}{1+\alpha}\right)^\alpha. \]

Suppose the ball \(S_0\)

\[ \|x-x_0\|\leq N\eta_0, \]

where \(N\) is the smaller root of the equation

\[ \frac{1+q}{1+\alpha}h_0N^{1+\alpha}-(1-q)N+1=0, \]

is contained in \(\Omega\).

Then the successive approximations (15) converge to the solution \(x^*\) of equation (1). The rate of convergence is characterized by the inequality

\[ \|x_n-x^*\|\leq q_1\|x_{n-1}-x^*\|, \]

where \(q_1=(1+q)h_0N^\alpha+q\).

Obviously, for \(q=0\) the methods (5) and (15) turn, respectively, into the ordinary and modified Newton–Kantorovich methods. In this case Theorems 1–3 coincide with the corresponding theorems of B. A. Vertgeim \({}^{6}\), which generalize the known theorems of L. V. Kantorovich (Theorems 1 and 3) and the theorem of I. P. Mysovskikh (Theorem 2). The conditions of Theorems 1–3 are, naturally, more restrictive than the conditions of the corresponding theorems of B. A. Vertgeim. In this connection, the existence of solutions under the conditions of Theorems 1–3 also follows from the theorems of B. A. Vertgeim.

1. In applying iterative methods for the approximate solution of equations, the successive approximations are in fact found again with certain errors, caused by rounding errors, the use of formulas of numerical integration, etc. In this connection there arises the problem of investigating the behavior of such systematic errors. As for contraction operators, for the methods considered above one can show that, under certain conditions, such systematic errors do not accumulate.

Theorem 4. Let the successive approximations to the solution \(x^*\) of equation (1) be found from the formulas

\[ x_n=V\bigl(x_{n-1};P'(x_0),P'(x_0)x_{n-1}-P(x_{n-1})\bigr)+h_n, \tag{16} \]

where \(h_n\) is a random vector, \(\|h_n\|<\delta\) \((n=1,2,\ldots)\). Let

\[ h_0\equiv B_0k\eta_0^\alpha< \frac{(1-q)^{1+\alpha}\left(\dfrac{\alpha}{1+\alpha}\right)^\alpha}{(1+q)(1+\delta)^\alpha}, \]

and let condition (11) be satisfied in the ball

\[ \|x-x_0\|\leq N_1\eta_0, \]

where \(N_1\) is the smaller root of the equation

\[ \frac{1+q}{1+\alpha}h_0N^{1+\alpha}-(1-q)N+1+\delta=0. \]

Then for the successive approximations (16) the relation

\[ \varlimsup_{n\to\infty}\|x_n-x^*\|\leq \frac{\varlimsup_{n\to\infty}\|h_n\|}{1-q_1}, \]

holds, where \(q_1=(1+q)h_0N^\alpha+q\).

Analogous assertions are also valid for the methods (5).

Received
12 VII 1961

CITED LITERATURE

\({}^{1}\) L. V. Kantorovich, G. P. Akilov, Functional Analysis in Normed Spaces, 1959.
\({}^{2}\) L. Kivistik, Izv. AN EstSSR, Ser. Phys.-Math. and Tech. Sci., 9, No. 3, 229 (1960).
\({}^{3}\) M. Altman, Bull. Acad. Polon. Sci., Cl. III, 5, No. 11, 1031 (1957).
\({}^{4}\) M. A. Krasnosel’skii, S. G. Krein, Matem. sborn., 31, No. 2, 315 (1952).
\({}^{5}\) V. M. Fridman, DAN, 128, No. 3, 482 (1959).
\({}^{6}\) B. A. Vertgeim, DAN, 110, No. 5, 719 (1956).