MATHEMATICS

V. M. FRIDMAN

AN ITERATIVE PROCESS WITH MINIMAL ERRORS FOR A NONLINEAR OPERATOR EQUATION

(Presented by Academician V. I. Smirnov on 28 III 1961)

In the note (¹) an iterative method was considered for solving the operator equation

\[ Lx = 0, \tag{1} \]

in which \(Lx = Ax - y\), \(A\) is a linear bounded operator and \(y \in H\). The method consists in the fact that the successive approximations are computed by the formula

\[ x_{n+1} = x_n + \varepsilon_n z_n, \tag{2} \]

where

\[ z_n = \operatorname{grad} lx_n, \tag{3} \]

where the functional

\[ lx = (Lx, Lx), \tag{4} \]

and the coefficient \(\varepsilon_n\) is determined from the condition

\[ \|x_{n+1} - x^*\|^2 = \min, \tag{5} \]

where \(x^*\) is a solution of equation (1). An analogous method can be developed for a nonlinear operator equation.

Suppose that in a Hilbert space \(H\) a continuously differentiable (in the Fréchet sense) operation \(Lx\) is given and it is required to find a solution of equation (1). Let us define the gradient of the functional \(lx\). We have

\[ \frac{d}{d\varepsilon} l(x+\varepsilon z)\big|_{\varepsilon=0} = \frac{d}{d\varepsilon} \bigl(L(x+\varepsilon z),\, L(x+\varepsilon z)\bigr)\big|_{\varepsilon=0} = \]

\[ = 2\operatorname{Re}(Pz, Lx) = 2\operatorname{Re}(z, P^*Lx), \]

where the linear bounded operator

\[ P = L'(x). \]

By definition

\[ \operatorname{grad} lx = P^*Lx. \]

We take

\[ z_n = \operatorname{grad} lx_n = P_n^* Lx_n, \tag{6} \]

where

\[ P_n = L'(x_n). \tag{7} \]

The least value of the quantity \(\|x_{n+1}-x^*\|^2\) is given by

\[ \varepsilon_n=-\frac{(x_n-x^*,z_n)}{(z_n,z_n)} \]

or

\[ \varepsilon_n=-\frac{(x_n-x^*,P_n^*Lx_n)}{(P_n^*Lx_n,P_n^*Lx_n)} =-\frac{(P_n(x_n-x^*),Lx_n)}{(P_n^*Lx_n,P_n^*Lx_n)}. \]

If the element \(x_n\) is close to the solution \(x^*\), then \(L'(x_n)(x_n-x^*) \simeq Lx_n\). Thus, condition (4) is approximately satisfied by the coefficient

\[ \varepsilon_n=-\frac{(Lx_n,Lx_n)}{(P_n^*Lx_n,P_n^*Lx_n)}. \tag{8} \]

Concerning the convergence of the process (2), (6), and (8), the following is true.

Theorem 1. Let the norm of the residual of the initial approximation be \(\|Lx_0\|=\eta\), and in the sphere \((S)\)

\[ \|x-x_0\|\leq 2\frac{\eta}{m} \tag{9} \]

the operation \(Lx\) satisfies the conditions

\[ \frac{\|L'(x)z\|}{\|z\|}\geq m, \tag{10} \]

\[ \|L''(x)\|\leq K, \tag{11} \]

where

\[ \frac{\eta K}{m^2}=\gamma<1. \tag{12} \]

Then:

1. The equation \(Lx=0\) has in \((S)\) a unique solution \(x^*\).

2. The sequence of elements \(\{x_n\}\), connected by the recurrence formula

\[ x_{n+1}=x_n-\frac{\|Lx_n\|^2}{\|P_n^*Lx_n\|^2}\,P_n^*Lx_n, \tag{13} \]

converges monotonically and strongly to \(x^*\).

1. The following error estimate holds

\[ \|x_n-x^*\|\leq \|x_0-x^*\| \left[1-\frac{m^2}{M^2}(1-\gamma)\right]^{n/2} \leq \frac{\eta}{m} \left[1-\frac{m^2}{M^2}(1-\gamma)\right]^{n/2}, \tag{14} \]

where \(M=\max \|L'(x)\|\) for \(x\in S\).

Proof. The existence and uniqueness of the solution of the equation \(Lx=0\) in \((S)\) is established analogously to how this was done in the work of L. V. Kantorovich \((^2)\).

Let us prove the convergence of the process (13) to \(x^*\). From formula (13) it follows that

\[ \|x_{n+1}-x^*\|^2= \]

\[ =\|x_n-x^*\|^2 -2\frac{(Lx_n,Lx_n)}{(P_n^*Lx_n,P_n^*Lx_n)} (P_n(x_n-x^*),Lx_n) +\frac{(Lx_n,Lx_n)^2}{(P_n^*Lx_n,P_n^*Lx_n)}. \]

Using the expansion

\[ Lx^*=0=Lx_n+L'(x_n)(x^*-x_n)+\frac12 L''(\bar{x}_n)(x^*-x_n)^2, \]

where \(\bar{x}_n=\vartheta(x^*-x_n)\) and \(0\leq \vartheta \leq 1\), we rewrite this equality as

\[ \|x_{n+1}-x^*\|^2 = \|x_n-x^*\|^2 -\frac{(Lx_n,Lx_n)^2}{(P_n^*Lx_n,P_n^*Lx_n)}(1-\alpha_n), \]

where the real number

\[ \alpha_n=\frac{(\bar L''(x_n)(x_n-x^*)^2,Lx_n)}{(Lx_n,Lx_n)}. \]

Estimate:

\[ |\alpha_0|\leq \frac{\|L''(\bar x_0)(x_0-x^*)^2\|}{\|Lx_0\|} = \frac{\|L''(\bar x_0)(x_0-x^*)^2\|}{\|L'( \bar x_0)(x_0-x^*)\|}, \]

where \(\bar x=\theta(x_0-x^*)\), \(0\leq\theta\leq 1\).

The inequalities (10) and (11), which hold in the sphere \((S)\), also hold in the sphere \((N)\):

\[ \|x-x^*\|\leq\|x_0-x^*\|\leq \frac{\eta}{m}<\frac{K}{m}, \tag{15} \]

since in this case

\[ \|x-x_0\|\leq\|x-x^*\|+\|x_0-x^*\|<2\frac{\eta}{m}. \]

Obviously \(\bar x_0,\ \bar{\bar x}_0\in N\). Hence, by virtue of (10) and (11),

\[ |\alpha_0|\leq \frac{K\|x_0-x^*\|}{m} \leq \frac{K\eta}{m^2} =\gamma<1, \]

and for the error of the first approximation the estimate is valid

\[ \begin{aligned} \|x_1-x^*\|^2 &\leq \|x_0-x^*\|^2-\|Lx_0\|^2\frac{(1-\gamma)}{M^2} \\ &= \|x_0-x^*\|^2-\|L'(x_0)(x_0-x^*)\|^2\frac{(1-\gamma)}{M^2} \\ &\leq \|x_0-x^*\|^2 \left[1-\frac{m^2}{M^2}(1-\gamma)\right] <\|x_0-x^*\|^2 . \end{aligned} \]

The point \(x_1\) lies inside the sphere \((N)\). Therefore the reasoning can be repeated for the second approximation. We obtain that

\[ |\alpha_1|<\gamma, \]

\[ \|x_2-x^*\|^2 \leq \|x_1-x^*\|^2 \left[1-\frac{m^2}{M^2}(1-\gamma)\right] \leq \|x_0-x^*\| \left[1-\frac{m^2}{M^2}(1-\gamma)\right]^2 . \]

Thus, for the \(n\)-th approximation we arrive at estimate (14), from which it obviously follows that \(\lim x_n=x^*\), as was required to prove.

In the case where estimate (10) is known only at the point \(x_0\), the following may be useful.

Theorem 2. If \(\|Lx_0\|=\eta\),

\[ \frac{\|L'(x_0)z\|}{\|z\|}\geq m \tag{16} \]

and in the sphere \((S')\), defined by the inequality

\[ \|x-x_0\|\leq\frac{\eta}{m}, \tag{17} \]

the norm

\[ \|L''(x)\|\leq K, \]

and moreover

\[ \frac{\eta K}{m^2}=\frac{\gamma}{2}<\frac{1}{2}, \tag{18} \]

then the equation has in \((S')\) a unique solution \(x^*\), and the sequence of elements \(\{x_n\}\), constructed by formula (13), converges monotonically and strongly to \(x^*\), and the error estimate

\[ \|x_n-x^*\|\leq \|x_0-x^*\| \left[1-\frac{m^2}{4M^2}(1-\gamma)\right]^{n/2} \leq \frac{\eta}{m} \left[1-\frac{m^2}{4M^2}(1-\gamma)\right]^{n/2}. \tag{19} \]

holds.

Under condition (16), in the sphere \((S')\), defined by the inequality

\[ \|x-x^*\|\leq \|x_0-x^*\|\leq \frac{\eta}{m}<\frac{1}{2}\frac{K}{m}, \]

the quantity

\[ \frac{\|L'(x)z\|}{\|z\|} \geq m-K\|x_0-x^*\| \geq m-\frac{K\eta}{m} \geq m\left(1-\frac{\gamma}{2}\right) \geq \frac{m}{2}. \tag{20} \]

The rest of the proof of this theorem and of the preceding one is analogous. The existence and uniqueness of the solution of equation (1) under the indicated conditions were established in paper \((^2)\).

In the particular case when the operator \(Lx=Ax-y\), where \(A\) is a linear bounded operator, formula (13) takes the form (see \((^1)\))

\[ x_{n+1}=x_n-\frac{\|Lx_n\|^2}{\|A^*Lx_n\|^2}\,A^*Lx_n . \]

If, however, \(Lx=f(x)\) is a function of a complex variable, then relation (13) becomes Newton’s formula:

\[ x_{n+1}=x_n-\frac{|f(x_n)|^2}{|\bar f'(x_n)f(x)|^2}\,\bar f'(x_n)f(x_n) = x_n-\frac{f(x_n)}{f'(x_n)} . \]

A generalization of Newton’s formula is known, due to L. V. Kantorovich, consisting in the fact that, in constructing an approximate solution of the nonlinear operator equation (1), successive approximations are determined from the recurrent linear operator equation

\[ L'(x_n)(x_{n+1}-x_n)+Lx_n=0. \tag{21} \]

Formula (13) has, over equation (21), the advantage that it permits the successive approximations to be computed directly. At the same time, the method of L. V. Kantorovich may prove more effective in those cases when equation (21) is easily solvable.

Received
22 III 1961

REFERENCES

\(^{1}\) V. M. Fridman, DAN, 128, No. 3, 482 (1959).
\(^{2}\) L. V. Kantorovich, UMN, 3, 6 (28), 89 (1948).