MATHEMATICS

S. V. SIMEONOV

ON THE APPLICATION OF A PROCESS OF SUCCESSIVE APPROXIMATIONS TO THE SOLUTION OF CERTAIN TYPES OF FUNCTIONAL EQUATIONS

(Presented by Academician A. N. Kolmogorov on 2 VIII 1962)

In the work \((^5)\) the convergence of the process

\[ x_i' = x_{i-1} + \alpha (x_{i-1} - Ax_{i-1}) \tag{1} \]

was considered in those cases where the operation \(Ax\) is monotone, admits a first derivative \(A'(x)\) in the Fréchet sense, and maps the semi-ordered space \(C\) into itself, while in the case of monotonically decreasing operators it maps the semi-ordered space \(L_2\) into itself. The coefficient \(\alpha\) is determined from the condition of convergence of the process. The limiting element \(x_i = \lim x_i\) is a solution of the functional equation

\[ x = Ax. \tag{2} \]

Here a generalization will be given of the results obtained in \((^5)\), and their application to the approximate solution of certain types of integral equations and systems of nonlinear equations will also be shown. We shall consider equations of the form

\[ Ax = 0, \tag{3} \]

which also include equations of type (2), although some important properties of the latter, indicated in \((^5)\), do not appear when they are considered in the form (2).

If an operator \(A\), defined in a complete metric space \(X\), is a contraction operator, i.e.

\[ \rho(Ax, Ay) < \nu \rho(x, y) \quad (x, y \in X;\ \nu < 1), \tag{4} \]

then, as is known, the solution \(x^*\) of the functional equation (2) can be obtained by successive approximations \(x_{n+1} = Ax_n\). The error of the \(n\)-th approximation is estimated by the inequality

\[ \rho(x^*, x_n) \leqslant \frac{\nu^n}{1-\nu}\rho(x_1, x_0). \tag{5} \]

Equation (2) can also be represented in the form

\[ x = x + B(x - Ax), \tag{6} \]

where \(B\) is an arbitrary continuous operator mapping \(X\) into itself, and the equation \(Bx = 0\) has the unique solution \(x = 0\). Then the solution of equation (6) will also be a solution of equation (2).

If the operator \(B\) can be chosen so that \(Fx = x + B(x - Ax)\) is a contraction operation, then the solution of equation (6) can be found by successive approximations

\[ x_{n+1} = x_n + B(x_n - Ax_n). \tag{7} \]

For equation (3), instead of (7) we shall correspondingly have

\[ x_{n+1} = x_n + BAx_n. \tag{8} \]

Finding an operator \(B\) for an arbitrarily given operator \(A\) is a difficult problem, admitting a nonunique solution. Among all possible operators \(B\), it is desirable to choose the one for which (7) or (8) will have

the simplest form and, moreover, the process will converge sufficiently rapidly.

If in (8) we take \(B=[A'(x)]^{-1}\), the recurrence formula of Newton’s method \((^1)\) is obtained.

Suppose that \(Y\) is a semi-ordered space with a defined multiplication of an element by a number. If two operators \(P\) and \(Q\) are given in it such that \(Px<Qx\) for every positive \(x\in Y\), then we shall write \(P<Q\). We shall call positive numbers \(m\) and \(M\) bounds of the operator \(P\) if \(mI\leq P\leq MI\), where \(I\) is the identity operator in \(Y\).

Let us now consider the functional equation (3), given in the semi-ordered space \(M_T\) of real bounded functions \(x(t)\) \((t\in T)\), where \(T\) is an arbitrary infinite set. Without loss of generality we shall assume that the operator \(A\) is monotonically increasing \((^5)\), i.e., if \(x>y\), then \(Ax>Ay\) \((x,y\in M_T)\), and, for brevity, we shall call it monotone.

Theorem 1. If the operator \(A\) of the functional equation (3) is monotone, is given in the semi-ordered space \(M_T\), and admits a first derivative \(A'(x)\) in the sense of Fréchet such that \(mI\leq A'(x)\leq MI\) \((m>0;\ x\in M_T)\), then equation (3) has a unique solution \(x^*\), which can be determined by successive approximations (8), taking \(B=-\alpha I\), where \(0<\alpha<2/M\).

The best convergence in this case takes place for \(\alpha=1/M_{\mathrm{av}}\), where \(M_{\mathrm{av}}=\frac12(M+m)\). The rate of convergence is estimated by inequality (5), where the coefficient \(\nu\) is equal to the greater of the absolute values \(|1-\alpha M|\) and \(|1-\alpha m|\).

Remark. If \(K\) is the Lipschitz constant of the operator \(A\), then \(\alpha\) can be computed by the formula \(\alpha=1/K^*\). However, in the general case this constant is insufficient for determining the rate of convergence. It is sufficient only for functional equations of type (2), if the operator \(A\), given in the space \(L_2\), is monotonically decreasing in the sense indicated in (5), or if the scalar product \((A'(\bar{x})x,\bar{x})<0\) \((x,\bar{x}\in L_2)\). In this case theorem 3 of paper \((^5)\) holds.

For linear self-adjoint equations with strictly positive operators, given in the space \(L_2\), results analogous to those of theorem 1 were obtained by I. P. Natanson \((^2)\) (see also \((^{1,3})\)).

Theorem 1, together with the theorems of paper \((^5)\), makes it possible to obtain the solution of various concrete types of equations comparatively easily. Let us consider, for example, integral equations of the form

\[ Ax=\int_a^b G[t,s,x(s),x(t)]\,ds+f(t)=0, \]

where \(f(t)\) is an arbitrary bounded function.

Denote \(G[t,s,x(s),x(t)]\equiv G[t,s,u(s),v(t)]\), i.e. \(u(t)\equiv v(t)\equiv x(t)\). Then, if it is known that the solution \(x^*\) is contained in the set \(\Omega\subset M_T\) \((T\equiv [a,b])\), it is not difficult to verify that the operator \(A\) will be monotone in the cases when \(\dfrac{\partial}{\partial u}G[t,s,u(s),v(t)]\) and

\[ \int_a^b \frac{\partial}{\partial v}G[t,s,u(s),v(t)]\,ds \]

are positive functions of their arguments.

The coefficient \(\alpha\) is conveniently computed with the aid of the Lipschitz constant

\[ K\geq \int_a^b \left(\frac{\partial}{\partial u}+\frac{\partial}{\partial v}\right)G[t,s,u(s),v(t)]\,ds \quad (u(t)\equiv v(t)\equiv x(t)\in\Omega). \]

\[ {}^* \]
In the analogous formula given in the remark to theorem 2 of paper \((^5)\), it is printed \(\alpha=\dfrac{2}{\pm K-2}\); it should read \(\dfrac{\alpha}{\pm K-1}\).

If, however, the integral equation has the form

\[ x(t)=\int_a^b G(t,\ s,\ x(s))\,ds+f(t), \]

then it is more convenient to consider it as a functional equation of the form (2) and to seek the solution as indicated by (5). In this case the operation

\[ Ax=\int_a^b G(t,\ s,\ x(s))\,ds \]

will be monotonically increasing or monotonically decreasing if \(\dfrac{\partial}{\partial x}G(t,\ s,\ x)\) is, respectively, a positive or negative function of its arguments. Here it is also convenient to define the coefficient \(\alpha\) by means of the Lipschitz constant \(K\).

Finally, let us consider a system of nonlinear equations of the form

\[ f_k(x_1,\ x_2,\ldots,\ x_n)=0\qquad (k=1,\ 2,\ldots,\ n), \tag{9} \]

where the functions \(f_k\) are given in the \(n\)-dimensional Euclidean space \(E_n\). In this case it is convenient to take the operator \(B\) in the form of the diagonal matrix \(B=|-\alpha_1,\ -\alpha_2,\ldots,\ -\alpha_n|\). Then the recurrence formula for successive approximations will be

\[ x_k^{(i+1)}=x_k^{(i)}-\alpha_k f_k(x_1^{(i)},\ x_2^{(i)},\ldots,\ x_n^{(i)}) \qquad (k=1,\ 2,\ldots,\ n). \tag{10} \]

Theorem 2. Let the functions \(f_k(x_1,\ x_2,\ldots,\ x_n)\) of the system (9) satisfy the following conditions:

1) they are differentiable with respect to all variables \(x_j\);

2) the derivative \(\partial f_k/\partial x_k\) of each function is positive and bounded, i.e.

\[ m_k\leq \frac{\partial f_k}{\partial x_k}\leq M_k\qquad (m_k>0); \]

3) \(M'_k<m_k\), where

\[ M'_k\geq \sum_{j=1}^{n}{}' \left|\partial f_k/\partial x_j\right| \]

(the prime indicates that the summation is over all values of the index \(j\) except \(j=k\)).

Then there exists a unique solution \(x^*\) of the system (9). The successive approximations (10) converge to it independently of the choice of the initial element
\[ x^{(0)}=(x_1^{(0)},\ x_2^{(0)},\ldots,\ x_n^{(0)}), \]
provided that the coefficient \(\alpha_k\) is taken in the interval
\[ 0<\alpha_k<\frac{2}{M_k+M'_k}\qquad (k=1,\ 2,\ldots,\ n). \]

The best convergence in this case occurs for

\[ \alpha_k=\frac{2}{M_k+m_k}=\frac{1}{M_{k\,\mathrm{av}}}. \]

The rate of convergence is estimated by the inequality

\[ |x_k^*-x_k^{(n)}|\leq \frac{v_k^n}{1-v_k}\,|x_k^{(1)}-x_k^{(0)}|, \]

where \(v_k\) is the larger of the quantities
\[ |1-\alpha_k M_k|+\alpha_k M'_k \]
and
\[ |1-\alpha_k m_k|+\alpha_k M'_k. \]

Remark. If condition 2) of the theorem is replaced by the condition

\[ m\leq \frac{\partial f_k}{\partial x_k}\leq M \qquad (m>0;\ k=1,\ 2,\ldots,\ n) \]

and \(\alpha_k=\alpha=\mathrm{const}\) is adopted, then the process will also converge in the case when

\[ m>M'\geq \max_k \sum_{j=1}^{n}{}' \left|\frac{\partial f_j}{\partial x_k}\right|. \]

The formulas for \(\alpha\) and the convergence estimates remain the same.

the same; it is only necessary to replace in them \(m_k, M_k, M'_k\), and \(\nu_k\), respectively, by \(m, M, M'\), and \(\nu\).

Theorem 2 gives a generalization of the well-known convergence criterion for an iterative process in solving systems of linear algebraic equations to the solution of a system of nonlinear equations given in the form (9) \({}^{4}\). Moreover, the restrictions which it imposes on the system (9) are more general than those imposed by the known theorem on the convergence of an iterative process for solving nonlinear equations given in the form

\[ x_k=\varphi_k(x_1,x_2,\ldots,x_n)\qquad (k=1,2,\ldots,n), \]

namely

\[ \sum_{j=1}^{n}\left|\frac{\partial\varphi_k}{\partial x_j}\right|<1 \quad\text{or}\quad \sum_{j=1}^{n}\left|\frac{\partial\varphi_j}{\partial x_k}\right|<1, \tag{11} \]

whence it follows that necessarily

\[ \left|\frac{\partial\varphi_k}{\partial x_k}\right|<1. \]

Condition 3) of Theorem 2 coincides with conditions (11) only when each of the functions \(\varphi_k\) does not depend on the corresponding variable \(x_k\).

In cases where the method of solution set forth here is applicable, it is, possibly, usually easier than the application of Newton’s method \({}^{1}\), despite its slower convergence.

Engineering and Construction Institute
Sofia, Bulgaria

Received
2 VIII 1962

REFERENCES

\({}^{1}\) L. V. Kantorovich, G. P. Akilov, Functional Analysis in Normed Spaces, Moscow, 1959.
\({}^{2}\) I. G. Natanson, Scientific Notes of the Leningrad Pedagogical Institute named after A. I. Herzen, 64 (1948).
\({}^{3}\) G. N. Polozhii et al., Mathematical Practicum, Moscow, 1960.
\({}^{4}\) B. P. Demidovich, I. A. Maron, Foundations of Computational Mathematics, Moscow, 1960.
\({}^{5}\) S. V. Simeonov, DAN, 138, No. 5 (1961).