UDC 513.88:513.83+517.948:518

MATHEMATICS

V. Ya. STETSKO

ON A METHOD FOR ACCELERATING THE CONVERGENCE OF ITERATIVE PROCESSES

(Presented by Academician A. N. Tikhonov, 14 IV 1967)

1. As is well known, in the theory of approximate methods an essential role is played by various techniques for accelerating iterative processes. In the present article a new technique is proposed for accelerating iterative processes, based on the use of ideas from spaces with a cone.

Let \(E\) be a real Banach space, partially ordered by a normal cone \(K\). The relations \(x \ge y\), \(x > y\), \(x \gg y\) mean, respectively, that \(x-y \in K\), \(x-y \in K\) and \(x \ne y\), and that \(x-y\) is an interior element of \(K\). We denote by \(\langle u_0, v_0\rangle\) the set of all elements \(x\) for which \(u_0 \le x \le v_0\); this set is called a cone segment. For further terminology and notation see \((^1,{}^2)\). Let \(B\) be a monotone operator such that, for some \(u_0, v_0\) \((u_0 < v_0)\), the inequalities \(Bu_0 \ge u_0\), \(Bv_0 \le v_0\) hold. Then, as is known \((^{2-4})\), under natural assumptions concerning the properties of the operator \(B\) and the cone \(K\), the operator \(B\) has in \(\langle u_0, v_0\rangle\) at least one fixed point \(x^*\). The sequences \(u_n = Bu_{n-1}\), \(v_n = Bv_{n-1}\) \((n=1,2,\ldots)\) converge to fixed points \(x^*\) and \(x^{**}\) of the operator \(B\), and
\(u_n \le x^* \le x^{**} \le v_n\) \((n=0,1,\ldots)\); in the case when \(x^* = x^{**}\), the sequences \(u_n, v_n\) give two-sided approximations to the unique fixed point of the operator \(B\) on \(\langle u_0, v_0\rangle\). Iterative methods of this kind have been studied in \((^{2-4})\); in \((^4)\) various applications of this method are also considered. Below a technique is proposed for accelerating the convergence of the sequences \(u_n\) and \(v_n\). The case of a nonmonotone operator \(B\) is also considered.

2. Let first \(Bx \equiv Ax+f\), where \(A\) is a linear positive operator \(A(E \to E)\), \(f \in E\). Suppose that \(B\) leaves invariant the cone segment \(\langle u_0, v_0\rangle\) and has on this segment a unique fixed point \(x^*\), to which the sequences \(u_n = Bu_{n-1}\), \(v = Bv_{n-1}\) converge. Denote by \(p_1\) and \(q_1\) such numbers for which

\[ u_1-v_0 \ge p_1(v_0-v_1), \qquad v_0-v_1 \ge q_1(u_1-u_0). \tag{1} \]

Obviously, \(p_1, q_1 \ge 0\). Let

\[ u_1^*=(u_1+p_1v_1)/(1+p_1), \qquad v_1^*=(v_1+q_1u_1)/(1+q_1). \tag{2} \]

Theorem 1. The operator \(B\) leaves invariant the cone segment
\(\langle u_1^*, v_1^*\rangle \subset \langle u_1, v_1\rangle\).

Corollary. The inequality \(u_1^* \le x^* \le v_1^*\) holds.

We note that, for \(p_1>0\) and \(q_1>0\),
\(u_1 < u_1^* \le x^* \le v_1^* < v_1\), i.e. the constructed elements \(u_1^*, v_1^*\) are situated closer to the solution than the elements \(u_1, v_1\). Since the operator \(B\) leaves invariant the cone segment \(\langle u_1^*, v_1^*\rangle\), the given technique can be applied again to the segment \(\langle u_1^*, v_1^*\rangle\), and so on. As a result we arrive at the following method for constructing two-sided approximations \(u_n^*, v_n^*\) \((n=1,2,\ldots)\) to the solution \(x^*\) of the equation \(x=Ax+f\). Suppose that \(u_{n-1}^*, v_{n-1}^*\) \((n \ge 2)\) have been constructed. Put

\[ \bar u_n=Au_{n-1}^{*}+f,\quad \bar v_n=Av_{n-1}^{*}+f. \]
Next choose nonnegative numbers \(p_n,q_n\) so that the inequalities
\[ \bar u_n-u_{n-1}^{*}\succcurlyeq p_n(v_{n-1}^{*}-\bar v_n),\quad v_{n-1}^{*}-\bar v_n\succcurlyeq q_n(\bar u_n-u_{n-1}^{*}) \]
hold, and let
\[ u_n^{*}=(\bar u_n+p_n\bar v_n)/(1+p_n),\quad v_n^{*}=(\bar v_n+q_n\bar u_n)/(1+q_n). \tag{3} \]

Then \(\bar u_n\preccurlyeq u_n^{*}\preccurlyeq x^{*}\preccurlyeq v_n^{*}\preccurlyeq \bar v_n\). The method described is an iterative process in which, after each step, the extraneous approximation is “improved,” which leads to an acceleration of convergence. It is easy to see that the indicated device for accelerating convergence leads, at the \(n\)-th step, to a contraction by a factor \(q\) of the conic segment \(\langle \bar u_n,\bar v_n\rangle\) containing the solution:
\[ (v_n^{*}-u_n^{*})=q(\bar v_n-\bar u_n), \]
where
\[ q=(1-p_nq_n)/(1+p_n)(1+q_n)\quad (n=1,2,\ldots). \]
Thus, convergence is indeed accelerated at every step if the numbers \(p_n\) and \(q_n\) are positive. The following theorem contains sufficient conditions for the positivity of these numbers.

Theorem 2. Let \(K\) be a solid cone, and let \(A\) be a linear operator such that from the inequality \(x>\theta\) there follows the inequality \(Ax\succ\theta\). Let \(p_1,q_1>0\). Then the numbers \(p_n,q_n\) \((n=2,3,\ldots)\) are positive.

In the case of spaces with a non-solid cone one can also indicate sufficient conditions ensuring accelerated convergence. The corresponding conditions are formulated in terms of the \(u_0\)-positivity of the operator \(A\).

As the examples show, the method proposed above substantially accelerates convergence, especially in the case when the spectral radius \(r(A)\) of the operator \(A\) is close to unity, i.e., when the ordinary iterative process converges slowly. Let us describe the implementation of the indicated method in the case when \(A\) is an integral operator with kernel \(K(t,s)\). Let \(K(t,s)\) be nonnegative and continuous jointly in the variables \(t,s\in\Omega\) (\(\Omega\) is a bounded closed set in the space \(E^m\)), and let \(f(t)\) be a function continuous on \(\Omega\). Suppose that there exist (the question of existence and of finding such functions is considered in item 4) two continuous functions \(u_0(t),v_0(t)\) such that
\[ u_1(t)\succcurlyeq \int_{\Omega}K(t,s)u_0(s)\,ds+f(t),\quad v_1(t)\preccurlyeq \int_{\Omega}K(t,s)v_0(s)\,ds+f(t). \]

Then the equation
\[ x(t)=\int_{\Omega}K(t,s)x(s)\,ds+f(t) \]
has a solution \(x^{*}(t)\). Moreover,
\[ u_1(t)\preccurlyeq x^{*}(t)\preccurlyeq v_1(t). \]
Let
\[ \min (u_1(t)-u_0(t))/(v_0(t)-v_1(t))\succcurlyeq p_1, \]
\[ \min (v_0(t)-v_1(t))/(u_1(t)-u_0(t))\succcurlyeq q_1. \]
Then
\[ u_1^{*}(t)=[u_1(t)+p_1v_1(t)]/(1+p_1)\preccurlyeq x^{*}(t)\preccurlyeq \]
\[ \preccurlyeq [v_1(t)+q_1u_1(t)]/(1+q_1)=v_1^{*}(t) \]
and
\[ u_1^{*}(t)\succcurlyeq \int_{\Omega}K(t,s)u_1^{*}(s)\,ds+f(t),\quad v_1^{*}(t)\preccurlyeq \int_{\Omega}K(t,s)v_1^{*}(s)\,ds+f(t). \]

Starting from \(u_1^{*}(t),v_1^{*}(t)\), we similarly obtain more accurate two-sided approximations to the solution \(x^{*}(t)\).

1. Let now \(Bx\equiv Ax+f\), where \(A=A_1+A_2\), \(A_1-A_2\) are positive linear operators acting in \(E\). Suppose that for some

for \(u_0, v_0 \in E\) (\(u_0 \ll v_0\)) the inequalities
\[ A_1u_0 + A_2v_0 + f \gg u_0, \qquad A_1v_0 + A_2u_0 + f \ll v_0 \]
hold.

Then the operator \(B\) leaves invariant the conical segment \(\langle u_0, v_0\rangle\). If, moreover, the segment \(\langle u_0, v_0\rangle\) contains a unique fixed point \(x^*\) of the operator \(B\), then the sequences
\[ u_n=A_1u_{n-1}+A_2v_{n-1}+f,\qquad v_n=A_1v_{n-1}+A_2u_{n-1}+f \]
converge (under natural assumptions) to this fixed point, and \(u_n \ll x^* \ll v_n\). Here we shall consider the question of accelerating the convergence of the sequences \(u_n, v_n\). Let \(m_1 \geq 0\) satisfy the inequalities
\[ v_0-v_1 \gg m_1(u_1-u_0), \qquad u_1-u_0 \gg m_1(v_0-v_1). \]
Put
\[ u_1^*=(u_1+m_1v_1)/(1+m_1), \qquad v_1^*=(v_1+m_1u_1)/(1+m_1). \]

Theorem 3. The inequalities
\[ A_1u_1^*+A_2v_1^*+f \gg u_1^*, \qquad A_1v_1^*+A_2u_1^*+f \ll v_1^* \]
hold.

Corollary. The operator \(B\) leaves invariant the conical segment
\[ \langle u_1^*, v_1^*\rangle . \]

Starting from the segment \(\langle u_1^*, v_1^*\rangle\), construct a sequence of nested segments invariant with respect to the operator \(B\). Suppose the segment \(\langle u_{n-1}^*, v_{n-1}^*\rangle\) (\(n\geq 2\)) has been constructed, and
\[ \bar u_n=A_1u_{n-1}^*+A_2v_{n-1}^*+f \gg u_{n-1}^*, \qquad \bar v_n=A_1v_{n-1}^*+A_2u_{n-1}^*+f \ll v_{n-1}^* . \]
Choose \(m_n\geq 0\) so that
\[ v_{n-1}^*-\bar v_n \gg m_n(\bar u_n-u_{n-1}^*), \qquad \bar u_n-u_{n-1}^* \gg m_n(v_{n-1}^*-\bar v_n) \]
and set
\[ u_n^*=(\bar u_n+m_n\bar v_n)/(1+m_n), \qquad v_n^*=(\bar v_n+m_n\bar u_n)/(1+m_n). \]
Then, by Theorem 3,
\[ A_1u_n^*+A_2v_n^*+f \gg u_n^*, \qquad A_1v_n^*+A_2u_n^*+f \ll v_n^*, \]
whence it follows that the operator \(B\) leaves invariant the conical segment
\[ \langle u_n^*, v_n^*\rangle . \]

Theorem 4. Suppose \(r(A_1-A_2)<1\) and one of the two conditions holds: 1) \(A_1, A_2\) are completely continuous operators; 2) the cone \(K\) is regular. Then the sequences \(u_n^*, v_n^*\) (\(n=1,2,\ldots\)) converge to the unique fixed point \(x^*\) of the operator \(B\), and \(u_n^* \ll x^* \ll v_n^*\).

We note that the passage from \(\bar u_n,\bar v_n\) to \(u_n^*,v_n^*\) constitutes an acceleration of the convergence of the usual iterative process.

4. Application of the results of the preceding sections requires finding a conical segment invariant with respect to the operator under study. A number of results in this direction were obtained in \((^4)\). Here we shall confine ourselves to only one simple assertion. For simplicity we shall assume that the cone \(K\) is solid and the operator \(A\) is positive.

Theorem 5. Let \(Bx\equiv Ax+f\), where \(A\) is a positive linear operator, and let \(r(A)<1\). Then for the operator \(B\) there exists an invariant conical segment \(\langle u_0,v_0\rangle\), \(v_0\gg \theta\). Conversely, if for the operator \(B\) there exists an invariant conical segment \(\langle u_0,v_0\rangle\), \(v_0\gg \theta\), then \(r(A)<1\).

Corollary. Let \(A\) be indecomposable \((^5)\), \(f>\theta\). Then the operator \(B\) has an invariant conical segment \(\langle u_0,v_0\rangle\) if and only if \(r(A)<1\).

We note that the conical segment whose existence is established in Theorem 5 can be constructed explicitly. To do this one must find an interior element \(u \in K\) such that \(Au \ll \lambda u\), where \(\lambda < 1\) (the procedure for constructing such an element is described in \((6)\)). Suppose that such an element \(u\) has been found. We choose \(n\) so that \(\lambda + \|f\|_{u}/n \le 1\). Then the operator \(Bx \equiv Ax + f\) leaves invariant the conical segment \(\langle -nu, nu\rangle\).

5. Concluding remarks. In an analogous way, one can obtain acceleration of convergence in some other iterative methods as well, for example in the Seidel method. The method described is also applicable in the case of equations with nonlinear operators of a certain type, namely with monotone concave (convex) operators \((2)\).

The author expresses his gratitude to his supervisor M. A. Krasnosel’skii.

Voronezh Civil Engineering Institute

Received
11 V 1967

REFERENCES

1. M. G. Krein, M. A. Rutman, UMN, 3, No. 1 (1948).
2. M. A. Krasnosel’skii, Positive Solutions of Operator Equations, Moscow, 1962.
3. L. V. Kantorovich, B. Z. Vulikh, A. G. Pinsker, Functional Analysis in Partially Ordered Spaces, 1950.
4. L. Collatz, Funktionalanalysis und numerische Mathematik, Berlin, 1964.
5. V. Ya. Stetsenko, UMN, 21, issue 5 (131) (1966).
6. V. Ya. Stetsenko, Matem. sborn., 67 (109), 2 (1965).