UDC 518.517.948

V. D. Charushnikov

On Universally Optimal Iterative Processes

(Presented by Academician S. L. Sobolev on 15 IV 1970)

1°. In this note we consider iterative processes for the solution of operator equations

\[ Au=f \tag{1} \]

in a Hilbert space \(H\). Here \(A\) is a linear self-adjoint and positive definite operator mapping the space \(H\) into itself. The desired solution and its approximations may be regarded as elements of an entire scale of Hilbert spaces \(\{H_\alpha\}\) (\(\alpha\) is any real number), in which the scalar product and norm are defined, respectively, by the equalities

\[ (u,v)_\alpha=(A^\alpha u,v), \qquad \|u\|_\alpha=\sqrt{(u,u)_\alpha}. \tag{2} \]

By a universal iterative process we shall mean an iterative process

\[ u_{n+1,\alpha}=u_0+P_{n,\alpha}(A)[f-Au_0], \tag{3} \]

where \(u_0\) is the initial approximation, and the sequence of polynomials \(P_{n,\alpha}\) is constructed for an entire class of operator equations (1) with self-adjoint, positive-definite operators whose spectrum is contained in the fixed interval \([m,M]\) \((0<m<M)\). Fixing \(n\), we shall, in the space \(H_\alpha\), among all universal algorithms (3), seek a universally optimal algorithm

\[ u^0_{n+1,\alpha}=u_0+P^0_{n,\alpha}(A)[f-Au_0], \tag{4} \]

i.e. one such that

\[ \|u^0_{n+1,\alpha}-u\|_\alpha \leq \|u_{n+1,\alpha}-u\|_\alpha . \tag{5} \]

Such an algorithm always exists. We shall construct iterative processes which, in the scale of spaces \(\{H_\alpha\}\) generated by a fixed self-adjoint positive-definite operator \(A\), will be universally optimal in order. The iterative process

\[ v^0_{n+1}=v_0+P^0_n(A)[f-Av_0] \tag{6} \]

will be called optimal in order in the space \(H_\alpha\) if

\[ \overline{\lim}_{h\to\infty} \frac{\|u-v^0_{n+1}\|_\alpha}{\|u-u^0_{n+1,\alpha}\|_\alpha}<\infty . \tag{7} \]

The iterative process (6) will be called universally optimal in order in the scale of spaces \(\{H_\alpha\}\) if relation (7) holds for every \(\alpha\).

Let us note that the content of this note is suggested by the problems of the theory of cubature formulas (see also \((^5,^6)\)).

2°. Thus, suppose that in a Hilbert space \(H\) a class of operator equations (1) is given with self-adjoint positive-definite operators whose spectrum is contained in the interval \([m, M]\) \((0<m<M)\). Obviously,
\(u-u_{n,\alpha}=[E-P_{n-1,\alpha}(A)\cdot A](u-u_0)\) (\(E\) denotes the identity operator). Hence,

\[ \|u-u_{n,\alpha}\|_\alpha^2 = \bigl(A^\alpha(E-P_{n-1,\alpha}(A)\cdot A)(u-u_0),\, (E-P_{n-1}(A)\cdot A)(u-u_0)\bigr). \tag{8} \]

Then, from the spectral theory of self-adjoint operators in a Hilbert space [1] it follows that

\[ \|u-u_{n,\alpha}\|_\alpha^2 = \int_m^M \lambda^\alpha\bigl(1-\lambda P_{n-1,\alpha}(\lambda)\bigr)^2\,d\nu(\lambda), \tag{9} \]

where

\[ \nu(\lambda)=\bigl(\varepsilon_\lambda(u-u_0),\,(u-u_0)\bigr), \tag{10} \]

and \(\varepsilon_\lambda\) is the projection operator, the spectral function of the operator \(A\). Represent the polynomial \(P_{n-1,\alpha}(\lambda)\) of degree \(n-1\) in the following form:

\[ P_{n-1,\alpha}(\lambda)=\sum_{m=0}^{n-1} c_{m,\alpha}p_{m,\alpha}(\lambda), \tag{11} \]

where \(p_{m,\alpha}(\lambda)\) are polynomials of degree \(m\) with leading coefficient equal to one, orthogonal with weight \(\lambda^{\alpha+2}\) on the interval \([m,M]\)

\[ \int_m^M \lambda^{\alpha+2}p_{m,\alpha}(\lambda)p_{l,\alpha}(\lambda)=0,\qquad l\ne m. \tag{12} \]

Thus, by virtue of (9),

\[ \|u-u_{n,\alpha}\|_\alpha^2 = \int_m^M \lambda^\alpha \left[ 1-\lambda\sum_{m=0}^{n-1}c_{m,\alpha}p_{m,\alpha}(\lambda) \right]^2 d\nu(\lambda). \tag{13} \]

It is clear from this that the coefficients of the optimal polynomial \(P^0_{n-1,\alpha}\), by means of which the optimal iterative process is formed, are found from the formulas

\[ c^0_{m,\alpha} = \int_m^M \lambda^{\alpha+1}p_{m,\alpha}(\lambda)\,d\nu(\lambda) \Big/ \int_m^M \lambda^{\alpha+2}p_{m,\alpha}(\lambda)\,d\nu(\lambda), \qquad m=0,1,\ldots,n-1. \tag{14} \]

The fact that the \(c^0_{m,\alpha}\) indeed give a point of minimum follows, for example, from Sylvester’s criterion. Let us first clarify questions connected with convergence.

Theorem 1. If equations (1) are solvable, then the universally optimal iterative process (4) converges for any initial approximation \(u_0\), and the following estimates hold:

\[ \|u-u^0_{n,\alpha}\|_\alpha \le 2M^{\alpha/2}\|u-u_0\|_0 \left( \frac{\sqrt M-\sqrt m}{\sqrt M+\sqrt m} \right)^n, \qquad \alpha\ge 0, \tag{15} \]

\[ \|u-u^0_{n,\alpha}\|_\alpha \le 2m^{\alpha/2}\|u-u_0\|_0 \left( \frac{\sqrt M-\sqrt m}{\sqrt M+\sqrt m} \right)^n, \qquad \alpha<0. \tag{16} \]

Let us first prove the convergence of the optimal process in the case \(\alpha=0\). Let
\(u^0_{n,0}=u_0+P^0_{n-1,0}(A)[f-Au_0]\) be the optimal iterative process in \(H_0\). Construct a majorant process
\(u^*_{n,0}=u^*_0+P^*_{n-1,0}(A)[f-\)

— \(Au_0]\), and we choose the polynomial \(P^*_{n-1,0}(\lambda)\) in such a way that the polynomial \(Q_n(\lambda)=1-\lambda P^*_{n-1,0}(\lambda)\) deviates least from zero on the interval \([m,M]\).

It is known that, among all polynomials \(Q_n(\lambda)\) of degree \(n\), normalized by the condition \(Q_n(0)=1\), the polynomial
\[ R_n(\lambda,m,M)= \frac{T_n((2\lambda-M-m)/(M-m))} {T_n(-(M+m)/(M-m))} \]
deviates least from zero on the interval \([m,M]\) (\(T_n(\lambda)\) is the Chebyshev polynomial of the first kind of degree \(n\)). Therefore
\[ \|u-u^0_{n,0}\|_0^2\leq \|u-u^*_{n,0}\|_0^2 =\int_m^M |R_n(\lambda,m,M)|^2\,d\nu(\lambda) \leq \max_{\lambda\in[m,M]} |R_n(\lambda,m,M)|^2 \times \]
\[ \times \|u-u_0\|_0^2, \tag{17} \]
Further, since
\[ \max_{\lambda\in[m,M]} |R_n(\lambda,m,M)| \leq \frac{2(1-m/M)^n} {1-\sqrt{m/M}^{\,2n}+(1-\sqrt{m/M})^{2n}} \leq 2\left(\frac{1-\sqrt{m/M}}{1+\sqrt{m/M}}\right)^n, \]
it follows from (17) that
\[ \|u-u^0_{n,0}\|_0 \leq 2\|u-u_0\|_0 \bigl((\sqrt M-\sqrt m)^n/(\sqrt M+\sqrt m)\bigr)^n. \]
Thus, for the case \(\alpha=0\) the theorem is proved.

Passing to the general case, we see that the convergence of the universally optimal iterative process (4) and the corresponding estimates easily follow from the inequalities
\[ \|u\|_\alpha \leq M^{\alpha/2}\|u\|_0,\quad \alpha\geq 0, \]
\[ \|u\|_\alpha \leq m^{\alpha/2}\|u\|_0,\quad \alpha<0. \]

We now find iterative processes universally optimal in order in the scale of spaces \(\{H_\alpha\}\) generated by the self-adjoint positive-definite operator \(A\).

Theorem 2. Any iterative process that is optimal in some space of the scale \(\{H_\alpha\}\) will be universally optimal in order throughout the entire scale.

Let \(H_\beta\) be an arbitrary space of the scale and
\[ u^0_{n,\beta}=u_0+P^0_{n-1,\beta}(A)[f-Au_0] \]
be the optimal iterative process in this space. And let
\[ u^0_{n,\alpha}=u_0+P^0_{n-1,\alpha}(A)[f-Au_0] \tag{18} \]
be the optimal process in the space \(H_\alpha\). We shall show that (18) will be an order-optimal process in the space \(H_\beta\). Indeed, by virtue of the obvious inequality
\[ m^{(\beta-\alpha)/2}\|u\|_\alpha\leq \|u\|_\beta \leq M^{(\beta-\alpha)/2}\|u\|_\alpha \]
(we assume, for definiteness, that \(\beta>\alpha\))
\[ \|u-u^0_{n,\alpha}\|_\beta \leq M^{(\beta-\alpha)/2}\|u-u^0_{n,\alpha}\|_\alpha \leq \]
\[ \leq M^{(\beta-\alpha)/2}\|u-u^0_{n,\beta}\|_\alpha \leq \left(\frac{M}{m}\right)^{(\beta-\alpha)/2} \|u-u^0_{n,\beta}\|_\beta. \]
Thus,
\[ 1\leq \frac{\|u-u^0_{n,\alpha}\|_\beta} {\|u-u^0_{n,\beta}\|_\beta} \leq \left(\frac{M}{m}\right)^{(\beta-\alpha)/2}, \]
whence, by the arbitrariness of \(\beta\), our theorem follows.

Since the polynomials \(p_{m,\alpha}(\lambda)\) satisfy the recurrence relation known in the theory of orthogonal polynomials (see \((^2)\))
\[ p_{m,\alpha}(\lambda)=(\lambda-a_{m-1,\alpha})p_{m-1,\alpha}(\lambda)-b_{m-2,\alpha}p_{m-2,\alpha}(\lambda), \]
one can give, for the class of iterative processes under consideration, a simple and convenient computational scheme. This scheme is analogous to other schemes using recurrence relations \((^{2,3})\). The coefficients \(c_{m,\alpha}^{0}\), \(a_{m-1,\alpha}\), \(b_{m-2,\alpha}\) are expressed in terms of known quantities determined in the course of the iterative process.

Remark. If in equation (1) the operator is not self-adjoint, then one can pass to the equivalent equation
\[ A^{*}Au=A^{*}f \]
with a self-adjoint operator \((^4)\).

\(3^\circ\). The general considerations presented above can be applied to the solution of various problems of algebra and analysis. We give here only one example. Namely, in a domain \(\Omega\) with smooth boundary \(S\), consider the boundary-value problem
\[ Lu=-\sum_{i=1}^{n}\frac{\partial}{\partial x_i}\left(a_i(x)\frac{\partial u}{\partial x_i}\right)+c(x)u=f(x), \tag{19} \]
\[ u|_{s}=0. \tag{20} \]
Assume that \(a_i(x)\in C'(\Omega)\), \(c(x), f(x)\in C(\Omega)\), and \(a_i(x)>0\), \(c(x)\ge 0\) in \(\Omega\). Passing from equation (19) to the equation
\[ Ku=-\Delta^{-1}Lu=g,\qquad g=-\Delta^{-1}f(x). \]
Taking as the basic space \(W_2^{\circ(1)}(\Omega)\) of S. L. Sobolev, it is easy to show that \(K\) is a self-adjoint operator and, by virtue of the embedding theorems,
\[ m(u,u)\le (Ku,u)\le M(u,u)\qquad (0<m<M<\infty). \]
Therefore in the present case all the considerations set forth above are applicable. The computational scheme of the universally optimal process makes it possible to reduce the solution of the boundary-value problem for a general elliptic equation of second order to the solution of a sequence of Poisson equations.

Institute of Mathematics
Siberian Branch of the Academy of Sciences of the USSR
Novosibirsk

Received
7 IV 1970

CITED LITERATURE

\(^1\) L. V. Kantorovich, G. P. Akilov, Functional Analysis in Normed Spaces, 1959.
\(^2\) D. K. Faddeev, V. N. Faddeeva, Computational Methods of Linear Algebra, 1963.
\(^3\) V. E. Shimanskii, Methods for the Numerical Solution of Boundary-Value Problems, Part 1, Kiev, 1963.
\(^4\) M. A. Krasnosel’skii, G. M. Vainikko et al., Approximate Solution of Operator Equations, “Nauka,” 1969.
\(^5\) G. I. Marchuk, Yu. L. Kuznetsov, DAN, 181, No. 6 (1968).
\(^6\) E. Stiefel, Nat. Bur. Stand. Appl. Math. Ser., 49, 1 (1958).