Full Text
Reports of the Academy of Sciences of the USSR
- Vol. 191, No. 3
MATHEMATICS
M. A. ALEKSIDZE
ON A METHOD FOR INVERTING SYMMETRIC MATRICES
(Presented by Academician A. N. Tikhonov, 23 VII 1969)
Let us consider two problems.
Problem A. It is required to solve the system of linear algebraic equations
\[ Ax = B, \]
or
\[ \sum_{j=1}^{n} a_{k,j}x_j = b_k \quad (k = 1, 2,\ldots,n). \tag{1} \]
Problem B. A functional equation is given,
\[ RC = F, \tag{2} \]
where \(R\) is a positive definite operator, and \(F\) is a given element in a certain functional space \(H\). The coefficients \(d_k\) of the best approximation, in the sense of the energy space \(H_R\), to the solution \(C\) of equation (2) by the series \(\sum_{k=1}^{n} d_kD_k\), where \(\{D_k\}\) is a system of linearly independent elements of the space \(H\), are sought from the Ritz system
\[ \sum_{j=1}^{n} (RD_k, D_j)d_j = (F,D_k). \tag{3} \]
Comparing systems (1) and (3), we find that the solutions of problems A and B coincide if the conditions
\[ (RD_k,D_j) = a_{k,j}, \qquad (F,D_k) = b_k \tag{4} \]
are satisfied.
However, the solution of problem B (and, consequently, of problem A) can be obtained without solving system (3), by preliminary orthonormalization of the vectors of the system \(\{D_k\}\).
Let \(\{R_k\}\) be a system of vectors orthonormal in the energy space \(H_R\), obtained by orthonormalizing the system \(\{D_k\}\):
\[ R_k = \sum_{i=1}^{k} \gamma_{k,i}D_i. \tag{5} \]
It will be shown below that, in order to compute the orthonormalization coefficients \(\gamma_{k,i}\), explicit expressions for the vectors \(D_k\) and the operator \(R\) are not required. They are determined, when conditions (4) are satisfied, only on the basis of the elements of the matrix \(\{a_{i,j}\}' = A\).
The coefficients \(l_k\) of the best approximation, in the sense of \(H_R\), to the element \(C\) by the series \(\sum_{k=1}^{n} l_kR_k\) are determined from the relations
\[ l_k = (RC,R_k) = (F,R_k). \tag{6} \]
Substituting (5) into (6), for \(l_k\) we obtain
\[ l_k = \sum_{i=1}^{k} \gamma_{k,i}b_i. \tag{7} \]
Knowing the coefficients \(l_k\), we easily find also the coefficients \(d_k\) of the best approximation of \(C\) by the series \(\sum_{k=1}^{n} d_k D_k\).
Indeed,
\[ \sum_{k=1}^{n} l_k R_k = \sum_{k=1}^{n} l_k \sum_{i=1}^{k} \gamma_{k,i} D_i = \sum_{k=1}^{n} d_k D_k, \]
where
\[ d_k=\sum_{i=k}^{n} \gamma_{i,k} l_i . \tag{8} \]
Substituting (7) into (8), we obtain a relation for computing \(d_k\) by means of the orthonormalization coefficients and the right-hand side of system (2):
\[ d_k=\sum_{j=1}^{n} c_{j,k} b_j, \tag{9} \]
where
\[ c_{j,k}=\sum_{i=r}^{n} \gamma_{i,j}\gamma_{i,k}, \qquad r=\max(k,j). \tag{10} \]
From formula (9) it is clear that, in essence, we have obtained the elements of the matrix \(A^{-1}\) inverse to \(A\).
Assuming uniqueness of the best, in the sense of \(H_R\), expansion of the element \(c\) in a series with respect to the system \(\{D_k\}\) (for this it is sufficient, for example, that \(H_R\) be a strictly normed (1) space*), we obtain that (9) gives the solution of problem B and, consequently, of problem A.
To compute the orthonormalization coefficients \(\gamma_{k,i}\), consider the orthogonal system \(\{\psi_k\}\), where
\[ \psi_k = D_k-\sum_{i=1}^{k-1} (D_k R_i) R_i, \tag{11} \]
and \(R_i\) is an element of the orthonormal system \(\{R_i\}\)
\[ R_i=\psi_k/\|\psi_k\|. \tag{12} \]
Taking into account (4) and (5), from (11) we obtain
\[ \psi_k = D_k - \sum_{i=1}^{k-1}\sum_{j=1}^{i} \gamma_{i,j}(D_k,D_j) \sum_{s=1}^{i}\gamma_{i,s}D_s = -\sum_{i=1}^{k} \alpha_{i,k}D_i, \tag{13} \]
where
\[ \alpha_{kk}=-1, \qquad \alpha_{i,k}= \sum_{s=i}^{k-1}\sum_{j=1}^{s} \gamma_{s,i}\gamma_{s,j}a_{k,j} \quad (i=1,2,\ldots,k-1). \tag{14} \]
* It may be assumed that \(R\) is the identity operator, \(R=E\). Then the energy space \(H_R\) becomes the Hilbert space \(L_2\), which is strictly normed.
From expression (13) we compute the denominator of formula (12)
\[ \|\psi_k\|=\left[(R\psi_k,\psi_k)\right]^{1/2} =\left[\sum_{i=1}^{k}\sum_{j=1}^{k}\alpha_{i,k}\alpha_{j,k}(RD_i,D_j)\right]^{1/2} =\left[\sum_{i=1}^{k}\sum_{j=1}^{k}\alpha_{i,k}\alpha_{j,k}a_{ij}\right]^{1/2}. \]
Substituting the last expression and (13) into (12), for the orthonormalization coefficients \(\gamma_{k,i}\) we obtain
\[ \gamma_{k,i} = -\frac{\alpha_{i,k}} {\left[\displaystyle\sum_{i=1}^{k}\sum_{j=1}^{k}\alpha_{i,k}\alpha_{j,k}a_{ij}\right]^{1/2}} \qquad (k=1,2,\ldots,n;\ i=1,2,\ldots,k). \tag{15} \]
Thus, the algorithm for obtaining the solution of system (1) is as follows. First, from (15), \(\gamma_{1,1}=1/\sqrt{a_{11}}\) is obtained; then from (14), \(\alpha_{12}=\gamma_{11}^2 a_{21}=a_{21}/a_{11}\); then again from (15), \(\gamma_{21}\) and \(\gamma_{22}\), and from (14), \(\alpha_{32}\) and \(\alpha_{31}\), etc. After all \(\gamma_{k,i}\) \((k=1,2,\ldots,n;\ i=1,2,\ldots,k)\) have been obtained, the elements of the inverse matrix \(c_{j,k}\) are computed by formula (10). The solution of system (1) is obtained from formula (9).
By means of the first or second Gaussian transformation (2), the method can also be transferred to an arbitrary nonsingular matrix \(A\).
Let \(S\) denote an upper triangular matrix satisfying condition (2)
\[ A=S'S, \tag{16} \]
where \(S'\) is the matrix transposed to \(S\).
The solution of system (1) can be written in the form
\[ x=A^{-1}B=(S'S)^{-1}B=S^{-1}S'^{-1}B=S^{-1}(S^{-1})'B, \]
or, in expanded form,
\[ x_k=\sum_{j=1}^{n} b_j \sum_{i=1}^{n} \widetilde S_{j,i}\widetilde S_{i,k} = \sum_{j=1}^{n} b_j \sum_{i=r}^{n} \widetilde S_{j,i}\widetilde S_{k,i}, \]
where \(r=\max(k,j)\), \(\widetilde S_{j,i}\) and \(\widetilde S_{i,k}\) are elements of the matrices \(S^{-1}\) and \((S^{-1})'\). Taking into account that \(x_k\equiv d_k\) \((k=1,2,\ldots,n)\) and comparing the last equality with formulas (9) and (10), we find that \(\gamma_{i,j}=\widetilde S_{j,i}\) for all \(i\) and \(j\). Thus, finding the orthonormalization coefficients is equivalent to inverting the matrix \(S\) satisfying equality (16). Therefore, in order to obtain \(\widetilde S_{j,i}\), one can first obtain the elements of the matrix \(S\) (by the square-root method (2)), and then invert the triangular matrix \(S\).
It is easy, however, to see that the computation scheme given above will differ from such an algorithm, since formulas (14) and (15) make it possible to obtain \(\widetilde S_{j,i}\) without first computing the elements of the matrix \(S\).
The sum
\[ \sum_{i=1}^{k-1}(D_k,R_i)R_i, \tag{17} \]
represents the projection of the element \(D_k\) onto the subspace of vectors \(R_1,R_2,\ldots,R_{k-1}\), or, by virtue of the equivalence of the subspaces of vectors \(D_1,D_2,\ldots,D_{k-1}\) and \(R_1,R_2,\ldots,R_{k-1}\), expression (17) represents the projection of \(D_k\) onto the subspace of vectors \(D_1,D_2,\ldots,D_{k-1}\).
Assuming the elements of the system \(\{D_k\}\) are normalized, for the norm of the difference on the right-hand side of expression (11) we obtain
\[ \|\psi_k\|=\sin \alpha_k, \]
where \(\alpha_k\) is the angle between the vector \(D_k\) and the hyperplane passing through the vectors \(D_1, D_2, \ldots, D_{k-1}\). In orthonormalizing \(n\) elements in all, it proves necessary to divide by the quantity
\[ \prod_{k=1}^{n} \sin \alpha_k, \]
which is equal to the square root of the determinant \(|A|\) of system (1) (the Gram determinant \(|A|\) is equal to the square of the volume of the parallelepiped constructed on the vectors \(D_1, D_2, \ldots, D_n\)). Hence it is clear that, when the determinant of system (1) is small, Cramer’s formulas, in whose denominator stands the determinant of system (1), give a less stable computational scheme.
It is easy to see that, when the columns and rows are orthogonalized, the division is by a quantity less than \((|A|)^{1/3}\). Indeed, the Gram determinant for the columns of the matrix of system (1) is equal to \(AA'\) and, consequently, is much more poorly conditioned than the Gram determinant \(A\) for the system \(\{D_k\}\).
Computing Center
Academy of Sciences of the Georgian SSR
Tbilisi
Received
4 VII 1969
REFERENCES
- G. I. Akhiezer, Lectures on the Theory of Approximation, “Nauka,” 1965.
- D. K. Faddeev, V. N. Faddeeva, Computational Methods of Linear Algebra, Moscow, 1960.