Reports of the Academy of Sciences of the USSR

1. Volume 136, No. 1

MATHEMATICS

V. N. Kublanovskaya

ON SOME ALGORITHMS FOR SOLVING THE COMPLETE EIGENVALUE PROBLEM

(Presented by Academician V. I. Smirnov on 14 VII 1960)

Below we consider several new algorithms for solving the complete problem for real nonsingular matrices having distinct eigenvalues in absolute value
\(|\mu_1| > |\mu_2| > \cdots > |\mu_n| > 0\). In all these algorithms a sequence of left triangular matrices \(\Lambda_k = (l_{ij}^{(k)})\) is constructed by multiplying a certain auxiliary sequence of matrices \(A_k\) by orthogonal matrices \(\tau_k = (t_{ij}^{(k)})\). The construction of the latter can be carried out from the given matrix \(A\), for example, as products of elementary rotations or reflections. This construction is indicated explicitly in the book \((^1)\).

1. Algorithm for solving the complete eigenvalue problem for the matrices \(A'A\) and \(AA'\).

Let

\[ \begin{aligned} A_1 &= A, & \Lambda_1 &= A_1\tau_1,\\ A_2 &= \Lambda'_1\tau_1, & \Lambda_2 &= A_2\tau_2,\\ &\cdots & &\cdots\\ A_k &= \Lambda'_{k-1}\tau_{k-1}, & \Lambda_k &= A_k\tau_k,\\ &\cdots & &\cdots \end{aligned} \]

Then:

\[ \begin{aligned} \text{a) }\quad [l_{ii}^{(k)}]^2 &= \mu_i + O\left[\left(\frac{\mu_{i+1}}{\mu_i}\right)^k\right] + O\left[\left(\frac{\mu_i}{\mu_{i-1}}\right)^k\right], \quad i = 1,2,\ldots,n-1;\\ [l_{nn}^{(k)}]^2 &= \mu_n + O\left[\left(\frac{\mu_n}{\mu_{n-1}}\right)^k\right]; \end{aligned} \tag{1} \]

b) the columns of the matrices \(T_{2k-1} = \tau_1\tau_2\ldots\tau_{2k-1}\) and \(T_{2k} = \tau_1\tau_2\ldots\tau_{2k}\), for sufficiently large \(k\), are arbitrarily close to the eigenvectors of the matrices \(A'A\) and \(AA'\), respectively.

2. Algorithm for solving the complete eigenvalue problem for the matrix \(AA'\) with quadratic convergence.

Let

\[ \begin{aligned} A_1 &= A, & \Lambda_1 &= A_1\tau_1,\\ A_2 &= \Lambda'_1\tau_1, & \Lambda_2 &= A_2\tau_2,\\ A_3 &= \Lambda'_2\Lambda_2, & \Lambda_3 &= A_3\tau_3,\\ &\cdots & &\cdots\\ A_k &= \Lambda'_{k-1}\Lambda_{k-1}, & \Lambda_k &= A_k\tau_k,\\ &\cdots & &\cdots \end{aligned} \]

Then:

a)
\[ [l_{ii}^{(k)}]^2=\mu_i^{2^k-2} +O\left[\left(\frac{\mu_{i+1}}{\mu_i}\right)^{2^k-1}\right] +O\left[\left(\frac{\mu_i}{\mu_{i-1}}\right)^{2^k-1}\right], \]
\[ i=1,2,\ldots,n-1; \tag{2} \]
\[ [l_{nn}^{(k)}]^2=\mu_n^{2^k-2} +O\left[\left(\frac{\mu_n}{\mu_{n-1}}\right)^{2^k-1}\right]; \]

b) the columns of the matrix \(T_k=\tau_1\tau_2\cdots\tau_k\), for sufficiently large \(k\), are arbitrarily close to the eigenvectors of the matrix \(AA'\).

3. Algorithm for solving the complete problem for a nonsymmetric matrix \(A\) with real distinct eigenvalues.

Let
\[ \begin{aligned} A_1&=A, & \Lambda_1&=A_1\tau_1,\\ A_2&=\tau'_1\Lambda_1, & \Lambda_2&=A_2\tau_2,\\ \cdots&&\cdots\\ A_k&=\tau'_{k-1}\Lambda_{k-1}, & \Lambda_k&=A_k\tau_k,\\ \cdots&&\cdots \end{aligned} \]

Then:

a)
\[ [l_{ii}^{(k)}]^2=\mu_i^2 +O\left[\frac{\mu_{i+1}}{\mu_i}\right]^k +O\left[\frac{\mu_i}{\mu_{i-1}}\right]^k, \qquad i=1,2,\ldots,n-1; \]
\[ [l_{nn}^{(k)}]^2=\mu_n^2 +O\left[\frac{\mu_n}{\mu_{n-1}}\right]^k; \tag{3} \]

b) if, beginning with some \(k\), the orthogonal matrices \(\tau_k\) can be taken arbitrarily close to the identity matrix, then the matrix \(A_k\) becomes arbitrarily close to a left triangular matrix similar to the matrix \(A\):
\[ A_k=T'_kAT_k. \]

4. Algorithm with a shift for solving the complete eigenvalue problem for the matrix \(A\).

Algorithm 3 admits a modification with a shift which, for a certain choice of shifts, makes it possible to obtain quadratic convergence.

Let
\[ \varphi_k(t)=t(t-t_2)\cdots(t-t_k), \qquad t_k\to\sigma \quad \text{as } k\to\infty \]
and let the eigenvalues of \(A\), in some numbering, satisfy the relation
\[ |\lambda_1-\sigma|>|\lambda_2-\sigma|>\cdots>|\lambda_k-\sigma|. \]
A sequence of left triangular matrices is constructed:
\[ \begin{aligned} A_1&=A, & \Lambda_1&=A_1\tau_1,\\ A_2&=\tau'_1\Lambda_1-t_2E, & \Lambda_2&=A_2\tau_2,\\ A_3&=\tau'_2\Lambda_2-(t_3-t_2)E, & \Lambda_3&=A_3\tau_3,\\ \cdots&&\cdots\\ A_k&=\tau'_{k-1}\Lambda_{k-1}-(t_k-t_{k-1})E, & \Lambda_k&=A_k\tau_k,\\ \cdots&&\cdots \end{aligned} \]

Then:

a)
\[ [l_{ii}^{(k)}]^2=(\mu_i-t_k)^2 +O\left[\frac{\varphi_k(\mu_{i+1})}{\varphi_k(\mu_i)}\right] +O\left[\frac{\varphi_k(\mu_i)}{\varphi_k(\mu_{i-1})}\right], \]
\[ i=1,2,\ldots,n-1; \tag{4} \]
\[ [l_{nn}^{(k)}]^2=(\mu_n-t_k)^2 +O\left[\frac{\varphi_k(\mu_n)}{\varphi_k(\mu_{n-1})}\right]; \]

b) if, beginning with some \(k\), the orthogonal matrices \(\tau_k\) are taken to be as close as desired to the identity matrix, then the matrix \(A_k\) becomes as close as desired to a lower triangular matrix similar to \(A-t_kE\):

\[ A_k=T'_k(A-t_kE)T_k; \]

c) if at some step of the process (with a shift or without a shift) it is obtained that

\[ \left|\,l_{nn}^{(k)}-(\lambda_n-t_k)\,\right|<\varepsilon, \]

then, taking \(t_{k+1}=t_k+l_{nn}^{(k)}\) and, in passing to the \((k+1)\)-st step, choosing the orthogonal matrix \(\tau_{k+1}\) so that the element \(l_{nn}^{(k+1)}\) has the same sign as the element \(l_{nn}^{(k)}\), we obtain

\[ \left|\,l_{nn}^{(k+1)}-(\lambda_n-t_{k+1})\,\right|<\mu\varepsilon^2,\qquad \text{where } \mu=\mathrm{const}^*. \]

Remark 1. For a symmetric matrix \(A\), algorithms 1 and 3 coincide: the matrix \(A_k=T'_kAT_k\), for sufficiently large \(k\), becomes as close as desired to a diagonal matrix; moreover, the columns of the matrix \(T_k=\tau_1\tau_2\cdots\tau_k\) are as close as desired to the eigenvectors of \(A\).

Remark 2. Algorithm 1 has a direct connection with the \(LR\)-algorithm \((^2)\), applied to the matrix \(AA'\). Thus, the matrices

\[ L_k=(\Delta_1\Delta_2\cdots\Delta_{k-1})\Lambda_k(\Delta_1\Delta_2\cdots\Delta_k)^{-1} \]

and

\[ R_k=(\Delta_1\Delta_2\cdots\Delta_k)A'_k(\Delta_1\Delta_2\cdots\Delta_{k-1})^{-1} \]

coincide with the matrices of the same name in the \(LR\)-algorithm \((\Delta_r\) is the diagonal \(\Lambda_r)\). It is easy to establish the connection of algorithm 1 with the \(QD\)-algorithm \((^3)\) and to derive the formulas

\[ \lambda_i \approx \rho_i^{(m)}+\sigma_{i+1}^{(m)} +\frac{\rho_i^{(m)}\sigma_i^{(m)}}{\rho_i^{(m)}+\sigma_{i+1}^{(m)}-\rho_{i-1}^{(m)}-\sigma_i^{(m)}} +\frac{\rho_{i+1}^{(m)}\sigma_{i+1}^{(m)}}{\rho_i^{(m)}+\sigma_{i+1}^{(m)}-\rho_{i+1}^{(m)}-\sigma_{i+2}^{(m)}} \]

\[ (i=0,1,\ldots,n-1;\quad \sigma_n^{(m)}=\sigma_0^{(m)}=0;\quad m\text{ is the step number}), \]

which make it possible to obtain the eigenvalues of a symmetric matrix with accuracy up to \(\varepsilon^{3/2}\), if the corresponding values \(\rho_i\sigma_i\) and \(\rho_{i+1}\sigma_{i+1}\) have order of smallness equal to \(\varepsilon\).

Remark 3. Algorithm 1 is especially expedient to use for solving the complete eigenvalue problem for matrices of band structure (i.e., matrices for whose elements \(a_{ij}\) the relations \(a_{ij}=0\) for \(|i-j|>s\) hold, where \(s\) is an integer considerably smaller than the order of the matrix).

Remark 4. The algorithms described make it possible, for refining eigenvalues and eigenvectors, to apply the formulas of the Jacobi method \((^1)\).

In conclusion, the author expresses gratitude to D. K. Faddeev and V. N. Faddeeva for the opportunity to become acquainted with the manuscript \((^1)\).

Received
5 VII 1960

CITED LITERATURE

1. D. K. Faddeev, V. N. Faddeeva, Computational Methods of Linear Algebra, 1960.
2. H. Rutishauser, Nat. Bur. Stand., Appl. Math. Ser., No. 49, 47 (1958).
3. H. Rutishauser, Mitt. Inst. angew. Math., Eidgenoss. Hochschule Zürich, No. 7, 745 (1957).

* The proof of the assertion in point c) is analogous to the proof set forth in \((^1)\) for the \(QD\) algorithm with shift.