MATHEMATICS

I. I. EREMIN

AN ITERATIVE METHOD FOR CHEBYSHEV APPROXIMATIONS OF INCONSISTENT SYSTEMS OF LINEAR INEQUALITIES

(Presented by Academician A. N. Kolmogorov, 8 XII 1961)

Let

[
f_j(x)-a_j \leq 0 \quad (j=1,\ldots,m)
\tag{1}
]

be an arbitrary (consistent or inconsistent) system of linear inequalities, in which

[
f_j(x)=\sum_{i=1}^{n} a_{ji}x_i \not\equiv 0
]

and (a_{ji}, a_j) are real numbers.

In the present note, for the system (1) an iterative process is constructed that converges to a solution of the system (1) in the case when it is consistent, and to a point of best (in the Chebyshev sense) approximation in the case when it is inconsistent.

I. To the system (1) let us associate the function (d(x)): if

[
\max_j [f_j(x)-a_j]=\Delta_x,
]

then

[
d(x)=
\begin{cases}
\Delta_x, & \Delta_x \geq 0,\
0, & \Delta_x < 0.
\end{cases}
]

This function is continuous and convex.

The number

[
\varepsilon_0=\inf_x d(x)
]

was called in note ((^1)) the defect of the system (1). The lower bound of the values of the function (d(x)) is attained; therefore

[
\varepsilon_0=\min_x d(x).
]

The defect of the system (1) can also be characterized as the lower bound of the nonnegative numbers (\varepsilon) for which the system

[
f_j(x)-a_j \leq \varepsilon \quad (j=1,\ldots,m)
]

is consistent.

In the case of inconsistency of the system (1) (this corresponds to (\varepsilon_0>0)), the number (\varepsilon_0) will also be called the Chebyshev deviation of this system, and any (x=x^0) satisfying the system

[
f_j(x)-a_j \leq \varepsilon_0 \quad (j=1,\ldots,m),
\tag{2}
]

a point of its Chebyshev approximation.

Definition. A sequence ({s_k}) of elements (s_k) of the (n)-dimensional vector space (R^{(n)}) will be called resolving for the system (1) if it converges to some solution of the system (2).

Let us specify a sequence ({\lambda_k}) of positive numbers (\lambda_k) such that

[
\lambda_k \to 0 \quad (k\to\infty)
]

and

[
\sum_{k=1}^{\infty} \lambda_k = +\infty
]

(the numbers (\lambda_k) will below be called iteration coefficients). For an arbitrarily chosen (s_0\in R^{(n)}), define a sequence

[
{s_k}
\tag{3}
]

inductively by the equality

[
s_{k+1}=s_k-\lambda_{k+1}d(s_k)e_{j_k};
\tag{4}
]

here

[
e_{j_k}=(a_{j_k1},a_{j_k2},\ldots,a_{j_kn})
]

is the normal to that hyperplane

[
f_{j_k}(x)-a_{j_k}=0,
]

for which

[
f_{j_k}(s_k)-a_{j_k}=d(s_k).
]

Generally speaking, such ...

there may be several such hyperplanes. We may agree that in this case the hyperplane with the smallest number is chosen.

Theorem A. For any choice of positive iteration coefficients (\lambda_k), satisfying the conditions (\lambda_k \to 0) ((k \to \infty)), (\sum_{k=1}^{\infty} \lambda_k = +\infty), the sequence (3) is resolving for system (1).

Let us note that if system (1) is consistent, then the theorem asserts the convergence of the sequence (3) to one of its solutions.

II. Relation (4) is transformed, as is easy to establish, into the form

[
s_{k+1}=s_k+\lambda_{k+1}|f_{j_k}|^2(q_k-s_k),
\tag{5}
]

where (q_k) is the projection of the point (p_k) onto the hyperplane (f_{j_k}(x)-a_{j_k}=0), for which (f_{j_k}(s_k)-a_{j_k}=d(s_k)), (d(s_k)\ne 0), and (q_k=s_k) if here (d(s_k)=0) (i.e., (s_k) is a solution of system (1)).

Put (\lambda'k=\lambda_k|f|^2). Since (\lambda'k\to 0) ((k\to\infty)) and (\sum\lambda'_k=+\infty), it follows, in view of Theorem A, that the sequence}^{\infty

[
{P_k},
\tag{6}
]

defined inductively by the equality

[
P_{k+1}=P_k+\lambda_{k+1}'(q_k-P_k)
]

(with arbitrary (P_0\in R^{(n)})), is resolving for system (1).

In this connection let us note that in the works ((^2,^3)) the sequence ({P'k}) was considered, defined (with arbitrary (P'_0\in R^{(n)})) inductively by the equality (P'=P'_k+\lambda(q_k-P'_k)), in which (\lambda) is a fixed iteration coefficient from the interval ((0,2)); (q_k) is the projection of the point (P'_k) onto the boundary hyperplane of that half-space, determined by one of the inequalities of system (1), which is farthest from (P'_k). In ((^2,^3)) it is proved that if system (1) is consistent, then the sequence ({P'_k}) converges to its solution.

It is not difficult to see that if system (1) is inconsistent, then, for a fixed iteration coefficient (\lambda), the sequence ({P'_k}) diverges (in contrast to the sequences (3) and (6)).

III. The choice of the iteration coefficient (\lambda_k) at each step should be subordinated to the problem of faster convergence of the sequence (3) (or (6)). It is clear that if (d(s_k)=f_{j_{k-1}}(s_k)-a_{j_{k-1}}) (recall that (d(s_{k-1})=f_{j_{k-1}}(s_{k-1})-a_{j_{k-1}})), then in computing (s_{k+1}) it is inexpedient to decrease the iteration coefficient participating in the preceding step, i.e., in the computation of (s_k). This consideration makes it possible to approach the computation of a resolving sequence ({s_k}) in the following way.

Let ({\lambda_k}) be a sequence of positive numbers defined earlier. Put (\mu_1=\lambda_1) and compute (s_1=s_0-\mu_1 d(s_0)e_{j_1}). If (\mu_l=\lambda_r), then (\mu_{l+1}=\lambda_r) when (d(s_l)=f_{j_{l-1}}(s_l)-a_{j_{l-1}}), and (\mu_{l+1}=\lambda_{r+1}) otherwise. Thus, here the choice of the iteration coefficient (\mu_{l+1}) is determined after the element (s_l) has been computed.

The sequence ({s_l}), computed in the manner described, is still resolving for system (1).

IV. In this section we shall dwell on elements of the proof of Theorem A.

Definition. A system (S) of points (x_\gamma\in R^{(n)}) ((\gamma) ranges over some set of indices (\Delta)) will be called weakly all-sided with respect to a point (x'\notin S) if for any hyperplane (f(x)-a=0), pro-

passing through (x'), there will be such (x_\alpha) and (x_\beta) ((\alpha,\beta \subset \Delta)) that (f(x_\alpha)-a \leq 0), (f(x_\beta)-a \geq 0).

Let us note that an irreducible (i.e., minimal) weakly two-sided subsystem of the system (S) with respect to (x') contains no fewer than 2 and no more than (n+1) points.

Lemma. A point (x') of the space (R^{(n)}) is a point of Chebyshev approximation of the inconsistent system (1) if and only if its projections onto those hyperplanes (f_j(x)-a_j=0) ((j=1,\ldots,m)) for which (f_j(x')-a_j=d(x')) constitute a weakly two-sided system with respect to (x').

Corollary. The Chebyshev deviation of the inconsistent system of linear inequalities (1) coincides with the Chebyshev deviation of some subsystem of it consisting of (n+1) inequalities (in ((^{1})) a more precise result is proved, namely: the Chebyshev deviation of an inconsistent system (1) of rank (r>0) coincides with the Chebyshev deviation of some subsystem of it of rank (r) consisting of (r+1) inequalities).

Omitting the proof of the lemma, we outline the proof of Theorem A. (We note that, in proving the theorem, it is immaterial to us whether we speak of sequence (3) or (6).)

1) The sequence (6) is bounded. Hence (\sup\limits_k |q_k-P_k|<+\infty) and (|P_{k+1}-P_k|\to 0) ((k\to\infty)). This leads to the existence of the limit (\lim\limits_{k\to\infty} P_k=x').

2) If (\inf\limits_k d(P_k)=0), then the system (1) is consistent and (x') is one of its solutions.

3) Let (\inf\limits_k d(P_k)>0), and let (i_1,i_2,\ldots,i_l) be the numbers of those inequalities of the system (1), each of which participates infinitely many times in the formation of the sequence ({P_k}). Then (d(x')=f_{i_s}(x')-a_{i_s}>0) ((s=1,\ldots,l)). The system of projections (x_s) ((s=1,\ldots,l)) of the point (x') onto the hyperplanes (f_{i_s}(x)-a_{i_s}=0) ((s=1,\ldots,l)) is weakly two-sided with respect to (x'). Taking into account (d(x')>0), one may therefore conclude that the system (1) is inconsistent. In view of the lemma, (x') is a point of Chebyshev approximation of the system (1).

V. In conclusion we wish to indicate an iterative method (analogous to that considered above) for finding the Chebyshev center of a finite system (S) of points (x_s\in R^{(n)}).

Let (P_0) be an arbitrary point of (R^{(n)}). Define the sequence ({P_k}) inductively by the equality
[
P_{k+1}=P_k+\lambda_{k+1}(x_{s_{k+1}}-P_k),
]
where (x_{s_{k+1}}) is the point of (S) farthest from (P_k), (\lambda_k\to 0) ((k\to\infty)), and
[
\sum_{k=1}^{\infty}\lambda_k=+\infty .
]

The sequence ({P_k}) converges to some point (P'). This point is the Chebyshev center of the system of points (S), i.e., the center of the ball of smallest radius containing all points of the set (S).

The proof of this assertion is analogous to the proof of Theorem A.

Sverdlovsk Branch
of the V. A. Steklov Mathematical Institute
Academy of Sciences of the USSR

Received
8 XII 1961

CITED LITERATURE

1. I. I. Eremin, DAN, 138, No. 6, 1280 (1961).
2. S. Agmon, Canad. J. Math., 6, 382 (1954).
3. T. S. Motzkin, I. J. Schoenberg, Canad. J. Math., 6, 393 (1954).