MATHEMATICS

V. A. BULAVSKII

AN ITERATIVE METHOD FOR SOLVING A LINEAR PROGRAMMING PROBLEM

(Presented by Academician V. I. Smirnov, 28 XI 1960)

Let (R_n) be an (n)-dimensional Euclidean space consisting of vectors
(h=(h^{(1)}, h^{(2)}, \ldots, h^{(n)})). We formulate the linear programming problem as follows. A system of linear inequalities is given

[
\sum_{j=1}^{n} a_j^{(i)} h^{(j)} \geqslant p^{(i)}, \qquad i=1,2,\ldots,m;
]

[
h^{(j)} \geqslant 0,\quad j=1,2,\ldots,n,
\tag{1}
]

where (a_j^{(i)}, p^{(i)}) are given numbers. In addition, a vector (c\in R_n) is given. The system (1) defines a closed convex set (Q\subset R_n). It is required to find a vector (h_0\in Q) such that ((c,h_0)\leqslant (c,h)) for all (h\in Q). We shall call this problem problem I.

In addition to it, we pose another problem II: find a vector (h_\sigma\in Q) such that
((c+\sigma Bh_\sigma,h_\sigma)\leqslant (c+\sigma Bh_\sigma,h)) for all (h\in Q). Here (B) is a given matrix and (\sigma>0) is a number.

The proposed method for solving problem I consists in replacing it by problem II for some small (\sigma) and a specially chosen matrix (B). To solve the latter problem an iterative process is constructed.

Lemma. If (Q') is a nonempty, bounded, convex, closed set in (R_n), and (F) is a continuous operation from (Q') into (R_n), then there exists a vector (h'\in Q') such that ((Fh',h')\leqslant (Fh',h)) for all (h\in Q').

The proof of this assertion can be obtained by slightly modifying the arguments in Theorem 3 of ((^3)).

Theorem 1. If there exists a vector (h_0) solving problem I, and the matrix (B) is positive definite: ((Bh,h)\geqslant \gamma |h|^2), then for any (\sigma>0) there exists a vector (h_\sigma) solving problem II, and moreover
[
|h_\sigma|\leqslant \frac{|B|}{\gamma}|h_0|.
]

Proof. Adjoin to (1) the inequality

[
\sum_{j=1}^{n} h^{(j)} \leqslant \frac{\sqrt{n}}{\gamma}|B|\cdot |h_0|+1.
]

The system of inequalities thus obtained defines a domain (Q'). By the lemma, there is a vector (h_\sigma\in Q') such that
((c+\sigma Bh_\sigma,h_\sigma)\leqslant (c+\sigma Bh_\sigma,h)) for all (h\in Q'). Since (h_0\in Q'), we have
((c+\sigma Bh_\sigma,h_\sigma)\leqslant (c+\sigma Bh_\sigma,h_0)). Taking into account that
((c,h_0)\leqslant (c,h_\sigma)), we obtain
(\sigma\gamma |h_\sigma|^2\leqslant \sigma(Bh_\sigma,h_\sigma)\leqslant \sigma(Bh_\sigma,h_0)\leqslant \sigma|B|\,|h_\sigma|\,|h_0|),

[
|h_\sigma|\leqslant \frac{|B|}{\gamma}|h_0|.
\tag{2}
]

We shall show that (h_\sigma) solves problem II. If (h\in Q'), then, by the construction of (h_\sigma), we have
((c+\sigma Bh_\sigma,h_\sigma)\leqslant (c+\sigma Bh_\sigma,h)). Let (h\in Q\setminus Q'). From (2) we obtain

[
\sum_j h_\sigma^{(j)} \leqslant \sqrt n\,|h_\sigma|
\leqslant \sqrt n\,\frac{|B|}{\gamma}\,|h_0|.
]

Consequently,

[
\Delta=\sum_i h^{(i)}-\sum_j h_\sigma^{(j)}>1.
]

Put (\lambda=1:\Delta) and (h'=\lambda h+(1-\lambda)h_\sigma). It is not hard to verify that (h'\in Q'), and consequently
((c+\sigma Bh_\sigma,h_\sigma)\leqslant (c+\sigma Bh_\sigma,h')). Substituting in place of (h') its expression, we obtain
((c+\sigma Bh_\sigma,h_\sigma)\leqslant (c+\sigma Bh_\sigma,h)). This proves the theorem.

Corollary. Since the vector (h_\sigma) solves the problem of minimizing the functional (L(h)=(c+\sigma Bh_\sigma,h)), it follows from ((1,2)), under the conditions of Theorem 1, that there exists a vector
(v_\sigma=(v_\sigma^{(1)},v_\sigma^{(2)},\ldots,v_\sigma^{(m)})) such that

[
\sum_{i=1}^{m} a_j^{(i)}v_\sigma^{(i)}
\leqslant c^{(j)}+\sigma(Bh_\sigma)^{(j)}, \quad
\text{if } h_\sigma^{(j)}=0;
]

[
\sum_{i=1}^{m} a_j^{(i)}v_\sigma^{(i)}
= c^{(j)}+\sigma(Bh_\sigma)^{(j)}, \quad
\text{if } h_\sigma^{(j)}>0;
\tag{3}
]

[
v_\sigma^{(i)}\geqslant 0,\quad i=1,2,\ldots,m;
]

[
v_\sigma^{(i)}=0,\quad \text{if } \sum_{j=1}^{n} a_j^{(i)}h_\sigma^{(j)}>p^{(i)}.
]

Theorem 2. The following assertions are valid:
a) the solution of problem II is unique;
b) for (\sigma>0), (h_\sigma) depends continuously on (\sigma);
c) if (\sigma<\sigma'), then ((c,h_\sigma)\leqslant(c,h_{\sigma'}));
d) there exists (\sigma_0>0) such that for (\sigma\leqslant\sigma_0) the vector (h_\sigma) is a solution of problem I.

We omit the proof.

Denote (a_j=(a_j^{(1)},a_j^{(2)},\ldots,a_j^{(m)})),
(p=(p^{(1)},p^{(2)},\ldots,p^{(m)})), and assume that all (a_j\ne0), (j=1,2,\ldots,n). As (B) take the triangular matrix in which zeros stand above the main diagonal, and the ((\mu,\nu))-th element ((\nu\leqslant\mu)) is equal to ((a_\mu,a_\nu)). For this matrix
((Bh,h)\geqslant \gamma|h|^2), where (2\gamma=\min|a_j|^2).

From the first two relations (3) we find

[
h_\sigma^{(j)}=
\begin{cases}
\dfrac{(v_{\sigma,j-1},a_j)-c^{(j)}}{\sigma|a_j|^2},
& \text{if } (v_{\sigma,j-1},a_j)\geqslant c^{(j)},\[1.2ex]
0,
& \text{if } (v_{\sigma,j-1},a_j)<c^{(j)},
\end{cases}
\tag{4}
]

where

[
v_{\sigma,j-1}=v_\sigma-\sigma\sum_{\nu=1}^{j-1}h_\sigma^{(\nu)}a_\nu.
]

Put (v_\sigma'=v_{\sigma,n}+\sigma p). From (1) and the last two relations (3) we conclude that

[
v_\sigma^{(i)}=
\begin{cases}
v_\sigma'^{(i)}, & \text{if } v_\sigma'^{(i)}\geqslant 0;\
0, & \text{if } v_\sigma'^{(i)}<0.
\end{cases}
]

We now construct the following iterative process:

[
\text{I.}\qquad v_{k,0}=v_k.
]

[
\text{II.}\qquad
h_k^{(j)}=
\begin{cases}
\dfrac{(v_{k,j-1},a_j)-c^{(j)}}{\sigma|a_j|^2},
& \text{if } (v_{k,j-1},a_j)\geqslant c^{(j)},\[1.2ex]
0,
& \text{if } (v_{k,j-1},a_j)<c^{(j)};
\end{cases}
\tag{4'}
]

[
v_{k,j}=v_{k,j-1}-\sigma h_k^{(j)}a_j.
]

III.

[
v'k = v + \sigma p.
]

IV.

[
v^{(i)}_{k+1} =
\begin{cases}
v'^{(i)}_k, & \text{if } v'^{(i)}_k \geq 0,\
0, & \text{if } v'^{(i)}_k < 0.
\end{cases}
]

V.

[
\left(v^{(1)}{k+1}, v^{(2)}}, \ldots, v^{(m){k+1}\right)=v, \qquad
\left(h^{(1)}_k, h^{(2)}_k, \ldots, h^{(n)}_k\right)=h_k.
]

Put

[
\Delta^{(j)}\sigma
=
\sigma |a_j|^2 h^{(j)}\sigma
-
(v_{\sigma,j-1},a_j)
+
c^{(j)};
\qquad
\Delta^{(j)}k
=
\sigma |a_j|^2 h^{(j)}_k
-
(v,a)
+
c^{(j)}.
]

From (4) and (4′) we conclude that
(\Delta^{(j)}\sigma \geq 0), (\Delta^{(j)}_k \geq 0),
(h^{(j)}\sigma \Delta^{(j)}_\sigma = h^{(j)}_k \Delta^{(j)}_k = 0). Consequently,

[
\left(h^{(j)}k-h^{(j)}\sigma\right)
\left(\Delta^{(j)}k-\Delta^{(j)}\sigma\right)
=
-\left(\Delta^{(j)}k h^{(j)}\sigma+\Delta^{(j)}_\sigma h^{(j)}_k\right)
\leq 0.
]

Next,

[
\begin{aligned}
|v_{k,j}-v_{\sigma,j}|^2
&=
|v_{k,j-1}-v_{\sigma,j-1}-\sigma h^{(j)}k a_j+\sigma h^{(j)}\sigma a_j|^2 \
&=
|v_{k,j-1}-v_{\sigma,j-1}|^2
-\sigma^2|a_j|^2\left(h^{(j)}k-h^{(j)}\sigma\right)^2 \
&\quad
+2\sigma\left(h^{(j)}k-h^{(j)}\sigma\right)
\left(\Delta^{(j)}k-\Delta^{(j)}\sigma\right).
\end{aligned}
]

Denoting (\tau=\sigma^2 \min |a_j|^2), we obtain

[
|v_{k,j}-v_{\sigma,j}|^2
\leq
|v_{k,j-1}-v_{\sigma,j-1}|^2
-
\tau\left(h^{(j)}k-h^{(j)}\sigma\right)^2.
]

Adding these inequalities for (j=1,2,\ldots,n), we obtain

[
|v_{k,n}-v_{\sigma,n}|^2
\leq
|v_{k,0}-v_{\sigma,0}|^2
-
\tau|h_k-h_\sigma|^2.
]

Next,

[
|v_{k+1}-v_\sigma|^2
\leq
|v'k-v'\sigma|^2
=
|v_{k,n}-v_{\sigma,n}|^2
\leq
|v_{k,0}-v_{\sigma,0}|^2
-
\tau|h_k-h_\sigma|^2
=
|v_k-v_\sigma|^2
-
\tau|h_k-h_\sigma|^2.
]

Applying the obtained inequality for smaller (k), we obtain

[
|v_{k+1}-v_\sigma|^2
\leq
|v_0-v_\sigma|^2
-
\tau \sum_{\nu=0}^{k}|h_\nu-h_\sigma|^2.
]

Since (\tau>0), it follows from this that (h_\nu \to h_\sigma) as (\nu\to\infty).

Thus, we have shown that the proposed iterative process gives, in the limit, the solution of problem II, which in turn approximates problem I, as follows from Theorems 1 and 2. It can be shown that the sequence ({v_k}) also converges to some vector (v_\sigma) satisfying a system of inequalities analogous to (3).

Leningrad Branch
of the V. A. Steklov Mathematical Institute
Academy of Sciences of the USSR

Received
19 XI 1960

CITED LITERATURE

1. L. V. Kantorovich, DAN, 115, No. 3, 441 (1957).
2. L. V. Kantorovich, The Economic Calculation of the Best Use of Resources, Publishing House of the Academy of Sciences of the USSR, 1959.
3. G. U. Kuhn, in the collection of translations Linear Inequalities, IL, 1959, pp. 363–371.