Full Text
UDC 512.25/26+519.3
CYBERNETICS
AND CONTROL THEORY
E. G. GOL’SHTEIN
ON A GENERAL APPROACH TO THE SOLUTION OF LINEAR PROGRAMMING PROBLEMS OF BLOCK STRUCTURE
(Presented by Academician L. V. Kantorovich, May 29, 1965)
1. Consider the linear programming problem
\[ (C,X)\to \max, \tag{1} \]
\[ AX=B, \tag{2} \]
\[ A^{(1)}X=B^{(1)}, \tag{3} \]
\[ X\geq 0. \tag{4} \]
Here \(A\) and \(A^{(1)}\) are matrices of dimensions \(m\times n\) and \(m_1\times n\), respectively; \(B\), \(B^{(1)}\), \(C\) are vectors with \(m\), \(m_1\), \(n\) components, respectively. It is assumed that the matrix \(A^{(1)}\) has a special structure (for example, block-diagonal or transportation), which substantially facilitates the analysis of the linear problem with conditions (3), (4). By a block programming method is meant an algorithm for solving problem (1)—(4) based on the analysis of a series of problems with conditions (3), (4). One such algorithm is the well-known decomposition method [1]. In the present note we propose a sufficiently general approach to the formation of block programming methods, having a natural geometric meaning and connected with the search for a vector of estimates of the conditions (2).
2. Define the function
\[ f(\Lambda)=\sup_{X\in G}(C_\Lambda,X)+(B,\Lambda), \tag{5} \]
where \(C_\Lambda=C-\Lambda A\), \(\Lambda=(\lambda_1,\lambda_2,\ldots,\lambda_m)\), and \(G\) is the set of points \(X\) satisfying conditions (3), (4). We shall call the points \(X\in G\) and the unit direction vectors of rays contained in \(G\) the elements of \(G\). An element coinciding with a vertex of \(G\) or going along an unbounded edge of \(G\) will be called an extreme one. To each element \(X\) of the set \(G\) there corresponds a hyperplane and a half-space of the space of points \(\bar{\Lambda}=(\lambda,\Lambda)\):
\[ \delta\lambda+(AX-\delta B,\Lambda)=(C,X);\qquad \delta\lambda+(AX-\delta B,\Lambda)\geq (C,X), \]
where \(\delta=1\) or \(0\) depending on whether \(X\) is a point of \(G\) or not.
Let \(X_i,\ i=1,2,\ldots,N\), be the set of all extreme elements of \(G\), the points having the first \(N_1\) numbers, and the direction vectors the subsequent \(N-N_1\) numbers. Introduce the polyhedral set \(\bar R\) (\(R\)) of points \(\bar{\Lambda}=(\lambda,\Lambda)\) (points \(\Lambda\)), coinciding with the intersection of the half-spaces corresponding to the extreme elements \(X_i\) for \(i=1,2,\ldots,N\) (for \(i=N_1+1,\ldots,N\)). The set \(R\) consists of the points \(\Lambda\) at which \(f(\Lambda)<\infty\); the set \(\bar R\) consists of the points \(\bar{\Lambda}=(\lambda,\Lambda)\) for which \(\lambda\geq f(\Lambda)\), \(\Lambda\in R\). Thus, \(f(\Lambda)\) is a convex downward piecewise-linear function defined on the set \(R\), which in what follows is assumed nonempty. On each of the intervals of linearity of the function \(f(\Lambda)\), the surface \(\lambda=f(\Lambda)\) coincides with one of the hyperplanes corresponding to the vertices of \(G\).
Let \(f(\Lambda)<\infty\), and let \(A_\Lambda\) be one of the optimal bases of the problem
\[ (C_\Lambda,X)\to \max,\quad X\in G. \tag{6} \]
Denote by \(\Delta_j(\Lambda)\) the estimate of the \(j\)-th vector of the conditions of problem (6),
associated with the basis \(A_\Lambda\). Introduce the set \(G(\Lambda,\varepsilon)\), \(\varepsilon \geqslant 0\), of points \(X \in G\) satisfying the requirement \(x_j = 0\) if \(\Delta_j(\Lambda) > \varepsilon\).
Lemma. For every \(\varepsilon \geqslant 0\) there exists an \(\alpha(\varepsilon) \geqslant 0\), independent of \(\Lambda\) \((\alpha(0)=0,\ \alpha(\varepsilon)>0\) for \(\varepsilon>0)\), such that if the hyperplane corresponding to an element \(X\) of the set \(G\) is at a distance from the point \(\bar{\Lambda}=(f(\Lambda),\Lambda)\) no greater than \(\alpha(\varepsilon)\), then \(X\) is an element of the set \(G(\Lambda,\varepsilon)\). In particular, any hyperplane supporting \(\bar{R}\) at the point \(\bar{\Lambda}=(f(\Lambda),\Lambda)\) and not parallel to the axis \(\lambda\) corresponds to some point of the set \(G(\Lambda)=G(\Lambda,0)\)—the set of solutions (optimal plans) of problem (6).
The analysis of problem (1)—(4) reduces to minimizing the function \(f(\Lambda)\) on the set \(R\), or, what is the same, to finding the lower (in the sense of the \(\lambda\)-axis) point of the set \(\bar{R}\).
If \(\inf_{\Lambda \in R} f(\Lambda)=-\infty\), then the constraints of problem (1)—(4) are inconsistent. If, however, \(\inf_{\Lambda \in R} f(\Lambda)=f(\Lambda^*)>-\infty\), then the solution of problem (1)—(4) is the point \(X^*\in G(\Lambda^*)\), to which there corresponds a hyperplane supporting \(\bar{R}\) at the point \(\bar{\Lambda}^*\) and orthogonal to the vector \((1,0,\ldots,0)\). Thus, the criterion of optimality of a point \(X\in G\) consists in satisfying the conditions
\[ X\in G(\Lambda)\quad \text{for some } \Lambda\in R, \tag{7} \]
\[ AX=B. \tag{8} \]
3. In solving problem (1)—(4) by the decomposition method \((^1)\), condition (8) is always fulfilled, while condition (7) is checked at each iteration. The geometric meaning of the decomposition method is as follows. At each iteration the initial point is \(\bar{\Lambda}=(\lambda,\Lambda)\)—the point of intersection of \(m+1\) independent hyperplanes corresponding to certain extreme elements of \(G\) (these elements participate in the formation of the current plan), and the hyperplane \(\lambda=\text{const}\) is supporting for the common part of the corresponding half-spaces (condition (8)). The point \(\bar{\Lambda}\) lies “below” the set \(\bar{R}\), i.e. \(\lambda \leqslant f(\Lambda)\). If \(\lambda=f(\Lambda)\), then the solution process is finished. Otherwise, one of the hyperplanes containing \(\bar{\Lambda}\) is replaced by the hyperplane corresponding to some element of the set \(G(\Lambda)\). As a result we obtain a new point \(\bar{\Lambda}'\), with \(\lambda'\geqslant \lambda\). The group of block-programming methods proposed below is directly connected with the minimization of the function \(f(\Lambda)\). The essence of one of these methods, consisting in moving along the vertices of the set \(\bar{R}\), is as follows. At the beginning of an iteration there is a vertex \(\bar{\Lambda}=(f(\Lambda),\Lambda)\) of the set \(\bar{R}\)—the common part of \(m+1\) independent hyperplanes corresponding to elements of \(G(\Lambda)\). If the hyperplane \(\lambda=\text{const}\) is supporting for the common part of the corresponding half-spaces, then it is supporting also for \(\bar{R}\) at the point \(\bar{\Lambda}\); the problem is solved. Otherwise, an edge of the set \(\bar{R}\) issuing from the point \(\bar{\Lambda}\) is chosen, along which \(f(\Lambda)\) decreases (the set \(\bar{R}\) is assumed to be nondegenerate).
Let \(\bar{l}=(l_0,l)\) be the direction vector of this edge. Moving along \(\bar{l}\), we arrive at a new vertex \(\bar{\Lambda}'\) of the set \(\bar{R}\), or ascertain that the function \(f(\Lambda)\) is unbounded below. In the latter case problem (1)—(4) is infeasible. Motion in the chosen direction is carried out by the method of parametric programming (below we shall discuss this stage in somewhat more detail). The method considered is close to the algorithm obtained from other considerations in \((^2)\), but differs from the latter in the manner of moving along the chosen edge.
4. The general scheme of the process of minimizing the function \(f(\Lambda)\) is as follows. Let \(\Lambda'\in R\), i.e. \(f(\Lambda')<\infty\). We shall call a direction \(l'\) suitable for the point \(\Lambda'\) if \(f(\Lambda'+\theta l')<f(\Lambda')\) for some \(\theta>0\). Suppose that we know a method \(T\) that makes it possible to choose a suitable-
direction, or establish that the set of such directions is empty. Using method \(T\), we choose a suitable direction \(l'\), or convince ourselves that no such directions exist. In the latter case \(\Lambda'\) is the desired point. If a suitable direction \(l'\) has been found, we pass from the point \(\Lambda'\) to the point \(\Lambda''=\Lambda'+\theta'l'\). The choice of the number \(\theta'\) (the step length) is made by means of some method \(E\). Thus, the process of minimizing the function \(f(\Lambda)\) consists of a series of iterations, each of which consists of two stages. The first stage—the choice of a suitable direction—is determined by method \(T\); the second—the computation of the step length along the chosen direction—is carried out by means of method \(E\). The methods \(T\) and \(E\) may vary from iteration to iteration.
- Let us note several methods for choosing a direction suitable for the point \(\Lambda'\). Put \(\psi_1(\Lambda,X)=(C_\Lambda,X)+(B,\Lambda)\) and introduce the function
\[ \varphi_1(l;\Lambda',\varepsilon)=\sup_{X\in G(\Lambda',\varepsilon)} \psi_1(\Lambda'+l,X),\qquad \varepsilon\geqslant 0. \]
In accordance with the lemma, \(f(\Lambda'+l)=\varphi_1(l;\Lambda',\varepsilon)\) for \(|l|\leqslant \gamma(\Lambda')\), \(\gamma(\Lambda')>0\), moreover \(\gamma(\Lambda')\geqslant \gamma>0\), if \(\varepsilon>0\).
Denote by \(M\) a polyhedral set of points \(l\), specified by the system of conditions \((D_i,l)\leqslant d_i\), \(i=1,\ldots,r\), and satisfying two requirements: a) \(0\) is an interior point of \(M\); b) \(\inf_{l\in M}\varphi_1(l;\Lambda',\varepsilon)>-\infty\).
Let \(l'\) be a solution of the problem \(\Gamma_1(\Lambda',\varepsilon,M)\) of minimizing the function \(\varphi(l;\Lambda',\varepsilon)\) on the set \(M\). The dual problem \(\widetilde{\Gamma}_1(\Lambda',\varepsilon,M)\) consists in maximizing the function \(\varphi_1(X;\Lambda',M)=\inf_{l\in M}\psi_1(\Lambda'+l,X)\) on the set \(G(\Lambda',\varepsilon)\). An equivalent formulation of the problem \(\Gamma_1(\Lambda',\varepsilon,M)\) has the form
\[ (C_{\Lambda'},X)-\sum_{i=1}^{r}d_i\mu_i\to\max \quad\text{under the conditions}\quad AX-\sum_{i=1}^{r}D_i\mu_i=B,\quad A^{(1)}X=B^{(1)}, \]
\(X\geqslant 0\) and the additional requirement \(x_j=0\), if \(\Delta_j(\Lambda')>\varepsilon\).
If \(\varphi_1(l';\Lambda',\varepsilon)<\varphi_1(0;\Lambda',\varepsilon)\), then \(l'\) is the desired suitable direction. Otherwise the set of directions suitable for \(\Lambda'\) is empty; \(f(\Lambda')=\min_{\Lambda\in R}f(\Lambda)\); the solution \(X'\) of the problem \(\widetilde{\Gamma}_1(\Lambda',\varepsilon,M)\) is a solution of problem (1)—(4).
To solve the mutually conjugate problems \(\Gamma_1(\Lambda',\varepsilon,M)\) and \(\widetilde{\Gamma}_1(\Lambda',\varepsilon,M)\), one may use a number of procedures. The first of them consists in direct minimization of the function \(\varphi_1(l;\Lambda',\varepsilon)\) with respect to \(l\in M\), which is carried out by moving over the vertices of the polyhedral set \(l_0\supset \varphi_1(l;\Lambda',\varepsilon)\), \(l\in M\) (see item 3). The second is connected with the direct maximization of \(\varphi_1(X;\Lambda',M)\) with respect to \(X\in G(\Lambda',\varepsilon)\) (the decomposition method (1) is applied to the problem \(\widetilde{\Gamma}_1(\Lambda',\varepsilon,M)\)). If the first two methods are finite algorithms, then the third, based on the method of a fictitious game (3), leads, generally speaking, to an infinite procedure. If \(l^{(k)}\) and \(X_k\) are the result of the \(k\)-th step of this method, then
\[ l^{(k+1)}=\frac{1}{k+1}\widetilde{l}^{(k)}+\frac{k}{k+1}l^{(k)}; \qquad X^{(k+1)}=\frac{1}{k+1}\widetilde{X}_k+\frac{k}{k+1}X_k; \]
\[ \psi_1(\Lambda'+\widetilde{l}^{(k)},X_k)=\min_{l\in M}\psi_1(\Lambda'+l,X_k); \]
\[ \psi_1(\Lambda'+l^{(k+1)},\widetilde{X}_k) = \max_{X\in G(\Lambda',\varepsilon)} \psi_1(\Lambda'+l^{(k+1)},X). \]
Convergence of the method is guaranteed when \(M\) and \(G(\Lambda',\varepsilon)\) are bounded. Put \(\psi_2(l,X)=\psi_1(\Lambda'+l,X)-\psi_1(\Lambda',X)=(B-AX,l)\). Replacing, in the preceding considerations, \(\psi_1(\Lambda'+l,X)\) by \(\psi_2(l,X)\), we arrive at a pair of mutually conjugate problems \(\Gamma_2(\Lambda',\varepsilon,M)\) and \(\widetilde{\Gamma}_2(\Lambda',\varepsilon,M)\). For \(\varepsilon=0\) these problems are equivalent to the problems \(\Gamma_1(\Lambda',\varepsilon,M)\) and \(\widetilde{\Gamma}_1(\Lambda',\varepsilon,M)\); for \(\varepsilon>0\) the equivalence is violated. If \(l'\) and \(X'\) are solutions of the problems \(\Gamma_2(\Lambda',\varepsilon,M)\) and
\(\Gamma_2(\Lambda', \varepsilon, M)\), respectively, then \(l'\) is a suitable direction for \(\psi_2(l, X') < 0\); \(X'\) is a plan of problem (1)—(4) for \(\psi_2(l', X') = 0\). In the latter case \((C, X^* - X') \leq \gamma \varepsilon\), where \(X^*\) is a solution of problem (1)—(4), \(\gamma = \mathrm{const}\). For the analysis of the problems \(\Gamma_2(\Lambda', \varepsilon, M)\) and \(\widetilde{\Gamma}_2(\Lambda', \varepsilon, M)\), one of the three methods described above is used.
-
The length \(\theta'\) of the step along the selected suitable direction \(l'\) is determined by means of the method of parametric programming (see, for example, (4), Ch. 8), applied to the problem of maximizing \((C_{\Lambda'} - \theta l' A, X)\) for \(X \in G\), \(\theta \geq 0\). As \(\theta'\) one takes either the smallest positive critical value of the parameter (method \(E_1\)), or the positive critical value of the parameter beginning with which the function \(f(\Lambda' + \theta l')\) does not decrease (method \(E_2\)). If it is impossible to choose a finite \(\theta'\), the unsolvability of problem (1)—(4) is recorded.
-
The decomposition method is a block analogue of the method of plan improvement. The method for solving problem (1)—(4) associated with movement along the vertices of the set \(\overline{R}\) is a block analogue of the method of refinement of estimates. If the selection of a suitable direction is carried out by means of the decomposition method applied to the problem \(\widetilde{\Gamma}_1(\Lambda', 0, M)\), where \(M\) is determined by the conditions \(l_i \geq -1,\ i = 1, 2, \ldots, m\), and the step length \(\theta'\) is computed using method \(E_1\), then we arrive at a block analogue of the method of reducing discrepancies. On the basis of the general approach considered here one can also formulate a block analogue of the method of two-sided estimates. The choice of a suitable direction in this method is connected with finding a “lower” point of the polyhedral set \(\overline{R}(\Lambda', \varepsilon, X')\), which is formed by half-spaces corresponding to the extreme elements \(G(\Lambda', \varepsilon)\) and to the \(m + 1\) extreme elements of \(G\) participating in the formation of the plan \(X'\). For choosing the step length \(\theta'\), the methods \(\widetilde{E}_1\) and \(E_2\) are used here.
Thus, the general approach considered here, in particular, makes it possible to obtain block analogues of the four basic finite methods of linear programming (see (5), Ch. 12). However, combined methods of block programming appear to be the most effective. At the beginning of the process of solving problem (1)—(4) it is expedient to choose a suitable direction on the basis of the fictitious-play method and to move in this direction using method \(E_2\). As one approaches the minimum of the function \(f(\Lambda)\), the rate of convergence of the game-theoretic method for selecting a suitable direction usually decreases sharply. Having detected this, one should switch to one of the finite methods for selecting a suitable direction and at the same time replace method \(E_2\) by \(E_1\). We formulate two assertions connected with the question of convergence of the proposed methods.
Theorem 1. If the suitable direction is selected by one of the finite methods, and the step length is computed using method \(E_1\), then the block programming method is finite.
Theorem 2. Suppose that, for computing the step length, method \(E_2\) is used, and that the number \(\varepsilon\) participating in the formation of the problems \(\Gamma_i(\Lambda', \varepsilon, M)\) and \(\widetilde{\Gamma}_i(\Lambda', \varepsilon, M)\) \((i = 1\ \text{or}\ 2)\) is positive. Under the condition that the function \(f(\Lambda)\) is bounded below, the corresponding block algorithm is finite. If \(f(\Lambda)\) is not bounded below, then either this is established at one of the iterations of the algorithm, or a sequence \(\{\Lambda^{(k)}\}\) is constructed for which
\[ \lim_{k \to \infty} f(\Lambda^{(k)}) = -\infty . \]
Central Economic-Mathematical Institute
Academy of Sciences of the USSR
Received
17 V 1965
CITED LITERATURE
- G. B. Dantzig, P. Wolfe, Operation Res., 8, 1 (1960).
- J. M. Abadie, A. C. Williams, Dual and Parametric Methods in Decomposition. N. Y., 1963.
- G. W. Brown, Iterative Solution of Game by Fictitions Play, Activity Analysis of Production and Allocation, N. Y., 1951.
- S. Gass, Linear Programming, Moscow, 1961.
- D. B. Yudin, E. G. Golshtein, Problems and Methods of Linear Programming, Moscow, 1964.