Abstract
Full Text
UDC 512.25/.26 + 519.3 : 330.115
MATHEMATICS
I. V. ROMANOVSKII
USE OF A LINEAR EXTENSION IN SOLVING INTEGER EXTREMAL PROBLEMS BY ENUMERATION METHODS
(Presented by Academician L. V. Kantorovich, July 28, 1967)
This note proposes a method for solving a certain well-known problem of integer programming, based on enumeration with evaluation of variants (the branch-and-bound method \((^1, ^2)\)) and using the dual variables of the corresponding linear-programming problem in order to improve estimates and reduce enumeration.
The problem considered by us is usually called the multidimensional knapsack problem (cf. \((^3)\)) and is formulated as follows. Given positive numbers \(b_i, c_j\) and nonnegative numbers \(a_{ij}\), \(i = 1, \ldots, m;\ j = 1, \ldots, n\). It is required to find numbers \(x_j \in B = \{0, 1\}\) maximizing the linear form
\[ \sum_j c_j x_j \tag{1} \]
under the conditions
\[ \sum_j a_{ij} x_j \leq b_i, \quad i = 1, \ldots, m. \tag{2} \]
There is an extensive literature \((^{4-7})\), etc., devoted to the solution of this problem and of the general problem of integer linear programming (to which it is equivalent) by enumeration methods. The proposed algorithm is close to the algorithm of E. Balas \((^8)\) in its method of using the variables of the dual problem, but the algorithm itself and—what substantially changes the laboriousness of the method—the way in which the variables are ordered are completely different.
A set of numbers \(x_j,\ j = 1, \ldots, n\), will be called a solution, and if it satisfies (2), a feasible solution. A set \(\mathbf{x} = \{x_j\},\ j = 1, \ldots, k;\ k \leq n\), will be called a partial solution (p.s.), and \(k\) its length \(l(\mathbf{x})\). To each p.s. \(\mathbf{x}\), in an obvious way, there corresponds a set of solutions \(X(\mathbf{x})\) and a set of feasible solutions \(X_f(\mathbf{x})\). We shall call a p.s. \(\mathbf{x}\) feasible (f.p.s.) if \(X_f(\mathbf{x})\) is nonempty. Since a p.s. can always be extended by zero \(x_j\)’s, testing the feasibility of a p.s. is not laborious. A p.s. \(\mathbf{x}' = (x'_1, \ldots, x'_{k_1})\) extends a p.s. \(\mathbf{x}^2 = (x^2_1, \ldots, x^2_{k_2})\) if \(k_1 > k_2\) and \(x'_i = x^2_i\) for \(i \leq k_2\). Introduce on the set of p.s. \(X\) the following lexicographic ordering \(<\): \(\mathbf{x}' < \mathbf{x}^2\) if either \(\mathbf{x}^2\) extends \(\mathbf{x}'\), or there exists an \(r \leq \min l(\mathbf{x}^i)\) such that \(x'_r > x^2_r\) and \(x'_i = x^2_i\) for \(i < r\).
The procedure of one-sided enumeration includes the sequence of examining solutions and evaluating them. We shall enumerate feasible partial solutions in accordance with the introduced ordering, assigning to each f.p.s. \(\mathbf{x}\) a quantity \(e(\mathbf{x})\) estimating from above the value of the functional (1) on the set \(X_f(\mathbf{x})\).
The state of the enumeration process must include all the necessary information about the prior history of enumeration in such a form as to facilitate its modification at subsequent steps of the process. We shall call this information a record. In various problems and in various algorithms
the record will be different, and the efficiency of the algorithm depends in large measure on the structure of the record. Using the terminology of the theory of Markov processes for the solution of (9), one may call the record a Markovian transitive sufficient statistic of the enumeration process.
Abstracting from computational conveniences, define a record as a set \(\langle \mathbf{x}, e(\mathbf{x}), \mathbf{y}, E\rangle\), consisting of a partial solution \(\mathbf{x}\), its estimate \(e(\mathbf{x})\), the best of the admissible solutions \(\mathbf{y}\) encountered earlier, and the corresponding value of the functional \(E\).
The algorithm consists in specifying rules for transition from one record to another. In our case they are as follows:
-
If \(e(\mathbf{x}) \leq E\), then pass to the partial solution \(T(\mathbf{x})\), following \(\mathbf{x}\) and nearest to it among all those of smaller length. The next record will be
\[ \langle T(\mathbf{x}), e(T(\mathbf{x})), \mathbf{y}, E\rangle . \] -
If \(e(\mathbf{x}) > E\) and \(l(\mathbf{x}) < n\), check whether the partial solution \(\mathbf{x}' = (\mathbf{x},1)\) will be admissible. In this case \(e(\mathbf{x}') = e(\mathbf{x})\) (see below). The new record will be
\[ \langle \mathbf{x}', e(\mathbf{x}), \mathbf{y}, E\rangle . \]
If, however, \(\mathbf{x}'\) is not a partial solution, then the next record will be
\[ \langle \mathbf{x}'', e(\mathbf{x}''), \mathbf{y}, E\rangle, \]
where \(\mathbf{x}'' = (\mathbf{x},0)\), and the estimate \(e(\mathbf{x}'')\) is computed anew. -
If \(e(\mathbf{x}) > E\) and \(l(\mathbf{x}) = n\), then the next record will be
\[ \langle \mathbf{x}, e(\mathbf{x}), \mathbf{x}, e(\mathbf{x})\rangle . \]
The process stops when we arrive at a partial solution \(\mathbf{x}\) for which \(T(\mathbf{x})\) is not defined, i.e., at the partial solution consisting of zero elements.
Let us now pass to the description of the estimate \(e(\mathbf{x})\) and of the numbering of the variables \(x_j\), on which the efficiency of the algorithm depends extremely strongly. Consider the linear-programming problem (1)—(2) with the constraints
\[
0 \leq x_j \leq 1,\quad j=1,\ldots,n. \tag{3}
\]
Let \(\mathbf{x}^0\) be some optimal basic solution of this problem, and let \(\alpha_i\) be the optimal values of the dual variables corresponding to conditions (2). Put
\[
\gamma_j=\sum_i a_{ij}\alpha_i,\qquad \beta=\sum_i b_i\alpha_i .
\]
Denote by \(N_b\) the set of basic variables. For \(j \in N_b\) we have \(\gamma_j=c_j\). The nonbasic variables are divided into the group \(N_0\) of variables equal to 0 and \(N_1\) of variables equal to 1. For \(j \in N_0\), \(\gamma_j \geq c_j\); for \(j \in N_1\), \(\gamma_j \leq c_j\). Note that the solution
\[
x_j=
\begin{cases}
1, & j\in N_1,\\
0, & j\notin N_1,
\end{cases}
\]
is an admissible solution of our original integer problem and gives a good initial approximation. We use this solution and
\[
E_0=\sum_{N_1} c_j
\]
in the initial record of the enumeration process.
The numbering of the variables introduced by us will be such that first come the variables from \(N_1\), then those from \(N_b\), and only then those from \(N_0\). Let \(l(\mathbf{x})=k\). Put
\[
\beta(\mathbf{x})=\beta-\sum_{j=1}^{k}\gamma_j x_j.
\]
Denote by \(s(\mathbf{x})\) the number \(s\) for which
\[
\sum_{j=k+1}^{s}\gamma_j \leq \beta(\mathbf{x}) < \sum_{j=k+1}^{s+1}\gamma_j . \tag{4}
\]
Put
\[
e(\mathbf{x})=\sum_{j=1}^{k} c_j x_j+\sum_{j=k+1}^{s(\mathbf{x})} c_j,\qquad
t(\mathbf{x})=\beta(\mathbf{x})-\sum_{j=k+1}^{s(\mathbf{x})}\gamma_j . \tag{5}
\]
Lemma 1. If the variables from \(N_0\) are arranged in decreasing order of the ratio \(c_j/\gamma_j\), then
\[ e(\mathbf{x})=\varepsilon(\mathbf{x})+c_{s(\mathbf{x})}\cdot t(\mathbf{x})/\gamma_{s(\mathbf{x})} \tag{6} \]
is an upper bound for the functional (1) on the set \(X_f(\mathbf{x})\).
Ordering is required only for \(N_0\), since on \(N_b\) the ratio \(N^b\) is constant, and, as follows from the definition of the set \(N_1\), one always has \(s(\mathbf{x})\ge |N_1|\), so that the inequality \(c_j/\gamma_j\ge 1\) is sufficient.
Without using any additional information about the data of the problem, it is impossible to consider the question of reducing the number of steps in the enumeration, and it is natural to try to reduce the estimate for the number of steps. Consider the set of feasible solutions of the problem of finding a vector \(x_j\in B\) satisfying the condition
\[ \sum_{j=1}^{n}\lambda_j x_j \le \Delta,\qquad \text{where } \lambda_j\ge 0 . \tag{7} \]
The number of feasible solutions is a function of the permutation \(\pi\), which determines the numbering of the variables. Denote this function by \(M(\pi)\).
Lemma 2. The minimum of \(M(\pi)\) is attained for the permutation that orders \(\lambda_j\) in decreasing order.
Put \(\delta_j=\gamma_j-c_j,\ \Delta=\sum_{N_b}\gamma_j x_j\). Every feasible solution \(\bar{\mathbf{x}}\) in problem (1)—(2) satisfying the condition \(e(\bar{\mathbf{x}})>E_0\) also satisfies the condition
\[ \sum_j \delta_j \bar{x}_j \le \Delta+\sum_{N_1}\delta_j . \tag{8} \]
Indeed, consider the vector
\[ \mathbf{x}=(\bar{x}_1,\ldots,\bar{x}_k,1,\ldots,1,x_{s(\bar{\mathbf{x}})+1},0,\ldots,0), \]
where
\[ x_{s(\bar{\mathbf{x}})+1}=t(\bar{\mathbf{x}})/\gamma_{s(\bar{\mathbf{x}})+1}. \]
Taking into account that
\[ \sum_j \gamma_j x_j^*=\beta,\qquad \sum_{j=1}^{k}\gamma_j \bar{x}_j+\sum_{k+1}^{s(\bar{\mathbf{x}})}\gamma_j+t(\bar{\mathbf{x}})=\beta, \]
and the equalities
\[ \sum_{j=1}^{n}\gamma_j x_j^* = \sum_{j=1}^{n}c_jx_j^* + \sum_{j=1}^{n}\delta_j x_j^* = E_0+\Delta+\sum_{N_1}\delta_j, \]
\[ \sum_{j=1}^{n}\gamma_j\bar{x}_j = e(\bar{\mathbf{x}})+ \sum_{j=1}^{n}\delta_j\bar{x}_j, \]
from the condition \(e(\mathbf{x})>E_0\) we obtain (8). Now associate with each feasible solution \(\mathbf{x}\) of the original problem a feasible solution \(\mathbf{x}'\) that coincides with \(\mathbf{x}\) on \(N_b\cup N_0\) and extends \(x_j\) to 1 for \(j\in N_1\). Put \(\lambda_j=|\delta_j|\). For any feasible solution of (7) we obtain, taking into account that \(x_j\in B\) for \(j\in N_1\),
\[ \sum_{N_1}\delta_j x_j=\sum_{N_1}\delta_j+\sum_{N_1}\lambda_j x_j' \]
and, consequently,
\[ \sum_{j=1}^{n}\lambda_j x_j'\le \Delta . \]
Thus, if one proceeds from the desire to reduce the estimate of the number of iterations, the variables in the set \(N_1\) must be arranged in the order of ...
in decreasing order of \(\lambda_j\). It is not expedient to order the set \(N_0\) by \(\lambda_j\), since here a small increase in the number of iterations is compensated for by the simplicity of constructing the estimate \(e(x)\). The set \(N_b\) and the variables in \(N_1\) with identical values of \(\lambda_j\), by virtue of the same Lemma 2 applied to the condition
\[ \sum_j \gamma_j x_j \leqslant \beta, \]
should expediently be arranged in decreasing order of \(\gamma_j\).
Remark. From the computational standpoint, the following notation proves more convenient:
\(\langle s(x), k(x), N_x, l(x), \varepsilon(x), t(x), y, E\rangle\), where
\[ k(x)=\sum_{j=1}^{l(x)} x_j, \]
and \(N_x\) is the \(k(x)\)-dimensional vector of the indices of the \(x_j\) equal to 1.
Although, for a multidimensional knapsack problem, any integer problem with linear constraints can be reduced to a rank problem, in problems of special form it is expedient to use one’s own variants of records and transitions. Nevertheless, ordering the variables on the basis of dual estimates significantly reduces the amount of enumeration here as well. As an example we mention the integer placement problem without taking transportation costs into account and the well-known traveling salesman problem.
Leningrad State University
named after A. A. Zhdanov
Received
25 VII 1967
REFERENCES
- A. L. Brudno, Problems of Cybernetics, vol. 10, 141 (1963).
- J. D. C. Little et al., Operations Research, 11, 972 (1965).
- A. A. Korbut, Economic-Mathematical Methods, vol. 2, 1965, p. 141.
- E. Balas, Operations Research, 13, 517 (1965).
- F. Glover, ibid., 13, 879 (1965).
- R. Faure, J. Malgrange, Ekon.-mat. obzor, 2, 11 (1966).
- P. L. Ivanescu, C. R., 263, A (1966).
- E. Balas, ibid., 262, A, 766 (1966).
- V. S. Mikhalevich, Cybernetics, No. 1, 45 (1965).