Mathematics

E. G. GOL′SHTEIN

ON A CLASS OF NONLINEAR EXTREMAL PROBLEMS

(Presented by Academician A. I. Berg on 2 III 1960)

In this note an extremal problem is studied which consists in maximizing the function

[
F(x_1, x_2, \ldots, x_n)=\sum_{j=1}^{n} f_j(x_j),
\tag{1}
]

whose variables satisfy linear conditions of the form

[
\sum_{j=1}^{n} a_{ij}x_j=b_i,\qquad i=1,2,\ldots,m;
\tag{2}
]

[
\alpha_j \leq x_j \leq \beta_j,\qquad j=1,2,\ldots,n.
\tag{3}
]

It is assumed that (f_j(x)) ((j=1,2,\ldots,n)) are continuous piecewise-smooth concave functions. Some of the numbers (\alpha_j(\beta_j)) may coincide with (-\infty) ((\infty)).

Problem (1)—(3), which we shall briefly call problem I, is a substantial extension of the general problem of linear programming, which consists, as is known, in finding the maximum (or minimum) of a linear function whose variables are subject to linear constraints of the type (2), (3).

1. We shall call a vector (X=(x_1,x_2,\ldots,x_n)) satisfying conditions (2), (3) a plan of problem I. A plan that makes the function (1) maximal will be called an optimal plan of problem I, or its solution.

The basis of the numerical analysis of an extremal problem is usually a criterion that makes it possible to test its plans for optimality. Below we formulate a necessary and sufficient condition for the optimality of a plan of problem I.

Optimality criterion. In order that a plan (X=(x_1,x_2,\ldots,x_n)) be a solution of problem I, it is necessary and sufficient that there exist a vector (\Lambda=(\lambda_1,\lambda_2,\ldots,\lambda_m)) satisfying the conditions:

a) [
f'{j+}(x_j)\leq (A_j,\Lambda)\leq f'(x_j)
\quad \text{when } \alpha_j<x_j<\beta_j;
]

b) [
f'_{j+}(x_j)\leq (A_j,\Lambda)
\quad \text{when } x_j=\alpha_j;
]

c) [
f'_{j-}(x_j)\geq (A_j,\Lambda)
\quad \text{when } x_j=\beta_j.
]

Here (f'{j+}(x)), (f')).}(x)) are respectively the right and left derivatives of the function (f_j(x)); (A_j=(a_{1j},a_{2j},\ldots,a_{mj

When (f_j(x)=c_jx) for (j=1,2,\ldots,n), problem I becomes the general problem of linear programming. In this case the criterion formulated becomes the known conditions for optimality of a plan of a linear programming problem, established by L. V. Kantorovich ((^{1,2})). Thus, the components of the vector (\Lambda) are an analogue of L. V. Kantorovich’s resolving multipliers.

1. Suppose now that, for any (j), the interval ((\alpha_j,\beta_j)) can be divided into (l_j) intervals
   ((d_{0j},d_{1j}); (d_{1j},d_{2j}); \ldots; (d_{l_j-1,j},d_{l_j j})), on each of which (f_j(x)) is a linear function. Thus, for
   (d_{t-1,j}\leq x\leq d_{tj}) ((t=1,2,\ldots,l_j)),
   [
   f_j(x)=c_{tj}(x-d_{t-1,j})+f_j(d_{t-1,j}).
   ]

Here (d_{0j}=\alpha_j;\ d_{l_j j}=\beta_j).

The quantities (d_{0j}, d_{1j}, \ldots, d_{l_j j}) will be called the critical values of the argument of the function (f_j(x)).

Problem I, whose functions (f_j(x)) satisfy the conditions just indicated, will be called Problem II. Below we present a general scheme of a method that makes it possible to determine optimal plans for problems of type II. The proposed method is a generalization of one of the basic devices of linear programming—the method of successive improvement of the plan.

The method of successive improvement of the plan can be realized in two computationally different forms. The first form is due to Dantzig ((^3)); in the foreign literature it is known as the simplex method. The second form of the plan-improvement method was first used in ((^4)); this form relies substantially on the already mentioned optimality criterion of L. V. Kantorovich. In the present section the second form of the method of successive improvement of the plan is carried over to nonlinear extremal problems of type II. The basis of the method is the optimality criterion for a plan of Problem I formulated in the preceding section.

We first introduce several definitions. Let (X=(x_1,x_2,\ldots,\ldots,x_n)) be a plan of Problem II. By (E_X) we denote the set of indices (j) for which (x_j) does not coincide with any of its critical values: (x_j\ne d_{tj}), (t=0,1,\ldots,l_j). A plan (X) will be called basic if the vectors
(A_j=(a_{1j},a_{2j},\ldots,a_{mj})), (j\in E_X), are linearly independent.

We shall assume that the rank of the matrix of the system of equations (2) is equal to (m). Naturally, (n>m). A basic plan (X) will be called nondegenerate if the set of indices (E_X) consists of (m) elements. Problem II, each basic plan of which satisfies the nondegeneracy condition, is called a nondegenerate problem.

At first we shall assume Problem II to be nondegenerate. Let (X=(x_1,x_2,\ldots,x_n)) be some basic plan of Problem II (it can be computed by means of one of the known devices of linear programming). The method of finding a solution of Problem II consists of a finite number of similar iterations. Each iteration can be divided into two stages. At the first stage the plan obtained as a result of the preceding iteration is tested for optimality. If it turns out not to be optimal, then at the second stage of the iteration an improved plan is constructed. We shall describe the first iteration, starting from the initial plan (X=(x_1,x_2,\ldots,x_n)).

Stage 1. From the system of linear equations
[
\sum_{i=1}^{m} a_{ij}\lambda_i=f'j(x_j),\qquad j\in E_X,
\tag{4}
]
we determine the vector (\Lambda=(\lambda_1,\lambda_2,\ldots,\lambda_m)). Let (\bar j\notin E_X). Then (x)}=d_{t_{\bar j}\bar j
((0\leq t_{\bar j}\leq l_{\bar j})). Put (c_{0j}=\infty;\ c_{l_j+1}=-\infty). If for any (\bar j\notin E_X) the inequalities
[
c_{t_{\bar j}+1,\bar j}\leq \sum_{i=1}^{m}\lambda_i a_{i\bar j}\leq c_{t_{\bar j}\bar j},
\tag{5}
]
hold,

then, in accordance with the optimality criterion, the plan (X) is a solution of problem II. If, however, the vector (\Lambda) does not satisfy all the inequalities of system (5), then one passes to stage 2, at which an improvement of the plan (X) is carried out.

Stage 2. Choose an index (k) for which:

a) either
[
\sum_{i=1}^{m}\lambda_i a_{ik}< C_{t_{k+1},k};
]

b) or
[
\sum_{i=1}^{m}\lambda_i a_{ik}> C_{t_k,k}.
]

Let
[
A_k=\sum_{i\in E_X} z_{ik}A_i .
\tag{6}
]

The transformation of the plan (X) leading to a new plan (X') is determined by which of the cases indicated occurs.

a) The components of the new plan (X') are computed by the formulas:
[
\begin{aligned}
x'j&=x_j-\theta z j\in E_X;\} &&\text{for
x'_k&=x_k+\theta;\
x'_j&=x_j &&\text{for } j\notin E_X,\ j\ne k .
\end{aligned}
\tag{7}
]

To pass to the new plan (X'), it remains to determine the value (\theta). Denote by (E'X) ((E''_X)) the set of those indices (j\in E_X) for which (z) are determined by relation (6).}>0) ((z_{jk}<0)), where the (z_{jk

Let (\theta') be equal to the smallest of the three numbers:
[
\min_{j\in E'X}\frac{x_j-\alpha_j}{z,\qquad}
\min_{j\in E''X}\frac{x_j-\beta_j}{z,\qquad}
\beta_k-x_k .
]

Compute the quantities (\dfrac{x_j-d_{tj}}{z_{jk}}) for (j\in E'X+E''_X,\ j=k) and such (t) that
[
0<\frac{x_j-d=-1),}}{z_{jk}}<\theta' \quad (z_{kk
]
and arrange them in increasing order. Let the corresponding sequence of pairs of indices ((t,j)) have the form
[
(t_1,j_1),\ (t_2,j_2),\ldots,\ (t_s,j_s).
\tag{8}
]

Introduce the numbers
[
\sigma_0=C_{t_{k+1},k}-\sum_{i=1}^{m}\lambda_i a_{ik},\qquad
\sigma_\mu=\sigma_0+\sum_{\nu=1}^{\mu}\bigl(C_{t_{\nu}+1,j_{\nu}}-C_{t_{\nu},j_{\nu}}\bigr)\,|z_{j_\nu k}|,\quad
\mu=1,2,\ldots,s .
]

By assumption (\sigma_0>0). If (\sigma_s\le 0), then determine the integer (r) from the conditions (\sigma_{r-1}>0,\ \sigma_r\le 0) ((\sigma_0>\sigma_1>\cdots>\sigma_s)) and set
[
\theta=\frac{x_{j_r}-d_{t_r j_r}}{z_{j_r k}}.
]

If (\sigma_s>0), then (\theta=\theta').

It may happen that (\theta=\infty). This means that function (1) is not bounded above on the set of plans of problem II.

b) In this case the components of the new plan (X') are determined by the relations
[
\begin{aligned}
x'j&=x_j+\theta z j\in E_X;\} &&\text{for
x'_k&=x_k-\theta;\
x'_j&=x_j &&\text{for } j\notin E_X,\ j\ne k .
\end{aligned}
\tag{9}
]

The quantity (\theta) is computed according to rules similar to those described for case a). The iterations are carried out until either the desired solution is found or the unboundedness of function (1) on the set of plans of the problem under study is established. In the nondegenerate case, each iteration leads to an increase of function (1), whence the finiteness of the process under consideration follows. We shall not dwell here on questions connected with the phenomenon of degeneracy. Let us only note that, in the degenerate case, the danger of cycling (returning after a certain number of iterations to a support plan already encountered) can be eliminated by means of devices close to those used in linear programming.

1. In conclusion, we make several remarks. Making use of the piecewise linearity of the functions (f_i(x)), one can reduce problem II to an equivalent linear problem and then use for its solution one of the methods of linear programming ((^5)). However, reducing the problem under consideration to a linear-programming model is associated with a substantial increase in the number of variables and constraints, which, in turn, considerably complicates the computational work. As for the proposed method, its implementation does not require increasing the size of the problem. One of the merits of the method is that an increase in the number of breakpoints of the functions (f_i(x)) has little effect on the laboriousness of an individual iteration. At the same time, each iteration of the algorithm includes, generally speaking, several iterations of the plan-improvement method applied to the equivalent linear problem. In this connection, it is sensible to reduce a number of linear problems containing a large number of constraints of a special kind (such problems often occur in practice) to piecewise-linear problems of type II, and then to use the method set forth here.

The method is also applicable to minimization problems. In this case the functions (f_i(x)) must be convex downward. We note that, similarly to linear-programming algorithms, the method presented can be implemented in the form of successive filling of tables whose elements are calculated by simple recurrence formulas.

Received
23 II 1960

REFERENCES

1. L. V. Kantorovich, Mathematical Methods in the Organization and Planning of Production, 1939.
2. L. V. Kantorovich, DAN, 115, No. 3, 441 (1957).
3. G. B. Dantzig, “Activity Analysis of Production and Allocations,” 1951, p. 359.
4. L. V. Kantorovich, M. K. Gavurin, “Problems of Increasing the Efficiency of Transport Operations,” 1949, p. 110.
5. A. Charnes, C. E. Lemke, Naval Res. Logist. Quart., 1, No. 4, 301 (1954).