Mathematics

Hoang Tuy (Hoàng Tụy)

On Linear Inequalities

(Presented by Academician L. V. Kantorovich, 27 V 1967)

In the study of linear inequalities the following question often arises: under what conditions does there exist, for a given convex cone (C \subset R^n), a supporting hyperplane whose normal vector (t) satisfies the inequalities (p \leq t \leq q) (i.e. (p_j \leq t_j \leq q_j,\ j=1,2,\ldots,n)), where (p,q) are given vectors? In the present note we give an answer to this question. The theorem that we obtain is a generalization of the well-known Farkas–Minkowski lemma and proves very useful in justifying the fundamental results of the theory of mathematical programming.

1. The main theorem. We shall say that a set (C \subset R^n) satisfies condition ((S_K)), where (K) is some subset of the set ({1,2,\ldots,n}), if for every (k \in K) and every (J \subset K) the sets ({x \in C: x_j = 0\ (j \in J),\ x_k > 0}) and ({x \in C: x_j = 0\ (j \in J),\ x_k < 0}) are either both empty or both nonempty. We shall call a positive basis of the set (C) any set (B \subset C) such that every point (x \in C) can be represented as a positive linear combination of elements from (B), and a scheme* of the set (C) any set (E \subset C) containing, for every (J \subset {1,2,\ldots,n}), some positive basis of the set ({x \in C: x_j = 0\ (j \in J)}). Introduce the notation:

[
\langle p,q,x\rangle=\sum_{x_j\leq 0} p_jx_j+\sum_{x_j>0} q_jx_j;\qquad
\langle t,x\rangle=\langle t,t,x\rangle=\sum_{j=1}^{n} t_jx_j.
]

Then the following theorem is true, which refines and generalizes results published earlier in ((^{2})).

Theorem. Let a convex set (C \subset R^n) be given that satisfies condition ((S_K)) for some (K \subset {1,2,\ldots,n}), and let two vectors (p,q) be given such that (p \leq q), while the components (p_k,q_k) ((k \in K)) may be equal respectively to (-\infty,+\infty).

1) If there exists a vector (t \in R^n) satisfying the inequalities

[
p \leq t \leq q,\qquad \langle t,x\rangle \geq 0
\tag{1}
]

for every element (x) of some positive basis (B) of the set (C), then for all (x \in C) we have

[
\langle p,q,x\rangle \geq 0.
\tag{2}
]

2) Conversely, if inequality (2) holds for every (x) of some scheme (E) of the set (C), then there exists a vector (t \in R^n) satisfying inequalities (1) for all (x \in C).

Proof. The first part of the theorem is trivial. We prove the second. For (n=1) it is easy to verify that the theorem is true. Assuming that it is true in (R^{n-1}) ((n>1)), we prove it in (R^n). Without loss of generality one may assume that (k>j) for any (k \in K,\ j \notin K) (in the case when (K\ne)

* This is fulfilled, in particular, if ((x \in C)(\exists x' \in C)(\exists \lambda>0)(\forall j \in K)\ x'_j=-\lambda x_j).

(\ne \varnothing)). Let (C_n={x\in C:\ x_n=0}), (E_n={x\in E:\ x_n=0}). (C_n) may be regarded as a set in (R^{n-1}), for which (E_n) is a scheme. It is clear that (C_n) is convex and satisfies the condition ((S_{K\setminus{n}})). Therefore, by the induction hypothesis, there exists a vector (\bar t\in R^{n-1}) such that (p_j\leq \bar t_j\leq q_j)

[
(j=1,2,\ldots,n-1),\qquad \sum_{j=1}^{n-1}\bar t_j\bar x_j\geq 0
]

for all (x\in C_n). If (x_n=0) for all (x\in E), then the vector (t=(\bar t_1,\bar t_2,\ldots,\bar t_{n-1},t_n)), where (t_n) is an arbitrary number satisfying (p_n\leq t_n\leq q_n), will be the desired one. In the contrary case let

[
E_n^+={x\in E:\ x_n>0},\qquad E_n^-={x\in E:\ x_n<0},
]

and put, for every (x\in E_n^+\cup E_n^-),

[
\hat x=x/|x_n|,\quad
\alpha(x)=-\sum_{x_j\leq 0}p_j\hat x_j-\sum_{\substack{x_j>0\ j\ne n}}q_j\hat x_j,\quad
\beta(x)=\sum_{\substack{x_j\leq 0\ j\ne n}}p_j\hat x_j+\sum_{x_j>0}q_j\hat x_j .
]

Then, by virtue of (2), (\alpha(x)\leq q_n) ((x\in E_n^+)), (\beta(x)\geq p_n) ((x\in E_n^-)); hence, putting

[
\alpha=
\begin{cases}
\sup\limits_{x\in E_n^+}\alpha(x),& \text{if } E_n^+\ne\varnothing,\
-\infty,& \text{if } E_n^+=\varnothing,
\end{cases}
\qquad
\beta=
\begin{cases}
\inf\limits_{x\in E_n^-}\beta(x'),& \text{if } E_n^-\ne\varnothing,\
+\infty,& \text{if } E_n^-=\varnothing,
\end{cases}
]

we obtain

[
\alpha\leq q_n,\qquad \beta\geq p_n. \tag{3}
]

On the other hand, for any (x\in E_n^+), (x'\in E_n^-), we have (\lambda(\hat x+\hat x')\in C_n), where (\lambda=|x_nx_n'|/(|x_n|+|x_n'|)>0). Therefore

[
\sum_{j=1}^{n-1}\bar t_j(\hat x_j+\hat x'j)\geq 0
\quad\text{or}\quad
\sum\bar t_j\hat x'}^{n-1j\geq -\sum\bar t_j\hat x_j,}^{n-1
]

and consequently, taking into account that (p_j\leq \bar t_j\leq q_j) ((j=1,2,\ldots,n-1)),

[
\alpha(x)\leq -\sum_{j=1}^{n-1}\bar t_j\hat x_j\leq
\sum_{j=1}^{n-1}\bar t_j\hat x'_j\leq \beta(x'). \tag{4}
]

Hence

[
\alpha\leq \beta. \tag{5}
]

By condition, (p_n) and (q_n) are finite numbers, except possibly in the case when (n\in K); but in the latter case, since both sets (E_n^+) and (E_n^-) are nonempty, relations (4) show that (\alpha<+\infty), (\beta>-\infty). Therefore from (3) and (5) follows the existence of at least one finite number (t_n) satisfying (p_n\leq t_n\leq q_n), (\alpha\leq t_n\leq \beta). If we replace (p_n) and (q_n) by (t_n), then the vectors (p'=(p_1,p_2,\ldots,p_{n-1},t_n)) and (q'=(q_1,q_2,\ldots,q_{n-1},t_n)) also satisfy inequality (2) for all (x\in E), since for (x\in E_n^+)

[
t_nx_n\geq \alpha x_n\geq \alpha(x)x_n
=-\sum_{x_j<0}p_jx_j-\sum_{\substack{x_j>0\ j\ne n}}q_jx_j,
]

i.e. (\langle p',q',x\rangle\geq 0), and this inequality is proved analogously for (x\in E_n^-). Consequently, the preceding reasoning can be repeated, starting from (p') and (q') and considering (C_{n-1}={x\in C:\ x_{n-1}=0}) instead of (C_n). As a result one obtains a finite number (t_{n-1}\in[p_{n-1},q_{n-1}]) such that the vectors (p''=(p_1,\ldots,p_{n-2},t_{n-1},t_n)) and (q''=(q_1,\ldots,q_{n-2},t_{n-1},t_n)) satisfy inequality (2) for all (x\in E), and so on. In the end one obtains the vector (t=(t_1,t_2,\ldots,t_n)), which will be the desired one.

Corollary 1. If the inequality (\langle p,q,x\rangle\geq 0) holds for all (x) of some scheme of a convex set (C), then it also holds for all (x\in C).

2. Particular cases.

I. Suppose that the convex set (C) is such that from (x\in C) it follows that (-\lambda x\in C) for some (\lambda>0) (this occurs, for example, when (C) is a subspace in (R^n)). Then condition ((S_{{1,2,\ldots,n}})) is satisfied, and in the application of the theorem to this case the numbers (p_j, q_j) ((j=1,2,\ldots,n)) may be equal respectively to (-\infty) and (+\infty). On the other hand, the fulfillment of the inequality (\langle t,x\rangle\geq 0) for all (x) from some positive basis entails (\langle t,x\rangle\leq 0), and consequently (\langle t,x\rangle=0) for all (x\in C); similarly, the fulfillment of the inequality (\langle p,q,x\rangle\geq 0) for all (x) from some scheme entails (\langle q,p,x\rangle=-\langle p,q,-x\rangle\leq 0). Therefore:

1) If there exists a vector (t\in R^n) satisfying the conditions

[
p\leq t\leq' q,\qquad \langle t,x\rangle=0
\tag{6}
]

for all (x) of some positive basis, then for all (x\in C) we have

[
\langle q,p,x\rangle\leq 0\leq \langle p,q,x\rangle .
\tag{7}
]

2) Conversely, if inequalities (7) hold for all (x) of some scheme, then there exists a vector (t\in R^n) satisfying conditions (6) for all (x\in C).

II. Let us now take as (C) the set of solutions (x) of the inequality (Ax\geq 0), where (A) is a given (m\times n) matrix. Put
[
x'=(x,y)=(x_1,x_2,\ldots,x_n,y_1,y_2,\ldots,y_m),
]
(A'x'=Ax-y),
[
p'=(p_1,\ldots,p_n,\underbrace{-\infty,\ldots,-\infty}{m}),\qquad
q'=(q_1,\ldots,q_n,\underbrace{0,\ldots,0}).
]
It is not hard to see that the inequality (\langle p,q,x\rangle\geq 0) for all (x\in C) is equivalent to the inequality (\langle p',q',x'\rangle\geq 0) for all (x') from the subspace (A'x'=0). Consequently, there exists a vector (t'\in R^{n+m}) such that (p'\leq t'\leq' q') and (\langle t',x'\rangle=0) for all (x') from this subspace. Recalling that, by virtue of a well-known theorem of linear algebra, the vector (t') is orthogonal to all solutions of the equation (A'x'=0) if and only if (t'=A'^u) for some (u\in R^m) (here () denotes transposition), and noting that (A'^u=(A^u,-u)), we obtain

Corollary 2. The inequality (\langle p,q,x\rangle\geq 0) holds for all solutions (x) of the inequality (Ax\geq 0) if and only if there exists a vector (u\in R^m) such that (u\geq 0,\; p\leq A^*u\leq q).

Obviously, the well-known Farkas--Minkowski lemma is a particular case of this corollary, when (p=q).

3. Applications.

The preceding results make it possible to derive, in a very simple way, almost all the basic theorems of mathematical programming (theorems on the existence of an admissible solution, optimality criteria, duality theory, etc.). We give some examples.

I. The saddle-point theorem. Consider the following general problem of convex programming: maximize (f(x)) under the conditions that (x\in D), (g_i(x)\geq 0) ((i=1,2,\ldots,s)), where (D) is a closed convex set in (R^n), and (g(x), g_i(x)) are concave functions defined on (D).

Let (\bar x\in D). The set (C) of all points (z=(z_0,z_1,\ldots,z_s)) such that, for some point (x\in D) and some vector (t\geq 0), we have

[
z_0\geq f(\bar x)-f(x)+\sum_{i=1}^s t_i g_i(\bar x),\qquad
z_i\geq -g_i(x)\quad (i=1,2,\ldots,s),
]

is a convex set in (R^{s+1}). It is easy to verify that (\bar x) is an optimal solution of the problem if and only if

[
z_i\leq 0\ (i=1,2,\ldots,s)\Rightarrow z_0\geq 0.
\tag{8}
]

Suppose now that the functions (g_i(x)) satisfy the Slater condition (see, for example, (1)) in the sense that there exists a point (x^0\in D) such that,

that (g_i(x^0)>0) ((i=1,2,\ldots,s)). Then condition ((S_{{1,2,\ldots,s}})) is satisfied for (C), since for any (J\subset{1,2,\ldots,s}) and any (k) the sets ({z\in C:\ z_j=0\ (j\in J),\ z_k>0}) and ({z\in C:\ z_j=0\ (j=J),\ z_k<0}) are nonempty. Taking (p_0=q_0=1,\ p_i=0,\ q_i=+\infty) ((i=1,2,\ldots,s)), we see that implication (8) is equivalent to the condition (\langle p,q,z\rangle\ge 0) for all (z\in C) and, consequently, is equivalent to the existence of a vector (\bar t=(\bar t_1,\bar t_2,\ldots,\bar t_s)\ge 0) such that

[
z_0+\sum_{i=1}^{s}\bar t_i z_i\ge 0
\quad \text{for all } z\in C,
]

i.e.

[
f(\bar x)-f(x)+\sum_{i=1}^{s}t_i g_i(\bar x)-\sum_{i=1}^{s}\bar t_i g_i(x)\ge 0
]

for all (x\in D) and all (t\ge 0). If we put

[
F(x,t)=f(x)+\sum_{i=1}^{s} t_i g_i(x),
]

then we have (F(x,\bar t)\le F(\bar x,t)) for all (x\in D) and all (t\ge 0). Thus, we have obtained the Kuhn–Tucker result (in a somewhat more general form):

The point (\bar x\in D) is an optimal solution of the problem if and only if there exists a vector (\bar t\ge 0) such that ((\bar x,\bar t)) is a saddle point of the function

[
F(x,t)=f(x)+\sum_{i=1}^{s} t_i g_i(x)
]

in the domain (x\in D,\ t\ge 0).

II. Separation theorem. We have proved the Kuhn–Tucker theorem without reference to the classical separation theorem for convex sets. This latter theorem can be established using only our main theorem. For this purpose we first prove that, if a convex set (G\subset R^n) does not contain the point (0), then there exists a vector (t\ne 0) with the condition (\langle t,y\rangle\ge 0) for all (y\in G). Indeed, the assertion is obvious in (R^1). Assuming that it is true in (R^{n-1}), we prove it in (R^n). It may always be assumed that the sets ({y\in G:\ y_n>0}) and ({y\in G:\ y_n<0}) are nonempty, for if, for example, (y_n\ge 0) for all (y\in G), then the vector (t=(0,\ldots,0,1)) will be the desired one. Since (0\in G_n={y\in G:\ y_n=0}), there exists a vector ((\bar t_1,\bar t_2,\ldots,\bar t_{n-1})) satisfying

[
\sum_{j=1}^{n-1}\bar t_j y_j\ge 0
]

for all (y\in G_n). Putting (p_j=q_j=\bar t_j) ((j=1,2,\ldots,n-1)), (p_n=-\infty,\ q_n=+\infty), we have (\langle p,q,y\rangle\ge 0) for all (y\in G). Consequently, there exists a vector (t\in R^n) such that (p\le t\le q) (i.e. (t_j=\bar t_j) for (j=1,2,\ldots,n-1)) and (\langle t,y\rangle\ge 0) for all (y\in G), which proves the assertion.

Now let (C,C') be two convex sets having no common points. Then the set (G=C-C') is convex and does not contain the point (0). Therefore, by what has been proved, there exists a vector (t\ne 0) with the condition (\langle t,y\rangle\ge 0) for all (y\in G), i.e. (\langle t,x\rangle\ge \langle t,x'\rangle) for all (x\in C,\ x'\in C'). Thus, we have obtained a brief proof of the aforementioned separation theorem: for two nonintersecting convex sets there always exists a hyperplane separating them.

Hanoi University
Hanoi, Democratic Republic of Vietnam

Received
13 V 1967

REFERENCES

1. G. B. Dantzig, Linear Programming and Extensions, Moscow, 1962.
2. Hoàng Tụy, Coll. Math., 13, Fasc. 1, 107 (1964).