Full Text
MATHEMATICS
S. I. ZUKHOVITSKII, R. A. POLYAK
AN ALGORITHM FOR SOLVING THE PROBLEM OF RATIONAL CHEBYSHEV APPROXIMATION
(Presented by Academician A. Yu. Ishlinskii, 2 VI 1964)
- Let a system of \(2p\) fractional-linear functions be given
\[ R_i(x;y)\equiv \frac{a_i^T x}{b_i^T y}+\gamma_i =\frac{\alpha_{i1}x_1+\ldots+\alpha_{in}x_n}{\beta_{i1}y_1+\ldots+\beta_{im}y_m}+\gamma_i,\quad i\in I=\{1,\ldots,2p\} \tag{1} \]
and a bounded domain \(\Omega\), containing interior points, defined by the inequalities
\[ \varphi_j(x;y)\equiv \varphi_j(x_1,\ldots,x_n;\,y_1,\ldots,y_m)\leqslant 0,\quad j\in J=\{1,\ldots,q\}. \tag{2} \]
We shall assume that in the system (2), in which the functions \(\varphi_j(x,y)\) are convex and smooth, the inequalities
\[ b_i^T y\geqslant \tau>0,\quad i\in I;\qquad |y_i|-1\leqslant 0,\quad i\in I_y=\{1,\ldots,m\}, \tag{2′} \]
are included, where \(\tau>0\) is sufficiently small.
The problem of rational Chebyshev approximation consists in finding the Chebyshev point of the system (1) in the domain \(\Omega\), i.e. such a point \((x^*;y^*)\in\Omega\) at which
\[ \max_{i\in I} R_i(x^*;y^*)=\min_{(x;y)\in\Omega}\max_{i\in I} R_i(x;y). \tag{3} \]
The problem of Chebyshev approximation of a continuous function \(f(t)\) on \([a,b]\) by means of the rational function
\[ R(x;y;t)=(x_1+x_2t+\ldots+x_nt^{\,n-1})(y_1+y_2t+\ldots+y_mt^{\,m-1})^{-1}, \]
where the collection of coefficients \((x_1,\ldots,x_n;\,y_1,\ldots,y_m)\equiv (x;y)\) is chosen in the bounded convex and closed domain \(\Omega\), reduces to problem (1)—(3) if, instead of approximation on the whole segment \([a,b]\), approximation is considered on its \(\varepsilon\)-net consisting of \(p\) points * . In such a formulation the problem was considered in \((^1)\).
The function \(\max_{i\in I} R_i(x;y)\) is not convex; however, as it turned out, it has no local minima in the domain \(\Omega\). Therefore, for solving problem (1)—(2) one may apply a certain method of steepest descent.
Recently a number of algorithms have appeared for finding
\[ \min_{(x;y)\in\Omega}\max_i R_i(x;y), \]
when the system of constraints (2) consists only of inequalities (2′) (see, for example, \((^3,^4)\)).
In the present work two new algorithms are presented for solving problem (1)—(3), which, as already noted, represent a certain method of steepest descent with the introduction at each step of a special parameter (cf. \((^2,^5)\)).
\[ \text{* Instead of } \max_{1\leq i\leq p}\left|\frac{a_i^T x}{b_i^T y}+\gamma_i\right| \text{ we consider } \max_{1\leq i\leq 2p}\left(\frac{a_i^T x}{b_i^T y}+\gamma_i\right), \text{ where } a_{p+i}=-a_i,\ \gamma_{p+i}=-\gamma_i,\ b_{p+i}=b_i\ (i=1,\ldots,p). \]
- For each point \((x; y) \in \Omega\) define the following sets of indices
\[ I(x; y)=\{i \mid R_i(x; y)=\max_{1\le i\le 2p} R_i(x; y)\}, \]
\[ I(x; y; \delta)=\{i \mid R_i(x; y)>\max_{1\le i\le 2p} R_i(x; y)-\delta\}, \]
\[ J(x; y)=\{j \mid \varphi_j(x; y)=0\}, \]
\[ J(x; y; \delta)=\{j \mid -\delta<\varphi_j(x; y)\le 0\}. \]
For the initial approximation choose some point \((x^0; y^0)\in\Omega\), which we determine by applying, for example, to system (2) algorithm \((5)\). In addition, choose an arbitrary sufficiently small number \(\delta_1>0\) and determine the sets \(I(x^0; y^0; \delta_1)\) and \(J(x^0; y^0; \delta_1)\). The direction of descent from the point \((x^0; y^0)\) is determined from the following two conditions.
First, require that in this direction the steepest descent be realized, i.e., that the smallest of the absolute values of the derivatives
\[ \frac{d}{dt}\,[R_i(x^0+t\zeta_x;\ y^0+t\zeta_y)]_{t=0}, \]
normalized, for example, by the conditions
\[ |\zeta_{xi}|\le 1,\quad i\in I_x=\{1,\ldots,n\}; \qquad |\zeta_{yi}|\le 1,\quad i\in I_y=\{1,\ldots,m\} \]
be maximal. Taking into account the negativity of the derivatives, we conclude that the required direction
\((\zeta_x^1;\zeta_y^1)=(\zeta_{x1}^1,\ldots,\zeta_{xn}^1;\zeta_{y1}^1,\ldots,\zeta_{ym}^1)\) must be Chebyshev:
\[ \max_{i\in I(x^0;y^0;\delta_1)} \frac{d}{dt}\,[R_i(x^0+t\zeta_x^1;\ y^0+t\zeta_y^1)]_{t=0} = \]
\[ =\min_{(\zeta_x;\zeta_y)} \max_{i\in I(x^0;y^0;\delta_1)} \frac{d}{dt}\,[R_i(x^0+t\zeta_x;\ y^0+t\zeta_y)]_{t=0}. \]
Second, require that this direction lead into the interior of the domain \(\Omega\). Therefore, together with the parameter \(\delta_1>0\), choose another sufficiently small parameter \(\eta_1>0\), and to find the descent direction solve the following linear programming problem: minimize
\[ u=\xi \tag{4} \]
subject to the constraints
\[ \frac{d}{dt}\,[R_i(x^0+t\zeta_x;\ y^0+t\zeta_y)]_{t=0}-\xi= \]
\[ =(b_i^T y^0)^{-2}\bigl[(a_i^T\zeta_x)(b_i^T y^0)-(a_i^T x^0)(b_i^T\zeta_y)\bigr]-\xi\le 0, \quad i\in I(x^0;y^0;\delta_1); \tag{5} \]
\[ g_{xj}^T\zeta_x+g_{yj}^T\zeta_y\le -\eta_1, \quad j\in J(x^0;y^0;\delta_1); \]
\[ |\zeta_{xi}|\le 1,\quad i\in I_x;\qquad |\zeta_{yi}|\le 1,\quad i\in I_y, \]
where
\[ g_{xj}=g_{xj}(x^0;y^0)= \left( \frac{\partial\varphi_j(x^0;y^0)}{\partial x_1}, \ldots, \frac{\partial\varphi_j(x^0;y^0)}{\partial x_n} \right), \quad j\in J(x^0;y^0;\delta_1); \]
\[ g_{yj}=g_{yj}(x^0;y^0)= \left( \frac{\partial\varphi_j(x^0;y^0)}{\partial y_1}, \ldots, \frac{\partial\varphi_j(x^0;y^0)}{\partial y_m} \right), \quad j\in J(x^0;y^0;\delta_1). \]
Denote by \(u_1\) the minimum of \(u\) subject to constraints (5).
- Leave the parameters \(\delta_1\) and \(\eta_1\) unchanged if \(u_1<-\delta_1\), and change them if \(u_1\ge -\delta_1\). In this case, when \(0>u_1\ge -\delta_1\), set \(\delta_2=\delta_1/2\) and \(\eta_2=\eta_1/2\), while when \(u_1\ge 0\) it is necessary to check whether the point \((x^0;y^0)\) is a solution of problem (1)—(3). For this purpose solve the following linear programming problem: minimize \(u=\xi\) subject to the constraints
\[ (b_i^T y^0)^{-2}\bigl[(a_i^T\zeta_x)(b_i^T y^0)-(a_i^T x^0)(b_i^T\zeta_y)\bigr]-\xi\le 0, \quad i\in I(x^0;y^0); \]
\[ g_{xj}^T\zeta_x+g_{yj}^T\zeta_y\le 0, \quad j\in J(x^0;y^0); \]
\[ |\zeta_{xi}|\le 1,\quad i\in I_x;\qquad |\zeta_{yi}|\le 1,\quad i\in I_y. \]
If \(\min u = u_1' = 0\), then the point \((x^0;\ y^0)\) is a solution of problem (1)—(3). If, however, \(u_1' < 0\), then we put \(\delta_2 = \delta_1/2\), and, to determine \(\eta_2\), we solve one more linear programming problem: maximize
\[ v=\xi \tag{6} \]
subject to the constraints
\[ (b_i^T y^0)^{-2}\left[(a_i^T \xi_x)(b_i^T y^0)-(a_i^T x^0)(b_i^T \xi_y)\right]-\frac{u_1'}{2}\leq 0,\quad i\in I(x^0;\ y^0); \]
\[ g_{xj}^T\xi_x+g_{yj}^T\xi_y+\xi\leq 0,\quad j\in J(x^0;\ y^0); \tag{7} \]
\[ |\xi_{xi}|\leq 1,\quad i\in I_x;\qquad |\xi_{yi}|\leq 1,\quad i\in I_y, \]
and put
\[ \eta_2=\min\{\eta_1/2,\ \max v\}. \]
- Suppose that the descent direction has already been determined and that, most slowly along this direction, the function \(R_{i_0}(x^0+t\xi_x^1;\ y^0+t\xi_y^1)\), where \(i_0\in I(x^0;\ y^0)\), decreases, i.e.
\[ (b_{i_0}^T y^0)^{-2}\left[(a_{i_0}^T\xi_x^1)(b_{i_0}^T y^0)-(a_{i_0}^T x^0)(b_{i_0}^T\xi_y^1)\right]\geq \]
\[ \geq (b_i^T y^0)^{-2}\left[(a_i^T\xi_x^1)(b_i^T y^0)-(a_i^T x^0)(b_i^T\xi_y^1)\right],\quad i\in I(x^0;\ y^0)\setminus\{i_0\}. \]
One should move along the direction \((\xi_x^1;\ \xi_y^1)\), i.e. increase \(t\) in the formulas
\[ x=x^0+t\xi_x^1,\qquad y=y^0+t\xi_y^1, \]
up to the value \(t_1=\min\{t',t''\}\), where \(t'\) is the smallest positive root of the equations
\[ R_{i_0}(x^0+t\xi_x^1;\ y^0+t\xi_y^1) = R_i(x^0+t\xi_x^1;\ y^0+t\xi_y^1), \quad i\in I\setminus I(x^0;\ y^0;\ \delta_1)^{*}, \]
and \(t''\) is the smallest positive root of the equations
\[ \varphi_j(x^0+t\xi_x^1;\ y^0+t\xi_y^1)=0,\quad j\in J. \]
Taking the point \((x^1;\ y^1)=(x^0+t_1\xi_x^1;\ y^0+t_1\xi_y^1)\) as the initial approximation, we continue the process.
- The second algorithm differs from the preceding one in the choice of the descent direction. Instead of seeking at each step to attain the maximum admissible rate of decrease of the function \(\max_i R_i(x;\ y)\) and, at some steps, the greatest possible distance from the boundary of the domain \(\Omega\), we shall now choose the descent direction by solving, as it were, both problems in an averaged way. After choosing \(\delta_1\), we take as the direction the solution of the following linear programming problem: minimize
\[ w=\xi \tag{8} \]
subject to the constraints
\[ (b_i^T y^0)^{-2}\left[(a_i^T\xi_x)(b_i^T y^0)-(a_i^T x^0)(b_i^T\xi_y)\right]-\xi\leq 0,\quad i\in I(x^0;\ y^0;\ \delta_1); \]
\[ g_{xj}^T\xi_x+g_{yj}^T\xi_y-\xi\leq 0,\quad j\in J(x^0;\ y^0;\ \delta_1); \tag{9} \]
\[ |\xi_{xi}|\leq 1,\quad i\in I_x;\qquad |\xi_{yi}|\leq 1,\quad i\in I_y. \]
Denoting by \(w_1\) the value of \(\min w\) under the constraints (9), we put \(\delta_2=\delta_1\), if \(w_1<-\delta_1\), and \(\delta_2=\delta_1/2\), if \(0>w_1\geq-\delta_1\). If \(w_1=0\), then we solve one more linear programming problem—minimize
\[ w=\xi \tag{10} \]
subject to the constraints
\[ (b_i^T y^0)^{-2}\left[(a_i^T\xi_x)(b_i^T y^0)-(a_i^T x^0)(b_i^T\xi_y)\right]-\xi\leq 0,\quad i\in I(x^0;\ y^0); \]
\[ g_{xj}^T\xi_x+g_{yj}^T\xi_y-\xi\leq 0,\quad j\in J(x^0;\ y^0); \tag{11} \]
\[ |\xi_{xi}|\leq 1,\quad i\in I_x;\qquad |\xi_{yi}|\leq 1,\quad i\in I_y. \]
If it turns out that \(w_1=\min w=0\), then \((x^0;\ y^0)\) is a solution of problem (1)—(3). If, however, \(w_1<0\), then we continue the process, moving in the direction obtained in the solution of problem (10)—(11).
\[ \underline{\phantom{xxxxxxxx}} \]
* If the descent direction is determined from problem (6)—(7), then \(i\in I\setminus I(x^0;\ y^0)\).
- It is not difficult to verify the convergence of the algorithms described. Let us outline the proof, for example, for the second algorithm, i.e., let us show that
\[ \lim_{k\to\infty} \max_i R_i(x^k; y^k) = \min_{(x;y)\in\Omega} \max_i R_i(x; y). \]
First note that \(\delta_k \to 0\). Indeed, if
\(\lim_{k\to\infty}\delta_k=\delta>0\), then for sufficiently large \(k\) we would have \(|w_k|>\delta\). But then
\[ \max_i R_i(x^k; y^k)-\max_i R_i(x^{k+1}; y^{k+1}) \ge t_k\alpha, \]
where \(\alpha>0\) is some constant, and \(t_k>t_0>0\), so that we would have
\[ \max_i R_i(x^k; y^k)\to -\infty. \]
Let \((\tilde x; \tilde y)\) be a limit point of the sequence \(\{(x^k; y^k)\}\), and let the subsequence \((x^{k_\nu}; y^{k_\nu})\) converge to \((\tilde x; \tilde y)\). We shall assume that the values of the parameter \(\delta\) are changed at the steps \(k_\nu+1\). Suppose that \((\tilde x; \tilde y)\) is not a solution of problem (1)—(3). Then the solution of problem (10)—(11), with the replacement of the set \(I(x^0; y^0)\) by \(I(\tilde x; \tilde y)\) and of \(J(x^0; y^0)\) by \(J(\tilde x; \tilde y)\), gives \(|\min \tilde w|=|\tilde w|>0\). In view of the continuous differentiability of the functions \(R_i(x; y)\) and \(\varphi_j(x; y)\), and of the fact that \(\delta_{k_\nu}\) tends to zero, we obtain \(w_{k_\nu}<\tilde w/2<0\) for sufficiently large \(k_\nu\), so that \(|w_{k_\nu}|>|\tilde w|/2\); but this contradicts the condition
\[ |w_{k_\nu}| \le \delta_{k_\nu} \to 0. \]
Kiev State Pedagogical Institute
named after A. M. Gorky
Ukrainian Road-Transport
Scientific Research Institute
Received
18 IV 1964
CITED LITERATURE
¹ S. I. Zukhovitskii, Some Questions in the Theory of Chebyshev Approximations, Dissertation, Kiev, 1950.
² S. I. Zukhovitskii, R. A. Polyak, M. E. Primak, DAN, 153, No. 5, 991 (1963).
³ N. L. Loeb, J. Soc. Ind. Appl. Math., 8, No. 3 (1960).
⁴ E. W. Cheney, N. L. Loeb, Num. Math., 3 (1961).
⁵ S. I. Zukhovitskii, R. A. Polyak, M. E. Primak, DAN, 151, No. 1, 27 (1963).