UDC 061.3(47):62—506.1+519.55

CYBERNETICS AND CONTROL THEORY

V. A. YAKUBOVICH

FINITELY CONVERGENT ALGORITHMS FOR SOLVING COUNTABLE SYSTEMS OF INEQUALITIES AND THEIR APPLICATION TO PROBLEMS OF SYNTHESIS OF ADAPTIVE SYSTEMS

(Presented by Academician V. I. Smirnov, April 3, 1969)

1°. Problems of synthesis of adaptive systems in the formulation set forth in ((^1, ^2)) lead to the following problem. Let (x) be a vector of a Euclidean space (\mathfrak R). For a given (x_0 \in \mathfrak G \subset \mathfrak R), a sequence (x_t \in \mathfrak G \subset \mathfrak R) is defined by a relation of the form

[
x_{t+1}=f(t,x_t), \qquad t=0,1,2,\ldots,
\tag{1}
]

which must be constructed in accordance with the requirements indicated below. It is assumed that there exists (but is unknown) some algorithm (A) which, for any (t=0,1,2,\ldots), from the vectors (x_0,\ldots,x_t) produces a function (\varphi_t(x)) and a number (\alpha_t>0). Consider the countable system of inequalities

[
\varphi_t(x)<\alpha_t, \qquad t=0,1,2,\ldots
\tag{2}
]

It is assumed below that the algorithm (A) satisfies the following basic condition: there exist a vector (x_* \in \mathfrak G) (which is unknown and, generally speaking, depends on the algorithm (1) and on (x_0)) and a number (\rho), (0<\rho<1), such that the “strengthened” system of inequalities is satisfied

[
\varphi_t(x_*)<\rho^2\alpha_t, \qquad t=0,1,2,\ldots
\tag{3}
]

Algorithm (1) is called a finitely convergent algorithm for solving inequalities (2) if there exists (t_0>0) such that (x_{t_0}=x_{t_0+1}=\cdots=z) and, for (t \geq t_0), (x=z) satisfies all inequalities (2). Thus, a finitely convergent algorithm delivers, in a finite number of steps, a solution of an infinite number of “not shown in advance” inequalities. The number (r) of distinct vectors in the sequence (x_0,x_1,\ldots) is called the number of corrections of the finitely convergent algorithm. The problem indicated above consists in constructing a finitely convergent algorithm (1) possessing, insofar as possible, “rapid” convergence, i.e., an algorithm with a “good” estimate of the number (r). The function (f(t,x)) in (1) must be defined through (\varphi_t(x)) and (\alpha_t).

Finitely convergent algorithms can also be applied to solve a given finite system of inequalities (\varphi_1(x)<\alpha_1,\ldots,\varphi_T(x)<\alpha_T). For this purpose one may, for example, cyclically continue the given system, i.e., set in (2) (\varphi_{t+T}(x)=\varphi_t(x)), (\alpha_{t+T}=\alpha_t), and regard the stopping condition for algorithm (1) as the fulfillment of the relations (x_{t-T+1}=\cdots=x_t), (\varphi_j(x_j)<\alpha_j), (j=t-T+1,\ldots,t). In paper ((^3)), finitely convergent algorithms were obtained, together with estimates of the number of corrections (r), for the case when (\varphi_t(x)=(c_t,x)+\gamma_t), (\varphi_t(x)=|(c_t,x)+\gamma_t|), and a method was also indicated for their construction in the general case when the inequalities (2) define convex domains in (\mathfrak R). Some problems of synthesis of adaptive systems lead, however, to inequalities (2) with functions (\varphi_t(x)) containing unknown quantities on which algorithm (1) must not depend. For these problems the method of ((^3)) is sometimes not applicable. Some of the algorithms given below

can be used in this situation as well. In addition, below are algorithms that are simpler than the algorithms obtained by method ((^3)), as well as algorithms (Theorems 3 and 5) pertaining to the case where the inequalities (2) may define in (\mathfrak R) “weakly” nonconvex regions. The proofs given below are based on the same simple idea as the proofs in ((^3)).

(2^\circ). Let us first consider quadratic inequalities(^*). The inequalities (\psi_t(x) \equiv (K_t x,x)+2(k_t,x)+\varkappa_t<0), where (K_t>0,\ k_t\in\mathfrak R), can be transformed, introducing the notation (a_t=K_t^{-1}k_t), to the form (1) with functions

[
\varphi_t(x)=(K_t(x-a_t),(x-a_t)).
\tag{4}
]

Theorem 1. Suppose that in (2) (a_t\geq \alpha>0); suppose (0\leq K_t\leq H), where (H>0) is some matrix, and (\beta', \beta'', \beta_t) are arbitrary numbers satisfying the inequalities (0<\beta'\leq \beta_t\leq \beta''<2(1-\rho)). For any (x_0\in\mathfrak R), the algorithm (x_{t+1}=x_t), if (\varphi_t(x_t)<a_t), (x_{t+1}=x_t-\beta_tH^{-1}K_t(x_t-a_t)), if (\varphi_t(x_t)\geq a_t), is a finitely convergent algorithm for solving the inequalities (2) with functions (4). For the number of corrections of the algorithm the estimate (r\leq (H(x_0-x_),(x_0-x_))\delta^{-1}) holds, where (\delta=2\alpha\beta'(1-\rho-\beta''/2)).

Proof. Delete from the sequence (x_0,x_1,x_2,\ldots) those (x_t) for which (\varphi_t(x_t)<a_t), and renumber, in order, by the indices (0,1,2,\ldots) the remaining (x_t). Since (x_{t+1}=x_t) when (\varphi_t(x_t)<a_t), the new sequence (x_0,x_1,\ldots) will begin with the same vector, and for it (x_{t+1}=x_t-\beta_tH^{-1}K_t(x_t-a_t)), (\varphi_t(x_t)\geq a_t), will hold. It is required to show that the new sequence contains only a finite number of vectors (x_t). Let (V_t=(H(x_t-x_),(x_t-x_))). We shall show that (V_{t+1}<V_t-\delta). Putting (x_t-a_t=y_t), (x_-a_t=z_t), (\Delta V_t=V_{t+1}-V_t), we obtain
[
(-\Delta V_t/2\beta_t)=(K_ty_t,y_t)-(K_ty_t,z_t)-\beta_t(H^{-1}K_ty_t,K_ty_t)/2.
]
From (0\leq K_t\leq H) it follows that (K_tH^{-1}K_t\leq K_t). Therefore
[
(-\Delta V_t/2\beta_t)\geq |y_t|t^2-|y_t|_t\cdot |z_t|_t-\beta_t|y_t|_t^2/2.
]
Here (|y_t|_t^2=(K_ty_t,y_t)=\varphi_t(x_t)\geq a_t), (|z_t|_t^2=(K_tz_t,z_t)=\varphi_t(x)<\rho^2a_t). Using the inequalities for (\beta_t), we find that
[
(-\Delta V_t)\geq 2a_t\beta_t(1-\rho-\beta_t/2)\geq \delta>0.
]
It follows that the number of undeleted terms (x_t) is finite and that for their number, i.e., for the number of corrections of the algorithm, the estimate (V_0\geq V_0-V_r\geq r\delta) indicated in the theorem holds.

Theorem 2. Suppose that in (2) (a_t\geq \alpha>0) and in (4) (K_t=\lambda_tL_t), where the numbers (\lambda_t) and matrices (L_t) satisfy the conditions (0<\lambda'\leq \lambda_t\leq \lambda''), (0\leq L_t\leq \varkappa I) ((\varkappa>0) is a number). Put (\xi=(1-\rho)\lambda'\varkappa^{-1}). The algorithm (x_{t+1}=x_t), if (\varphi_t(x_t)<a_t), (x_{t+1}=x_t-\xi L_t(x_t-a_t)), if (\varphi_t(x_t)\geq a_t), is, for any (x_0\in\mathfrak R), a finitely convergent algorithm for solving the inequalities (2) with functions (4), and for the number of its corrections the estimate (r\leq \varkappa |x_0-x_|^2\alpha^{-1}(1-\rho)^{-2}\lambda''/\lambda') holds.*

The algorithm of Theorem 2 can be applied to solving the inequalities (2), (4) with (K_t=\lambda_tL_t) and unknown (\lambda_t), (0<\lambda'\leq \lambda_t\leq \lambda''). For the proof it suffices to apply Theorem 1 with (H=\varkappa I), (\beta_t=\varkappa\xi\lambda_t^{-1}), (\beta'=\varkappa\xi/\lambda''), (\beta''=\varkappa\xi/\lambda'), and to choose the “most advantageous” (\xi), for which (\delta) is maximal.

Theorem 3. Consider inequalities (2) with functions (4), in which the (K_t) are not necessarily nonnegative matrices. Suppose (a_t\geq \alpha>0), (|a_t|\leq \mathrm{const}), and an unknown vector (x_) satisfies the relation ((H_0(x_0-x_),(x_0-x_))\leq \gamma_0^2), with known (H_0>0), (x_0), and (\gamma_0>0). Suppose that (-\varkappa_1H_0\leq K_t\leq \varkappa_2H_0) is fulfilled, where (\varkappa_1\gamma_0<\alpha(1-\rho^2)), (\varkappa_2>0). Define numbers (\gamma>0), (\mu>0) by the relations ((H_0(x_0-a_t),(x_0-a_t))\leq \gamma^2), (K_t^H_0^{-1}K_t\leq \mu H_0), (t=0,1,2,\ldots). Let (\beta',\beta'',\beta_t) be arbitrary numbers such that (0<\beta'\leq \beta_t\leq \beta''<[\alpha(1-\rho^2)-\varkappa_1\gamma_0^2]\mu^{-1}(2\gamma_0+\gamma)^{-2}). For the indicated (x_0), the algorithm (x_{t+1}=x_t), if (\varphi(x_t)<a_t), (x_{t+1}=x_t-\beta_tH_0^{-1}K_t(x_t-a_t)), if (\varphi(x_t)<a_t), is a finitely convergent algorithm for solving the inequalities

(^) Below, the notation (K>0), where (K) is a matrix, means that (K=K^) is a positive definite matrix. The notation (K'\geq K'') means that ((K'x,x)\geq (K''x,x)) for any (x\in\mathfrak E). The identity matrix is denoted by (I).

(2), (4). For the number of corrections of the algorithm the estimate (r \leqslant \gamma_0^2\delta_0^{-1}) is valid, where
[
\delta_0=\beta'\bigl[\alpha(1-\rho^2)-\varkappa_1\gamma_0^2-\beta''\mu(2\gamma_0+\gamma)^2\bigr]>0.
]

Proof. Repeating the proof of Theorem 1 with (H=H_0), we transform the expression for (\Delta V_t) to the form
[
(-\Delta V_t/\beta_t)=\varphi_t(x_t)-\varphi_t(x_)+(K_t(x_t-x_),(x_t-x_))-\beta_t(K_t^H_0^{-1}K_ty_t,y_t).
]
Assuming by induction that (V_t\leqslant \gamma_0^2), after a series of transformations using the inequalities indicated in the theorem, we obtain
[
-\Delta V_t\geqslant \delta_0>0.
]
Therefore (V_{t+1}\leqslant \gamma_0^2), (0\leqslant V_r\leqslant V_0-\delta_0r\leqslant \gamma_0^2-\delta_0r). Hence it follows that the number (r) of uncrossed-out values (x_t) is finite and (r\leqslant \gamma_0^2\delta_0^{-1}).

Theorem 4. Let (H>0) be an arbitrary matrix. Suppose that in (2) (a_t\geqslant \alpha>0) and the functions (\varphi_t(x)) satisfy the following conditions: (I) (\varphi_t(x)) is twice continuously differentiable; (II) there exists (\varkappa>0) such that
[
(H^{-1}\nabla\varphi_t(x),\nabla\varphi_t(x))\leqslant \varkappa\varphi_t(x)
]
in the region (\varphi_t(x)\geqslant \alpha); (III) (\nabla^2\varphi(x)\geqslant 0). Put (\beta=\varkappa^{-1}(1-\rho^2)). The algorithm
[
x_{t+1}=x_t,\ \text{if } \varphi_t(x_t)<\alpha;\qquad
x_{t+1}=x_t-\beta H\nabla\varphi_t(x_t),\ \text{if } \varphi_t(x_t)\leqslant \alpha_t,
\tag{5}
]
is, for any (x_0\in \mathfrak R), a finitely convergent algorithm for solving the inequalities (2), and for the number of its corrections the estimate
[
r\leqslant (H(x_0-x_),(x_0-x_*))\delta_0^{-1}
]
is valid, where
[
\delta_0=(1-\rho^2)^2\varkappa^{-1}\alpha.
]

Proof. We perform the same operation of crossing out and renumbering (x_t) as in the proof of Theorem 1. For the function
[
V_t=(H(x_t-x_0),(x_t-x_0))
]
with the remaining (x_t) we have
[
(-\Delta V_t)=V_t-V_{t+1}
=\beta(\nabla\varphi_t(x_t),2x_t-2x_-\beta H^{-1}\nabla\varphi_t(x_t)).
]
Let (x,y\in \mathfrak G) be arbitrary,
[
x(\lambda)=(1-\lambda)x+\lambda y,\qquad \Phi(\lambda)=\varphi_t(x(\lambda)).
]
Representing the difference (\Phi(1)-\Phi(0)) by Taylor’s formula, we obtain
[
\varphi_t(y)-\varphi_t(x)=(\nabla\varphi_t(x),y-x)+\frac12(\nabla^2\varphi_tx(\lambda_0),(y-x)).
]
Using this expression for (x=x_t,\ y=x_), we transform ((-\Delta V_t)) to the form
[
(-\Delta V_t/\beta)=2\varphi_t(x_t)-2\varphi_t(x_)+
(\nabla^2\varphi_tx(\lambda_0),(x_t-x_))
-\beta|H^{-1/2}\nabla\varphi_t(x_t)|^2.
]
Using properties (II), (III), inequality (3), (2>\varkappa\beta), and the inequality (\varphi_t(x_t)\geqslant \alpha_t), which is satisfied by condition for the uncrossed-out (x_t), we find
[
(-\Delta V_t)\geqslant (2-\varkappa\beta)\varphi_t(x_t)-2\rho^2\beta\alpha_t\geqslant \delta(\beta),
]
where
[
\delta(\beta)=\alpha\beta[2(1-\rho^2)-\varkappa\beta].
]
For the value of (\beta) chosen in the theorem, (\delta(\beta)=\delta_0) is maximal and (\delta_0>0). Therefore the number (r) of uncrossed-out values (x_t) is finite. The estimate for (r) follows from the inequalities
[
V_0\geqslant V_0-V_r\geqslant r\delta_0.
]

Theorem 5. Suppose that the unknown vector (x_) satisfies the estimate
[
(H(x_0-x_),(x_0-x_*))\leqslant \gamma
]
with known (H>0), (x_0), and (\gamma). Suppose that in (2) (a_t\geqslant \alpha>0), the functions (\varphi_t(x)) satisfy conditions (I), (II) of Theorem 4 and condition (IIIa)
[
\nabla^2\varphi_t(x)\geqslant -2\varepsilon H,
]
where
[
\varepsilon\gamma<(1-\rho^2)\alpha.
]
Put
[
\beta=(1-\rho^2-\alpha^{-1}\gamma\varepsilon)\varkappa^{-1}.
]
For the indicated values of (x_0) and (\beta), algorithm (5) is a finitely convergent algorithm for solving the inequalities (2), and for the number of its corrections the estimate
[
r\leqslant \gamma(\alpha\beta^2\varkappa)^{-1}
]
is valid.

The proof is analogous to the proof of Theorem 4.

Remark. Theorems 1–5 carry over almost without change to the case when (\mathfrak R) is a Hilbert space. Theorems 1–3, in particular, remain valid if (\mathfrak R) is a real Hilbert space and (K_t) are bounded self-adjoint operators.

(3^0.) Below we use the terminology and notation from (2)**. We shall assume that the following conditions are fulfilled:

* Below, by (\nabla\varphi(x)) we denote the gradient vector of the function (\varphi(x)), and by (\nabla^2\varphi(x)) the matrix of second derivatives:
[
\nabla\varphi(x)=|\partial\varphi/\partial x_j|,\qquad
\nabla^2\varphi(x)=|\partial^2\varphi/\partial x_j\partial x_k|.
]

** Correction. We give additions to the theorems from ((^1,^2)), estimating the convergence rate of the algorithms ((^1,^2)), which the author, through oversight, omitted when shortening the text of articles ((^1,^2)). We note that the proofs given in ((^1,^2)) refer precisely to the full formulations. In article ((^1)), Theorem 1 must be supplemented by the following assertion:

Suppose that in (II) the numbers (\varepsilon_j^) are not variable parameters. Define the number (\rho_0) from the conditions
[
\varepsilon_j^/\varepsilon_j<\rho_0<1,\quad j=1,\ldots,k,
]
and the number (N) from the condition
[
d_N<(1-\rho_0)\varepsilon/2,
]

((\mathrm{II}_1)) There exist new controls (v), where ({v}\subset R_q), connected with the old ones by the formula (u=u(v)) (where (u(v)) is some function) such that the robot wins the (j)-th game if, in the course of the (j)-th game, for every (t) (as long as (\mu_t=1)) the inequality (\gamma_t<\varepsilon_t) is satisfied, where
[
\gamma_t=(\gamma_t^{(1)})^2+\ldots+(\gamma_t^{(k)})^2,
]
[
\gamma_t^{(h)}=\alpha_t(c_h,v_t)+\beta_t^{(h)}.
]
Here (0<\alpha'\leq \alpha_t\leq \alpha''), (\varepsilon_t\geq \varepsilon>0), (c_h\in R_q), and (\alpha'), (\alpha''), (\varepsilon), (c_h) are parameters.

((\mathrm{II}_2)) There exists a function (V^u(\sigma,\xi)), called the ideal control, such that for any (\xi\in M), when (v_t=V^u(\sigma_t,\xi)), (\mu_t=1), one has
[
\gamma_t\leq \rho^2\varepsilon_t,
]
where (\rho) is a parameter, (0<\rho<1).

((\mathrm{II}3)) For any (v_t), the values (\gamma_t^{(h)}), (\varepsilon_t) are expressed through (v_t,\sigma_t,\sigma).

((\mathrm{II}_4)) There exist (\partial V^u/\partial\sigma_j) and
[
\left|\partial V^u/\partial\sigma_j\right|\leq C_1,\qquad |V^u|\leq C_2,
]
where (C_1,C_2) are parameters.

Theorem 6. If conditions ((\mathrm{II}_1))—((\mathrm{II}_4)) are satisfied, brain equations can be constructed so that the resulting robot becomes reasonable in the class of problems (M). These equations are constructed as follows. Let (\rho_1>0) be so small that
[
\rho_0^2=\rho^2+2\alpha''\rho_1C\varepsilon^{-1/2}+(\alpha''C\rho_1)^2\varepsilon^{-1}<1,
]
where
[
C=(|c_1|^2+\ldots+|c_k|^2)^{1/2}.
]
Let a number (N>0) and functions (v_1(\sigma),\ldots,v_N(\sigma)) with values in (R_q) be such that
[
\left|V^u(\sigma,\xi)-[\tau_1(\xi)v_1(\sigma)+\ldots+\tau_N(\xi)v_N(\sigma)]\right|<\rho_1
]
for all (\sigma\in{\sigma}), (\xi\in M), where (\tau(\xi)) are certain scalar functions. Let
[
\sum_{h=1}^{k}\sum_{j=1}^{N}(v_j(\sigma),c_h)^2\leq \chi_0.
]
Put ({\tau}=R_N) and define equation (IV)(^{(2)}) by the relations
[
u_t=u(v_t),\qquad v_t=\tau_{1t}v_1(\sigma_t)+\ldots+\tau_{Nt}v_N(\sigma_t).
]
Define equation (V)(^{(2)}) by the relations (\tau_{t+1}=\tau_t), if (\gamma_t<\varepsilon_t),
[
\tau_{t+1}=\tau_t-\xi\bigl(\gamma_t^{(1)}a_t^{(1)}+\ldots+\gamma_t^{(k)}a_t^{(k)}\bigr),
]
where
[
\tau_t=|\tau_{jt}|{j=1}^{N},\qquad
a_t^{(h)}=|(v_j(\sigma_t),c_h)|,}^{N
]
[
\xi=(1-\rho_0)\chi_0\alpha'/\alpha''.
]
Then the number of instants (t) for which (\mu_t=1), (\gamma_t\geq\varepsilon_t), and hence also the number of games lost by the robot, does not exceed
[
\chi_0\alpha''(1-\rho_0)^2\varepsilon'''|\tau_0-\tau(\xi)|^2,
]
where
[
\tau(\xi)=|\tau_j(\xi)|.
]

Theorem 6 is proved according to the scheme of (^{(2)}), using Theorem 2. Theorem 6 can be applied to the construction of brain equations for the robots “hawk,” “eye—hand,” “grasshopper” from (^{(1,2)}), functioning, unlike (^{(1,2)}), not in a plane but in three-dimensional space.

Leningrad State University
named after A. A. Zhdanov

Received
25 III 1968

REFERENCES

(^{1}) V. A. Yakubovich, DAN, 182, No. 3 (1968).
(^{2}) V. A. Yakubovich, DAN, 183, No. 2 (1968).
(^{3}) V. A. Yakubovich, DAN, 166, No. 6 (1966).

where
[
\varepsilon=\min \varepsilon_j
]
and (d_N) is the Kolmogorov width of the set of functions
[
f(\sigma)=(c_h,V^u(\sigma,\xi)),\qquad \xi\in M,\quad h=1,\ldots,k.
]
The brain equations of a reasonable robot can be constructed so that ({\tau}) is a Euclidean space of dimension (Nk), and, for the number (r) of instants at which the target condition is not fulfilled, the estimate holds
[
r\leq |\tau_0-\tau(\xi)|^2\varepsilon^{-2}(1-\rho_0)^{-2},
]
where (\tau_0) is the initial tactic and (\tau(\xi)\in{\tau}).

Theorem 2 (^{(1)}) must be supplemented by the following assertion:

For the number (r) of instants (t) at which the target condition is not fulfilled, the estimate holds
[
r\leq |\tau_0-\tau(\xi)|^2\varepsilon_1^{-2}(1-\rho^2)^{-2}\max[v_1(\sigma)^2+\ldots+v_N(\sigma)^2],
]
where
[
\varepsilon_1^*/\varepsilon_1<\rho_0<1/2,
]
(\tau_0) is the initial tactic and (\tau(\xi)\in{\tau}).

where
[
\varepsilon_1^*/\varepsilon_1<\rho_0<1/2,
]
where (\tau_0) is the initial tactic and (\tau(\xi)\in{\tau}).

In the article (^{(2)}), the text of Theorem 1 must be supplemented by the words:

Moreover, ({\tau}) will be a Euclidean space of dimension (Nk). The number (N) is determined from the condition
[
d_N<(1-\rho)\varepsilon''/2\chi,
]
where
[
\chi=\max \chi^{(h)},
]
and (d_N) is the Kolmogorov width in (C) of the set of functions
[
f(\sigma)=(c_h,V^u(\sigma,\xi)),\qquad \xi\in M,\quad h=1,\ldots,k.
]
For the number (r) of games lost by the robot, the estimate holds
[
r\leq |\tau_0-\tau(\xi)|^2\times \chi^2(\varepsilon'')^{-2}\times(1-\rho)^{-2},
]
where (\tau_0) is the initial tactic and (\tau(\xi)\in{\tau}).

The text of Theorem 2 (^{(2)}) must be supplemented by the following words:

For the number (r) of games lost by the robot, the estimate
[
r\leq |\tau_0-\tau(\xi)|^2\times(\chi^{(1)})^2(1-2\rho)^{-2}(\varepsilon'')^{-2}\max |a_t|^2
]
will be valid.

The text of Theorem 3 (^{(2)}) must be supplemented by the following words:

For the number (r) of corrections of the algorithm, the estimate holds
[
r\leq |\tau_0-\tau_*|^2\cdot(\chi^{(1)})^2\times(1-2\rho')^{-2}\cdot\max |a_t|^2.
]