Full Text
UDC 519.95
CYBERNETICS AND CONTROL THEORY
G. A. GOLITSYN
ON LOCAL PRINCIPLES OF ANIMAL BEHAVIOR
(Presented by Academician P. K. Anokhin on 21 III 1967)
In (¹) an integral formulation was proposed for the basic principle of animal behavior—the principle of homeostasis. According to this formulation, every organism strives to minimize the functional
\[ I=\int_0^T W\,dt=\int_0^T \sum_i \frac{a_i y_i^2}{2}\,dt, \tag{1} \]
where \(y_i\) is the deviation of some vitally important variable from the norm (a need); \(a_i\) is the subjective weight or importance of this need; \(t\) is time; the product \(M_i=a_i y_i\) is the subjective sensation of the need, or motivation (desire, drive).
The rates of change of the variables \(\dot y_i\) are, as a rule, subject to a nonintegrable constraint (the law of exclusivity (²)):
\[ \dot y_i \dot y_k = 0,\quad i\ne k. \tag{2} \]
A consequence of the law of exclusivity is the emergence of an alternative situation, when the animal is forced to choose between reaction \(i\) and reaction \(k\). The purpose of the present work is to find an algorithm for this choice.
Formally, the problem may be posed as follows: if, from two decisions \(u_i\) and \(u_k\) (for example, eating and drinking), the animal chooses decision \(u_i\), this means that decision \(u_i\) is in some respect better than \(u_k\), i.e., some quantity (choice criterion) \(E\) is greater in this case:
\[ E_i > E_k. \tag{3} \]
Let \(u_i=+1\), \(u_k=-1\); \(u=\pm1\); then the choice algorithms can be written as follows:
\[ u=\operatorname{sign}(E_i-E_k). \tag{4} \]
The problem is to determine the structure of \(E\), i.e., to find the functional dependence of \(E\) on \(a\), \(y\), \(t\), and possibly on some other variables.
Suppose that the animal has chosen decision \(u_i\). Let us divide time into intervals and imagine the subsequent behavior in the following form. At each interval the animal receives a triggering signal with probability \(s_i\) and reacts to it with probability \(r_i\). As a result of the reaction the animal receives reinforcement (a partial reduction of the need), whose expected magnitude is equal to \(\overline{\Delta y_i}\). Then the mean rate of satisfaction of the need is
\[ \dot y_i=-s_i r_i \frac{\overline{\Delta y_i}}{\Delta t}\,\frac{1+u}{2}. \tag{5} \]
Analogously, for decision \(u_k\):
\[ \dot y_k=-s_k r_k \frac{\overline{\Delta y_k}}{\Delta t}\,\frac{1-u}{2}. \tag{6} \]
Here \(\Delta t\) is the duration of the interval: \(\Delta t=\Delta t_0+\Delta t_c\); \(\Delta t_0\) is the waiting time for reinforcement; \(\Delta t_c\) is the proper time of satisfying the need.
The dependence of \(\dot y\) on \(u\) is chosen so as to satisfy the law of exclusivity
\[ \dot y_i \dot y_k = s_i s_k r_i r_k \frac{\overline{\Delta y_i}\,\overline{\Delta y_k}}{\Delta t^2} \frac{1-u^2}{4} =0 . \tag{7} \]
Let us find a choice algorithm that ensures a minimum of \(I\) under the additional conditions (5)—(6). Since \(\dot y\) does not enter the integrand explicitly, a sufficient condition for a minimum of \(I\) is \(\min W\) at each instant of time \(t\). In turn, in order to achieve \(\min W(t)\), it is sufficient to choose \(u\) in such a way that the increment \(dW\) on each time interval \(dt\) is minimal:
\[ dW=(W_{y_i}\dot y_i+W_{y_k}\dot y_k)\,dt = \]
\[ =\left( -M_i s_i r_i \frac{\Delta y_i}{\Delta t}\frac{1+u}{2} - M_k s_k r_k \frac{\Delta y_k}{\Delta t}\frac{1-u}{2} \right) = \min_u dW . \tag{8} \]
Hence it follows immediately that
\[ u=\operatorname{sign}\left( M_i s_i r_i \frac{\overline{\Delta y_i}}{\Delta t} - M_k s_k r_k \frac{\overline{\Delta y_k}}{\Delta t} \right). \tag{9} \]
Comparing this expression with (4), we find
\[ E = Msr\frac{\overline{\Delta y}}{\Delta t} = Msr\frac{\overline{\Delta y}}{\Delta t_0+\Delta t_c}. \tag{10} \]
Attempts are known to solve this problem in a purely empirical way. For comparison we cite the results of K. Hull \((^3)\). Hull introduces an intermediate variable \({}_sE_R\) (reaction potential), which determines all the basic characteristics of the reaction. In particular, in an alternative situation of two reactions, the one is chosen for which \({}_sE_R\) is larger. In this sense \({}_sE_R\) is identical to \(E\). According to Hull, the structure of \({}_sE_R\) is
\[ {}_sE_R = D \times V \times {}_sH_R \times K \times J, \tag{11} \]
where \(D\) is motivation (\(=M\)); \(V\) is the intensity of the triggering stimulus (analog of \(s\)); \({}_sH_R\) is habit (analog of \(r\)); \(K(\Delta y)\) is an increasing function of the magnitude of reinforcement; \(J(\Delta t_0)\) is a decreasing function of the waiting time for reinforcement.
For small values of \(\Delta y\), the quantities \(K\) and \(\Delta y\) coincide up to a constant factor. The same may be said with respect to \(J\) and \(\Delta t_0\). The probability \(s\) of perceiving a signal, taking into account the threshold of receptor sensitivity and the presence of internal and external noise, will be greater the greater the signal intensity \(V\). Thus, \(s\) is an increasing function of \(V\). The variables \({}_sH_R\) and \(r\) differ in that \(r\) reflects both the formation and the extinction of a reaction; in Hull, \({}_sH_R\) describes only the acquisition of a habit; to describe extinction of the reaction Hull introduces an additional variable—the inhibitory potential, which is subtracted from \({}_sE_R\).
Thus, there is an undoubted similarity between formulas (10) and (11). This fact may be regarded as empirical confirmation of the theory, since Hull’s formula is based on extensive experimental material. At the same time, the advantage of our approach should be noted: Hull’s formula was selected empirically and is therefore inevitably arbitrary in many respects. Formula (11), however, is obtained as a direct consequence of the single general and sufficiently firmly established principle—the principle of homeostasis.
First Moscow Medical Institute
named after I. M. Sechenov
Received
15 III 1967
CITED LITERATURE
- G. A. Golitsyn, Izv. AN SSSR, Technical Cybernetics, No. 3 (1965).
- P. K. Anokhin, Internal Inhibition as a Problem of Physiology, Moscow, 1957.
- C. L. Hull, Essentials of Behavior, New Haven, 1951.