Full Text
UDC 621.391.23:621.391.28–29:62-506.3
CYBERNETICS
AND CONTROL THEORY
V. V. PETROV, A. S. USKOV
CAPACITIES OF INFORMATION SYSTEMS WITH FEEDBACKS AND INTERNAL INTERFERENCE
(Presented by Academician B. N. Petrov, 11 IX 1969)
In the work \((^1)\), the connection is shown between Shannon’s ideas and the results of Kolmogorov—Wiener. The capacity determined by Shannon’s formula for arbitrary Gaussian processes (signal and noise) corresponds to the optimal characteristic obtained on the basis of the criterion of the minimum mean square of the random error (m.s.e.), without taking into account the condition of physical realizability. This characteristic is limiting from the point of view of the minimum m.s.e.; deviation from it makes it possible to obtain a physically realizable filter. Thus, it appears possible to propose a comparatively simple algorithm for finding the limiting characteristics and the corresponding capacities of dynamic systems with feedbacks and internal interference.
Fig. 1
The diagram of Fig. 1 corresponds to an information system of the reproduction type (sensor, instrument tracking system); \(W(s)\) is the transfer function of the unchangeable part of the system; \(W_k(s)\) is the transfer function of the correcting element.
In the system under consideration, a signal \(m(t)\) with noise \(n(t)\) superposed on it is applied to the input, so that the distorted signal has the form
\[ \varphi(t) = m(t) + n(t). \tag{1} \]
There is internal interference \(u(t)\); \(m(t)\), \(n(t)\), and \(u(t)\) are stationary, random, Gaussian processes with known correlation functions and zero mathematical expectations. The system must reproduce at the output the useful signal \(m(t)\) arriving at its input.
The optimal transfer function, obtained on the basis of the criterion of minimum m.s.e. without taking into account the condition of physical realizability, can be represented in the form
\[ \Phi(if) = D(if) / A(if), \tag{2} \]
where
\[ D(if) = S_m(f) + S_{mn}(f) - S_{mu}(f)W(-if) - S_{um}(f)W(if) - \]
\[ {} - S_{un}(f)W(if) + S_u(f)|W(if)|^2; \tag{3} \]
\[ A(if) = S_m(f) + S_n(f) + S_u(f)|W(if)|^2 - \]
\[ - S_{um}(f)W(if) - S_{un}(f)W(if) - S_{mu}(f)W(-if) - S_{nu}(f)W(-if); \tag{4} \]
\[ S_{m,n,u}(f)=\int_{-\infty}^{+\infty} R_{m,n,u}(\tau)e^{-i\omega\tau}\,d\tau; \tag{5} \]
\[ R_{m,n,u}(\tau)\simeq \frac{1}{T}\int_{0}^{T}\xi_{m,n,u}(t)\xi_{m,n,u}(t+\tau)\,dt; \tag{6} \]
\[ \xi_m(t)=m(t),\qquad \xi_n(t)=n(t),\qquad \xi_u(t)=u(t); \tag{7} \]
\[ S_{um,mu,un,nu}(f)=\int_{-\infty}^{+\infty} R_{um,mu,un,nu}(\tau)e^{-i\omega\tau}\,d\tau; \tag{8} \]
\[ R_{um,mu,un,nu}(\tau)\simeq \frac{1}{T}\int_{0}^{T}\xi_{u,m,u,n}(t+\tau)\xi_{m,u,n,u}(t)\,dt . \tag{9} \]
In paper \((^2)\), the optimal transfer function for the system under consideration is found under the additional condition of physical realizability.
The spectral density of the error for characteristic (2) is determined by formula (3)
\[ S_{\varepsilon}(f)=|\Phi_{\varepsilon}(if)|^{2}S_m(f)+|\Phi(if)|^{2}S_n(f)+|Y_u(if)|^{2}S_u(f), \tag{10} \]
where
\[ \Phi_{\varepsilon}(if)=1-\Phi(if)\quad \text{is the error transfer function;} \tag{11} \]
\[ Y_u(if)=\Phi_{\varepsilon}(if)W(if)\quad \text{is the transfer function of the internal interference.} \tag{12} \]
We represent the entropy of the error in the frequency band \(\Delta f\) in the form
\[ \bar H_{\varepsilon\Delta f}=T\Delta f\log 2\pi e S_{\varepsilon}(f)\Delta f . \tag{13} \]
Then the information rate of the error signal in the band \(\Delta f\) can be found from the formula
\[ R_{\varepsilon\Delta f}=\Delta f\log 2\pi e S_m(f)\Delta f-\Delta f\log S_m(f)/S_{\varepsilon}(f). \tag{14} \]
The total rate of information transmission by the error signal is determined by summation over all frequencies
\[ R_{\varepsilon}=\int_{W}\log[2\pi e S_m(f)\Delta f]\,df-\int_{W}\log\frac{S_m(f)}{S_{\varepsilon}(f)}\,df . \tag{15} \]
By virtue of expression (15), we obtain the formula for the capacity of the system
\[ C=\int_{W}\log\frac{S_m(f)}{S_{\varepsilon}(f)}\,df . \tag{16} \]
Below are expressions for capacities, obtained from formula (16) for various particular cases.
- \(m(t)\ne 0,\ n(t)\ne 0,\ u(t)=0,\ R_{mn}(\tau)=0:\)
\[ C=\int_{W}\log\frac{S_m(f)+S_n(f)}{S_n(f)}\,df . \tag{17} \]
- \(m(t)\ne 0,\ n(t)\ne 0,\ u(t)=0,\ R_{mn}(\tau)\ne 0:\)
\[ C=\int_{W}\log\frac{S_m(f)+S_n(f)}{S_n(f)+|S_{mn}(f)|^{2}/S_m(f)}\,df . \tag{18} \]
- \(m(t)\ne 0,\ n(t)\ne 0,\ u(t)\ne 0,\ R_{um}(\tau)=0,\ R_{un}(\tau)=0,\ R_{mn}(\tau)=0\):
\[ C=\int_W \log \frac{S_m(f)+S_n(f)+S_u(f)\lvert W(if)\rvert^2} {S_n(f)+\dfrac{S_u(f)S_n(f)}{S_m(f)}\lvert W(if)\rvert^2}\,df . \tag{19} \]
- \(m(t)\ne 0,\ n(t)\ne 0,\ u(t)\ne 0,\ R_{mn}(\tau)\ne 0,\ R_{un}(\tau)=0,\ R_{um}(\tau)=0\):
\[ C=\int_W \log \frac{S_m(f)+S_n(f)+S_u(f)\lvert W(if)\rvert^2} {S_n(f)+\dfrac{\lvert S_{mn}(f)\rvert^2}{S_m(f)} +\dfrac{S_n(f)S_u(f)\lvert W(if)\rvert^2}{S_m(f)}}\,df . \tag{20} \]
Analogous expressions can be obtained for the capacities when \(R_{un}(\tau)\ne 0,\ R_{um}(\tau)\ne 0\).
Moscow Aviation Institute
named after Sergo Ordzhonikidze
Received
20 VII 1969
CITED LITERATURE
- B. N. Petrov, V. V. Petrov et al., Proceedings of the IV All-Union Conference on Automatic Control, 1, Moscow, 1969.
- M. J. Pelegrin, Calcul statistique des systemes asservis, Paris, 1953.
- V. V. Solodovnikov, Statistical Dynamics of Linear Automatic-Control Systems, Moscow, 1960.