MATHEMATICS

A. L. BRUDNO and A. L. LUNTS

ON THE FILTERING OF RANDOM SEQUENCES

(Presented by Academician A. A. Blagonravov, 10 X 1959)

1. Consider a random function (u(t)), which is measured at discrete instants of time
[
\ldots, t_{n+2}, t_{n+1}, t_n
]
(the instant (t_{k+1}) precedes the instant (t_k); (t_n) is the instant of the last measurement). The measurements contain random errors (\xi_k), so that they yield the values
[
v_k = v(t_k) = u(t_k) + \xi_k .
]

In a number of practical problems connected with the use of discrete control machines, it is required, from the measurement results (v) ((k=n, n+1, n+2,\ldots)), to find the best approximation to the quantity (u_n = u(t_n)). In doing so, it is essential what amount of memory and how many operations are required for the computations.

As an example of such problems, we point to the determination of the wind velocity (u) on American bombers: from the results (v_k) of measurements, at equal time intervals, of the wind velocity (u_k), the quantity
[
s_n = \frac{31}{32}s_{n+1} + \frac{1}{32}v ,
]
is constantly (and recurrently) computed; this is then taken as the approximate value of (u).

The solutions of the problems considered may be different depending on the known properties of the random function (u(t)).

Below we give solutions for the case of Brownian motion, when (u(t)) has random increments with mathematical expectation equal to zero (Theorem 1), and for the case when (u(t)) is the sum of two functions—one having random increments, and the other having random second differences (Theorem 2). Theorem 3 has a negative significance: it shows the impossibility of generalizing Theorem 1 to a case weaker than in Theorem 2; there does not exist a best solution of the problem of determining the position of a Brownian particle in a flow with constant (unknown) velocity.

2.1. Let the values (u_i)
[
\ldots, u_2, u_1, u_0
]
have random increments (\zeta_i), so that
[
u_i = u_{i+1} + \zeta_i .
]

The values (u_i) are measured with errors (\xi_i), so that the measurement gives the result
[
v_i = u_i + \xi_i .
]

It is assumed that (\zeta_i) and (\xi_i) are independent random variables with mathematical expectations equal to zero:
[
M\zeta_i = M\xi_i = 0,
]

and constant variances

[
\mathbf{D}^{2}\zeta_i = D_1^2,\qquad \mathbf{D}^{2}\xi_i = D_2^2.
]

It is required to find a linear function (f({v_i})) of the results of the measurements (v_i), possessing the following properties:

a) for any fixed (but unknown) value (u_0), the mathematical expectation (\mathbf{M}f) must be equal to (u_0):

[
\mathbf{M}f({v_i})=u_0;
]

b) among all functions possessing property a), the required function (f) must have minimal variance (\mathbf{D}^{2}f):

[
\mathbf{D}^{2}f({v_i})=\mathbf{M}[f({v_i})-u_0]^2.
]

2.2. Theorem 1. The solution of the problem posed in 2.1 is the function

[
f=(1-\lambda)\sum_{i=0}^{\infty}\lambda^i v_i,
\tag{1}
]

where (\lambda) is determined from the equation

[
\frac{\lambda}{(1-\lambda)^2}=\frac{\mathbf{D}^{2}\xi}{\mathbf{D}^{2}\zeta},\qquad
\text{i.e. }\lambda=\delta-\sqrt{\delta^2-1};\qquad
\delta=1+\frac{D_1^2}{2D_2^2}.
]

In this case

[
\mathbf{D}^{2}f=(1-\lambda)\mathbf{D}^{2}\xi=\frac{\lambda}{1-\lambda}\mathbf{D}^{2}\zeta.
]

Remark 1. It follows from Theorem 1 that the best approximations (f) to the quantities (u_n) are found from the known results of previous measurements ({v_i}) ((i=n,n+1,\ldots)) by the recurrence formula

[
f=\lambda f_{n+1}+(1-\lambda)v_n,
]

since

[
f_k=(1-\lambda)\sum_{i=0}^{\infty}\lambda^i v_{k+i}\quad \text{(see (1)).}
]

Remark 2. In the case (\mathbf{D}^{2}\zeta \ll \mathbf{D}^{2}\xi), when the function (u) changes little and the measurements are rough, the number (\lambda) is close to 1, and

[
1-\lambda \simeq \sqrt{\mathbf{D}^{2}\zeta/\mathbf{D}^{2}\xi},\qquad
\mathbf{D}^{2}f \simeq \sqrt{\mathbf{D}^{2}\zeta \cdot \mathbf{D}^{2}\xi}.
]

3.1. Let us now consider the case when each (u_i),

[
\ldots,u_2,u_1,u_0,
]

is the sum of quantities (h_i) and (k_i),

[
u_i=h_i+k_i
]

where (k_i) is a quantity with independent random increments (first differences) (\zeta_i):

[
k_i-k_{i+1}=\zeta_i,\qquad \text{i.e. } k_i=k_{i+1}+\zeta_i,
]

and (h_i) is a quantity with independent random second differences (\eta_i):

[
h_i-h_{i+1}=z_i,\qquad z_i-z_{i+1}=\eta_i.
]

Suppose the values (u_i) are measured with error (\xi_i), so that the result of measurement is

[
v_i = u_i + \xi_i .
]

Suppose, further, that (\eta_i, \zeta_i) and (\xi_i) are independent random variables with mathematical expectation

[
\mathbf{M}\eta_i = \mathbf{M}\zeta_i = \mathbf{M}\xi_i = 0
]

and constant variances

[
\mathbf{D}^2\eta_i = D_1^2,\qquad \mathbf{D}^2\zeta_i = D_2,\qquad \mathbf{D}^2\xi_i = D_3^2 .
]

It is required to find a linear function (f_0=f({v_i})) of the results of measurements ({v_i}) possessing the following properties:

a(_1)) for any fixed (unknown) values (u_0) and (u_1), the mathematical expectation of (f_0) must be equal to (u_0):

[
\mathbf{M}[f({v_i})] = u_0;
]

b(_1)) among all functions possessing property a(_1)), the required function must have the least variance

[
\mathbf{D}^2 f({v_i})=\mathbf{M}[f({v_i})-u_0]^2 .
]

3.2. Before formulating Theorem 2, which gives the solution of this problem, we shall make some remarks about the quantities (D_1^2) and (D_3^2) and introduce the necessary notation.

We note that one may assume (D_3^2 \ne 0), since in the case (D_3^2=0), evidently, (f_0=v_0=u_0). Further, we shall assume (D_1^2\ne0); the case (D_1^2=0) is considered in Theorem 3.

Let

[
d_1=\frac{D_1^2}{D_3^2},\qquad d_2=\frac{D_2^2}{D_3^2}.
]

Next, denote by (\beta_1) and (\beta_2) those two of the four roots of the equation

[
\beta^4-(4+d_2)\beta^3+(6+2d_2+d_1)\beta^2-(4+d_2)\beta+1=0,
]

whose moduli are less than one. It is easy to see that under the assumption made, (D_1^2\ne0), such roots (\beta_1) and (\beta_2) always exist.

Theorem 2. The solution of the problem posed in subsection 3.1, under the conditions (D_1^2\ne0,\ D_3^2\ne0,\ d_2^2\ne4d_1), is the function

[
f_0=f({v_i})=
\frac{\beta_1\beta_2}{\beta_2(1-\beta_1)-\beta_1(1-\beta_2)}
\sum_{n=0}^{\infty}\left[(1-\beta_1)^2\beta_1^{\,n-1}-(1-\beta_2)^2\beta_2^{\,n-1}\right]v_n .
\tag{2}
]

In the case (d_2^2=4d_1) the problem has no unique solution.

From Theorem 2 it is easy to obtain recurrence relations for computing the best approximation (f_n) to the quantity (u_n). Namely, in the case of real (\beta_1) and (\beta_2) (which corresponds to (d_2>4d_1)), (f_n) can be obtained as the sum

[
f_n=A_n+B_n,
]

where (A_n) and (B_n) are quantities computed recurrently:

[
A_n=\beta_1 A_{n+1}+\delta_1 v_n,\qquad B_n=\beta_2 B_{n+1}+\delta_2 v_n;
\tag{3}
]

here

[
\delta_1=\frac{\beta_2(1-\beta_1)^2}{\beta_2(1-\beta_1)-\beta_1(1-\beta_2)},\qquad
\delta_2=\frac{\beta_1(1-\beta_2)^2}{\beta_2(1-\beta_1)-\beta_1(1-\beta_2)}.
]

In the case (d_2^2<4d_1), when (\beta_1) and (\beta_2) are complex conjugates, (\beta_1=a+ib), (\beta_2=a-ib), it is convenient to use formulas in which (f_n) is computed recurrently simultaneously with the auxiliary quantity (g_n):

[
f_n=af_{n+1}-bg_{n+1}+cv_n,\qquad
g_n=bf_{n+1}+ag_{n+1}-dv_n;
\tag{4}
]

here

[
c=1+a-2(a^2+b^2),\qquad
d=b+2\frac{1-a}{b}(a^2+b^2-a).
]

Since (|\beta_1|<1) and (|\beta_2|<1), formulas (3), as well as formulas (4), ensure stable computation; computational errors do not accumulate, and the influence of the quantities (v_{n+k}), (A_{n+k}), and (B_{n+k}) in formulas (3), respectively (v_{n+k}), (f_{n+k}), and (g_{n+k}) in formulas (4), tends to zero as (k\to\infty).

Theorem 2 admits a generalization to the case when (u_i) is the sum of a finite number (s) of quantities with random first, second, ..., (s)-th differences.

1. Let the values (u_i)

[
\ldots,u_2,u_1,u_0
]

satisfy all the requirements set forth in Sec. 2.1, with the exception of the condition (\mathrm M\zeta_i=0), which we now replace by the requirement that the mean value of (\zeta_i) be a constant:

[
\mathrm M\zeta_i=C.
]

It is essential to bear in mind that the quantity (C) is assumed by us to be unknown in advance.

Theorem 3. There does not exist a function

[
f=\sum_{i=0}^{\infty} a_i v_i
]

of the results ({v_i}) of measuring the quantities (u_i) ((i=0,1,2,\ldots)), satisfying conditions a) and b) of Sec. 2.1, if (\mathrm M\zeta_i) is an unknown constant.

Let us note that if only a finite number of values (v_i) ((i=0,1,2,\ldots,n)) is known, then a function

[
f=\sum_{i=0}^{n} a_i v_i,
]

satisfying conditions a) and b) of Sec. 2.1 for (\mathrm M\zeta_i=C), can be found. However, the set of limiting values ({a_i(n)}) as (n\to\infty) no longer satisfies the conditions of the problem.

Institute of Electronic Control Machines
Academy of Sciences of the USSR

Received
6 IX 1959