UDC 519.44

MATHEMATICS

R. A. ZAIDMAN, Academician Yu. V. LINNIK, I. V. ROMANOVSKY

SEQUENTIAL ESTIMATION PLANS AND MARKOV STOPPING TIMES

Recently, interest has increased in problems of sequential estimation (see, for example, \((^{2,3})\)). It should be noted that in this area even the simplest problems have not been solved, such as, for example, the problem of optimal unbiased estimation of a probability in the Bernoulli scheme under various types of loss.

In view of this, it is natural first to consider problems of sequential estimation only for the simplest stochastic processes: the binomial (I) and multinomial (II) schemes, the Poisson process (III), and the Wiener process with unknown drift (IV). The natural parameters of these processes will be estimated. Sequential observations are carried out (discrete in schemes I—II). A sequential estimation rule is determined by a stopping rule (plan) and by an estimating function specified at the stopping points. We shall restrict ourselves to plans corresponding to Markov stopping times.

In problems of optimal control of Markov processes, an optimal (or \(\varepsilon\)-optimal) stopping plan can usually be sought in the form of a “first-entry” plan. Under the additional condition of unbiasedness, or other analogous restrictions characteristic of the estimation problem, the class of such plans becomes too poor, and the theory of excessive majorants (see, for example, \((^4)\)) is not directly applicable.

Nevertheless, the study of first-entry plans is useful because of their simplicity and their connection with functions effectively estimable with quadratic weight \((^2)\).

In all the cases considered we shall use the following terms: a first-entry plan \(S\), the boundary of a plan \(\partial S\), attainable, boundary, and unattainable points, a closed plan, a complete plan, a bounded plan, a minimal plan, a plan of volume \(n\). For this terminology see \((^{1-3})\). In what follows, all our plans \(S\) will be first-entry plans without unattainable points; completeness will always mean boundedness (see \((^1)\)); by a bounded plan we shall mean a plan for which \(\partial S\) lies in a bounded region of the Cartesian product of phase space and time. In the cases of processes I—IV under consideration, upon reaching the boundary one can carry out an unbiased estimation of a scalar function of the parameter by means of a function \(f(X_\tau,\tau)\), where \((X_\tau,\tau)\) is a sufficient statistic; here \(\tau\) is the stopping time, and \(X_\tau\) is the phase at the stopping time. The function \(f(X_\tau)\) is unique for a complete plan.

Thus, for unbiased estimation of a given function, optimal in one sense or another, we must choose an appropriate plan \(S\). Such a plan need not be a first-entry plan (as we have noted, the known optimization theory (see \((^{4-6})\)) is not directly applicable here because of the complicated dependence of the estimate \(f(\cdot)\) on the plan \(S\)). But here we shall not dwell on the estimation problem itself, but only on the connection of plans \(S\) with Markov times \(\tau\). The simplest and most studied is the plan \(S\) of sampling a constant volume (\(\tau=N\), where \(N\) is a constant integer). It is natural to try to generalize this plan,

considering, for example, plans with \(E_p\tau=\mathrm{const}\) or imposing other analogous conditions on the stopping times \(\tau\).* Unexpectedly, the imposition of such conditions restricts the plans \(S\) themselves extremely severely. We shall prove the corresponding theorems for estimation plans for processes I–IV. Let us note that in all these cases minimal closed plans are determined by distributions for all values of the parameter.

I. The binomial process.

Theorem 1. A complete bounded plan is determined by the values of \(E_p\tau\) on an arbitrarily small interval of values of \(p\).

As was found by de Groot \((^2)\), functions \(g(p)\) that are unbiasedly and efficiently estimable at \(p=p_0\) must have the form \(g(p)=a(p-p_0)E_p\tau+b\) (\(a,b\) are arbitrary constants). This form is naturally called the type of efficiently estimable functions. According to Theorem 1, the number of types of functions \(g(p)\) efficiently estimable by means of complete plans of volume \(n\) is equal to the number of such plans. Therefore a formula enumerating the number of complete plans of volume \(n\) is useful. Such plans have exactly one point on each of the axes, at distances from the origin \(a\) and, respectively, \(b\). Let \(A_n(a,b)\) be the number of plans with given \(a,b\). Then, for \(a\le b\) (this does not violate generality, since \(A_n(a,b)=A_n(b,a)\)),

\[ A_n(a,b)=\sum_{i=a-1}^{n-1} A_{n-1}(i,b-1), \]

Theorem 2. For unbounded complete plans, closed for some \(p\ne 0,1\), one of the following three assertions holds:

A. The plan is closed for \(p\in[0,p_0)\) for some \(p_0\in(0,1)\) and is not closed for \(p\in(p_0,1]\). The boundary of the plan contains exactly one point on each vertical line and, consequently, is determined by some integer-valued function \(f(n)\). If \(\lim_{n\to\infty} f(n)n^{-1}=\alpha\), then \(p_0=(1+\alpha)^{-1}\) (analogously in case B).

B. The plan is closed for \((p_0,1]\) for some \(p_0\in(0,1)\) and is not closed for \(p\in[0,p_0)\). The boundary of the plan contains exactly one point on each horizontal line.

C. The plan is closed both in a neighborhood of 0 and in a neighborhood of 1. Its boundary consists of two parts, forming respectively plans of type A and B.

Theorem 3. An unbounded complete plan is uniquely determined by \(E_p\tau\). Moreover, if \(E_p\tau\) is bounded at \(p=0\) and \(p=1\), then the plan is of type C; if only at \(p=0\), respectively \(p=1\), then the plan is of type A, respectively B.

Some kinds of plans are determined by restrictions imposed on \(\tau\) at a single value of the parameter \(p=p_0\). Such, for example, is the inverse-sampling plan \(S\)—sampling until exactly \(k\) events with probability \(q=1-p\) appear. As is well known, in this case we have
\(E_{p_0}\tau=k/q_0;\ E_{p_0}'\tau=k/q_0^2;\ E_{p_0}\tau^2=(k^2+kp_0)/q_0\).

Theorem 4. Let \(p_0\in(0,1)\) be some prescribed value and suppose that for the plan \(S\) we have

\[ E_{p_0}\tau=k/q_0;\qquad E_{p_0}'\tau=k/q_0^2;\qquad E_{p_0}\tau^2\le (k^2+kp_0)/q_0^2. \tag{1} \]

Then \(S\) is an inverse-sampling plan.

This theorem is a special case of the following:

Theorem 5. Let the boundary \(\partial S\) of a minimal plan \(S\) consist of \(m\) straight-line segments and \(n\) points. Put \(2m+4n=K\). If \(p_0\in(0,1)\) is a prescribed value of \(p\) and there exist \(K(K+1)/2\) expectations and their derivatives:

\[ E_{p_0}^{(r)}(\tau^s);\qquad r+s\le K;\qquad s\ge 1, \tag{2} \]

then the plan \(S\) is determined by these values.

* The expectation of a stopping time at a given probability \(p\) in the binomial process is denoted by \(E_p\tau\).

We note that there are infinitely many “oblique” plans of type A or B. For plans of type A the boundary \(\partial S\) has \(y=kx+b\), where \(k>0\) and \(b\geq 0\) are integers; plans of type B are symmetric to plans of type A. To define them, three relations of type (1) are sufficient.

II. Multinomial random walk with \(n\) outcomes is considered analogously to the binomial one. Here we have \(n\) possible steps along the axes \(x_1,\ldots,x_n\) with probabilities \(p_1,\ldots,p_n\). Necessary and sufficient geometric conditions for completeness of bounded first-entry plans, analogous to the Lehmann–Stein condition \((^1)\), are unknown*.

Consider the set \(P_K\) of integer points \(x=\{x_i\}\) of the \(n\)-dimensional simplex

\[ \sum_i x_i=K,\qquad x_i\geq 0. \]

Let \(Q\subset P_K\). We shall say that a point \(\{x_i\}\) belonging to the set \(Q\) is controlled by the point \(Ke_j\) (\(e_j\) is the \(j\)-th unit vector) if all points \(\{\alpha_i'\}\), where \(\alpha_i'\leq x_i,\ i\ne j\), and \(x_j'>x_j\), belong to \(Q\). A plan \(S\) is called controlled if any unattainable point \(\{x_i\}\) is controlled by some one of the points

\[ \left(\sum_i x_i\right)e_i \]

in the set of unattainable points of level \(\sum_i x_i\).

Theorem 6. A bounded minimal controlled plan is complete.

For multinomial random walks one can formulate an analogue of Theorem 5. A minimal plan \(S\), whose boundary \(\partial S\) consists of a prescribed number of parts of linear subspaces \(R_n\), is determined by values of quantities of the form \(D^{l_1\ldots l_k-1}E_{p_0}(\tau^s)\), where \(p_0=(p_{0,1},\ldots,p_{0,k-1})\) lies inside the simplex of probabilities; \(D^{l_1\ldots l_k-1}\) is the sign of differentiation \(l_i\) times with respect to \(p_i\) at the point \(p_0\); \(l_1+\cdots+l_{n-1}+s\leq C\), where \(C\) is readily determined from the number and type of the linear subspaces forming \(\partial S\); it is assumed that the given quantities exist.

III. Poisson process. We shall consider the simplest homogeneous Poisson process with parameter \(\lambda\). A first-entry plan \(S\) will be assumed specified by a boundary \(\partial S\) on the set \((N\times T)\), where \(N\) is the set of nonnegative integers, and \(T\) is the set of positive numbers; moreover, we shall assume that the boundary consists of a finite or countable number of points and intervals whose endpoints have no limit points (see (3)).

Theorem 7. A bounded plan is determined by the value \(E_\lambda\tau\).

This makes it possible to classify the types of functions \(g(\lambda)\) effectively estimable at a given point (as in the case of the binomial process).

Theorem 8 (analogue of Theorem 4). Inverse-sampling plans and oblique plans are determined by the values \(E_{\lambda_0}(\tau)\), \(E_{\lambda_0}(\tau^2)\), and \(E'_{\lambda}(\tau)\) at some point \(\lambda_0>0\), where these quantities exist.

Theorem 9 (completeness conditions for bounded plans). For completeness of a bounded plan it is necessary and sufficient that \(\operatorname{mes}(\partial S)=T\). Here \(\operatorname{mes}(\partial S)\) is the Lebesgue measure of the boundary of the plan, and \(T\) is the exact upper bound of the possible stopping times.

IV. Wiener process. Let \(\xi(t)\) be standard Brownian motion; \(x(t)=\xi(t)+\lambda(t)\), where \(\lambda\) is the drift parameter. For estimating \(\lambda\), bounded first-entry plans \(S\) with boundary \(\partial S\) consisting of smooth curves are applicable: \(x=f_1(t)\); \(x=f_2(t)\); \(f_1(0)<0<f_2(0)\) \((0\leq t\leq T)\) and the vertical line \(t=T\).

Theorem 10. Let the boundary of a minimal plan satisfy an algebraic equation of degree \(k\) (for example, consist of a finite number of rectilinear segments). Then the plan is determined by \(k(2k+1)\) values of the form \(E_{\lambda_0}^{2}(\tau^s)\); \(s+2\leq 2k\); \(s\geq 1\) for some prescribed \(\lambda_0\).

* The necessary completeness conditions given in (7) are incorrect.

Analogous theorems hold for many other Markov processes. They have applications in the theory of sequential estimation.

Received
21 I 1969

REFERENCES

¹ E. L. Lehmann, Ch. Stein, Ann. Math. Stat., 21, 376 (1950).
² M. H. De Groot, Ann. Math. Stat., 30, 80 (1959).
³ S. Trybuła, Dissertationes Mathematicae, 60, Warszawa, 1968.
⁴ E. B. Dynkin, Markov Processes, Moscow, 1963.
⁵ J. A. Hunt, Markov Processes and Potentials, IL, 1962.
⁶ A. N. Shiryaev, Cybernetics, No. 3, 1 (1965).
⁷ R. Mukhamedzhanova, M. Suleimanov, Transactions of the Romanovskii Institute of Mathematics, Academy of Sciences of the Uzbek SSR, issue 22, 121 (1961).