UDC 519.281

MATHEMATICS

Academician Yu. V. LINNIK

APPROXIMATELY MINIMAX DETECTION OF A VECTOR SIGNAL ON A GAUSSIAN BACKGROUND

This note adjoins the note \((^2)\). A signal is considered in the form of a \(p\)-dimensional column vector \(\xi\), and the background in the form of a normal vector \(X-\xi \in N(0,\Sigma)\), where \(\Sigma\) is an unknown nondegenerate correlation matrix. A repeated sample \(X_1,\ldots,X_N\) is taken. The hypothesis of absence of signal \(H_0:\ \xi=0\) is tested against the composite hypothesis \(H_1:\ N\xi^T\Sigma^{-1}\xi=\delta\), and also against the composite hypothesis \(H_1':\ N\xi^T\Sigma^{-1}\xi \geqslant \delta\), where \(\delta>0\) is a given number.

The expression \(N\xi^T\Sigma^{-1}\xi\) may be interpreted as the relative energy of the signal, and the two problems posed—as problems of detecting a signal of prescribed relative energy \(\delta\), or of relative energy not less than \(\delta\). We seek a minimax procedure for signal detection, i.e., such a (randomized) test \(\Phi\) for which, at a given level \(\alpha\), the minimum power on the composite alternative \(H_1\) (or \(H_1'\)) is maximal among all tests \(\Phi\) of level \(\leqslant \alpha\).

The works of many authors (see \((^1)\)) lead one to suppose that such a test is the well-known Hotelling test, based on the statistic

\[ T^2=N(N-1)\bar X^T S^{-1}\bar X. \quad \text{Here } \bar X=\frac{1}{N}\sum_{i=1}^{N}X_i,\quad S=\sum_{i=1}^{N}(X_i-\bar X)(X_i-\bar X)^T. \]

The test has the form \(T^2 \geqslant T_0^2\). In \((^1)\) this supposition was proved for testing \(H_0\) against \(H_1\) for \(p=2\) and \(N=3\); in \((^2)\), for testing \(H_0\) against \(H_1\) and \(H_1'\) for \(p=2\) and \(N=4\). Further progress encounters great analytical difficulties.

In the present note the requirement of minimaxity is replaced by the weaker requirement of “\(\varepsilon\)-minimaxity” of a test: for any \(\varepsilon>0\), its minimum power on the alternatives \(H_1\) or \(H_1'\), at a given level \(\alpha\) and sample size \(N \geqslant N_0(\varepsilon)\), must differ from the maximum of the minimum powers over all tests of level \(\leqslant \alpha\) by no more than \(\varepsilon\).

Of course, for a given level \(\alpha\in(0,1)\) and increasing \(N\), only weak signals are of interest, for which \(\delta=N\xi^T\Sigma^{-1}\) can grow only slowly; otherwise the power will tend to 1, and the assertion of \(\varepsilon\)-minimaxity will become trivial.

Theorem 1. The Hotelling test \(\Phi_N:T^2\geqslant T_0^2\) for testing the hypothesis \(H_0\) against the hypothesis \(H_1\) is \(\varepsilon\)-minimax for every level \(\alpha\in(0,1)\): for any \(\varepsilon>0\) the relation holds

\[ \sup_{\Phi}\inf_{\theta\in H_1}E_\theta\Phi-\inf_{\theta\in H_1}E_\theta\Phi_N\leqslant \varepsilon \tag{1} \]

for \(N>N_0(\varepsilon)\).

Here \(\theta=(\xi,\Sigma)\) is the parameter (\(\xi\) is the parameter under study, \(\Sigma\) the nuisance parameter); \(\Phi\) ranges over all tests of level \(\leqslant\alpha\); \(E_\theta\) denotes the sign of mathematical expectation.

* In this form Theorem 1 can apparently also be derived from A. Wald’s theorems (see \((^3)\)).

More precise information is given by Theorem \(1'\).

Theorem \(1'\). If the level \(\alpha=\alpha_N\) is subject to the conditions

\[ O\left(\exp -(\ln N)^{1/6}\right)\leq \alpha \leq 1-O\left(\exp -(\ln N)^{1/6}\right) \tag{2} \]

and we have

\[ \exp\left[-(\ln N)^{1/6}\right]\leq \delta \leq (\ln N)^{1/6}, \tag{3} \]

then

\[ \sup_{\Phi}\inf_{\theta\in H_1} E_\theta\Phi - \inf_{\theta\in H_1} E_\theta\Phi_N = O_\varepsilon\left(\frac{1}{N^{1-\varepsilon}}\right) \tag{4} \]

for any \(\varepsilon>0\).

Here \(\Phi\) ranges over all tests of level \(\leq \alpha\).

Theorem 2. The Hotelling test \(\Phi_N: T^2 \geq T_0^2\) for testing \(H_0\) against the composite hypothesis \(H_1'\) is \(\varepsilon\)-minimax for any fixed level \(\alpha\in(0,1)\) and \(\delta>0\): under the conditions of Theorem 1, relation (1) holds with \(H_1\) replaced by \(H_1'\).

The proof of these theorems is based on the method of N. Giri, J. Kiefer, and C. Stein, developed in their paper \({}^{1}\). The central place in it is occupied by a lemma on the Giri—Kiefer—Stein integral equation (see \({}^{1}\), p. 1530). This equation has the form

\[ \int_{\Gamma_1} \exp\left\{ \gamma_1 \sum_{j=1}^{p}\tau_j \sum_{i>j}\beta_i \right\} \prod_{i=1}^{p} \Phi\left( \frac{N-i+1}{2},\,\frac{1}{2}\gamma_1\beta_i\tau_i \right) \times \lambda(\beta_1,\ldots,\beta_p)\,d\beta = \Phi\left(\frac{N}{2},\,\frac{p}{2},\,\gamma_1\right), \tag{5} \]

i.e. it is a Fredholm equation of the first kind.

Here \(\Gamma_1\) is a simplex; \(\beta_i\geq 0\), \(\sum_{i=1}^{p}\beta_i=1\); \(\gamma_1>0\) is a parameter; \(\tau_i\geq 0\), \(\sum_{i=1}^{p}\tau_i=1\), \(\tau_i\) are variables; \(\lambda(\beta_i,\ldots,\beta_p)\) is the desired function, \(d\beta\) is an element of Lebesgue measure on the simplex \(\Gamma_1\);

\[ \Phi(a,c,x)= \sum_{j=0}^{\infty} \frac{\Gamma(a+j)\Gamma(c)}{\Gamma(a)\Gamma(c+j)} \frac{x^j}{j!} \]

is the confluent hypergeometric function (E. Kummer’s series). The main lemma states:

Lemma. As \(N\to\infty\), \(\gamma_1=2\psi/N\) and \(0\leq\psi\leq 2(\ln N)^{1/3}\), the Giri—Kiefer—Stein equation (5) has the approximate solution

\[ \lambda_\infty(\beta_1,\ldots,\beta_p) = \frac{\Gamma(p/2)}{(\Gamma(1/2))^p} (\beta_1\beta_2\cdots\beta_p)^{-1/2}, \tag{6} \]

which is a probability density on the simplex \(\Gamma_1\). Namely, when (6) is substituted into the left-hand side of (5), one obtains a residual with the right-hand side of order \(O_\varepsilon(N^{-1+\varepsilon})\) for any \(\varepsilon>0\).

Further refinement of this method significantly improves this order and, at the same time, Theorem \(1'\) (formula (4)).

I express my gratitude to Prof. C. Stein (Stanford University), who drew my attention to this problem, and also to V. M. Kalinin, V. G. Maz’ya, and O. V. Shalaevskii for valuable scientific consultations.

Received
11 IV 1960

References

\({}^{1}\) N. Giri, J. Kiefer, C. Stein, Ann. Math. Stat., No. 34, 1524 (1963).
\({}^{2}\) Yu. V. Linnik, V. A. Plys, O. V. Shalaevskii, DAN, 168, No. 4 (1966).
\({}^{3}\) A. Wald, Trans. Am. Math. Soc., 54, No. 3, 426 (1943).