UDC 517.43

MATHEMATICS

Corresponding Member of the Academy of Sciences of the USSR I. M. GELFAND, M. I. GRAEV

COMPLEXES OF \(k\)-DIMENSIONAL PLANES IN THE SPACE \(C^n\) AND THE PLANCHEREL FORMULA FOR THE GROUP \(GL(n,C)\)

1. Let \(\bar H=H_{n,k}\) be the set of all \(k\)-dimensional planes in \(C^n\), endowed in the natural way with the structure of a complex analytic manifold. By a complex \(K\) of \(k\)-dimensional planes in \(C^n\) we shall mean any closed irreducible algebraic submanifold in \(H\), whose (complex) dimension is equal to \(n\), and such that through every point \(x\in C^n\) (with the possible exception of a manifold of points of lower dimension) there passes at least one plane \(h\in K\).*

If a complex \(K\) of \(k\)-dimensional planes in \(C^n\) is given, then the following problem of integral geometry may be formulated for it. To each function \(f(x)\in S\) (\(S\) is the space of infinitely differentiable and rapidly decreasing functions on \(C^n\)) assign its integrals \(\varphi(h)\) over the planes \(h\in K\). It is required to obtain an inversion formula, i.e., to recover the original function \(f(x)\) from the function \(\varphi(h)\).

In the paper a class of admissible complexes is defined, for which the inversion formula follows directly from the results of \((^1,^2)\). In Sec. 2, for the case \(k=1\) (i.e., for complexes of lines), a simple geometric description of admissible complexes in general position is obtained.

In Sec. 3 an example is considered of a complex of \(n(n-1)/2\)-dimensional planes in \(C^{n^2}\), arising in a natural way in the representation theory of the group \(GL(n,C)\). It is proved that this complex is admissible, and an inversion formula is obtained for it. This inversion formula was previously proved by other methods in \((^3,^4)\). The method by which it is obtained in the present paper is, in the authors’ opinion, the simplest and most natural.

We shall briefly present the results of the paper \((^2)\). We shall specify any \(k\)-dimensional plane \(h\) in \(C^n\) by the equation \(x=\alpha t+\beta\), where \(\alpha\) is a matrix whose columns are \(k\) linearly independent vectors \(\alpha_1,\ldots,\alpha_k\in C^n\), \(\beta\in C^n\), and \(t\) ranges over \(C^k\). Next fix an infinitely differentiable function \(g(\alpha)\), not vanishing anywhere and satisfying, for any nondegenerate linear transformation \(\mu\) in \(C^k\), the following condition:
\(g(\alpha\mu)=g(\alpha)|\det\mu|^2\). To each function \(f(x)\in S\) we associate a function \(\varphi(h)\) on \(H_{n,k}\) by the formula

\[ \varphi(h)=g(\alpha)\int f(\alpha t+\beta)\,dt\,d\bar t. \tag{1} \]

(It is easy to verify that the integral does not depend on the parametric specification of the plane \(h\).)

In the paper \((^2)\), for each point \(\beta\in C^n\) a linear mapping was defined

\[ \varkappa_\beta:\Phi_H\to \Phi_{G_\beta}^{(k,k)} \]

from the space \(\Phi_H\) of all infinitely differentiable functions on \(H_{n,k}\) into the space \(\Phi_{G_\beta}^{(k,k)}\) of differential forms of type \((k,k)\) defined on the submanifold \(G_\beta\) of \(k\)-dimensional planes passing through the point \(\beta\). The following inversion formula for the integral was established

\[ \text{* For simplicity, manifolds without singularities are considered, although all results remain valid also for manifolds with singularities.} \]

of the transform (1):

\[ \int_{\gamma} \chi_{\beta}\varphi=c_{\gamma}f(\beta), \tag{2} \]

where the integral is taken over an arbitrary cycle \(\gamma\subset G_{\beta}\) of real dimension \(2k\), and the factor \(c_{\gamma}\) depends only on the homology class to which the cycle \(\gamma\) belongs.

Definition. Let \(K\) be a complex analytic submanifold of \(H_{n,k}\) of dimension \(\geqslant n\) (in particular, a complex), and let

\[ i^{*}:\Phi_H\to\Phi_K,\qquad \Phi_{G_\beta}^{(k,k)}\to\Phi_{G_\beta\cap K}^{(k,k)} \]

be the natural mappings of functions and differential forms. We shall call the submanifold \(K\) admissible if, for almost every \(\beta\in C^n\), there exists a linear mapping
\[ \chi'_{\beta}:\Phi_K\to\Phi_{G_\beta\cap K}^{(k,k)} \]
such that the diagram

\[ \begin{array}{ccc} \Phi_H & \xrightarrow{\ \chi_\beta\ } & \Phi_{G_\beta}^{(k,k)}\\ {\scriptstyle i^*}\downarrow & & \downarrow{\scriptstyle i^*}\\ \Phi_K & \xrightarrow{\ \chi'_\beta\ } & \Phi_{G_\beta\cap K}^{(k,k)} \end{array} \]

is commutative. (It is clear that in this case \(\chi'_{\beta}\) is uniquely determined and is, just like \(\chi_\beta\), a differential operator.) If \(K\) is an admissible submanifold, then, by virtue of (2), the following inversion formula for (1), in which \(h\) ranges over \(K\), is valid:

\[ \int_{\gamma}\chi'_{\beta}\varphi=c_{\gamma}f(\beta), \]

where \(\gamma\) is an arbitrary cycle of real dimension \(2k\) in \(G_{\beta}\cap K\). In particular, if \(K\) is a complex, then for points \(\beta\in C^n\) in general position \(\dim(G_{\beta}\cap K)=2k\), and therefore one may take \(\gamma=G_{\beta}\cap K\). (Of course, this formula determines \(f(\beta)\) only if \(c_{\gamma}\ne 0\).)

2. Admissible complexes of lines in \(C^n\)

Theorem 1. A complex of lines \(K\) is admissible if and only if, for almost every line \(h\in K\), the following condition holds: if one removes from the line \(h\) some finite number of points and takes the submanifold of lines in \(K\) that meet \(h\) at the remaining points, then the closure in \(K\) of this submanifold is nonsingular at the point \(h\in K\).

In what follows we shall assume that \(C^n\) is embedded in the projective space \(CP^n\), and thereby each line \(h\) is completed by a point at infinity. Let \(K\) be a complex of lines, \(h\in K\), and \(x\in C^n\) a point on the line \(h\). Consider the tangent space \(\tau_H\) at \(h\) to the manifold \(H\) of all lines, and take in it two subspaces—the tangent space \(\tau_K\) to the complex \(K\) and the tangent space \(\tau_{G_x}\) to the manifold \(G_x\) of all lines passing through \(x\). Since \(\dim\tau_H=2n-2\), \(\dim\tau_K=n\), and \(\dim\tau_{G_x}=n-1\), in general position we have:
\[ \dim(\tau_K\cap\tau_{G_x})=1. \]
We shall call a point \(x\) of the line \(h\in K\) a critical point if
\[ \dim(\tau_K\cap\tau_{G_x})>1. \]

Let us find all critical points of the line \(h\in K\). Suppose, for definiteness, that \(h\) is not parallel to the hyperplane \(x^n=0\). Then every line close to \(h\) can be given by a system of equations
\[ x^i=a^i x^n+\beta^i x^0,\quad i=1,\ldots,n-1, \]
where \((x^0,\ldots,x^n)\) are homogeneous coordinates in \(CP^n\). We take the \(2n-2\) complex numbers \(a^i,\beta^i\) as local coordinates on the manifold of all lines in a neighborhood of \(h\), and let the complex \(K\) be defined in a neighborhood of \(h\) by the equations
\[ u^1(a,\beta)=0,\ldots,u^{n-2}(a,\beta)=0. \]
Then, in order that the point \(x=(x^0,\ldots,x^n)\in h\) be critical, it is necessary and sufficient that the rank of the matrix
\[ \left\|x^0\partial u^i/\partial a^j|_h-x^n\partial u^i/\partial\beta^j|_h\right\|_{i=1,\ldots,n-2;\ j=1,\ldots,n-1} \]
be less than \(n-2\). Let \(\Delta_i(x^0,x^n)\) be the minors of order \(n-2\) of this matrix. It can be shown that, in the case of an admissible complex, they dis-

differ by factors that do not depend on \(x\). Thus, if \(\Delta_i \not\equiv 0\) for at least one \(i\), then (since \(\Delta_i\) is a homogeneous polynomial of degree \(n-2\) in \(x^0,x^n\)) on the projective line \(h\) there are, counting multiplicities, \(n-2\) critical points. If, however, \(\Delta_i \equiv 0\) for all \(i\), then all points of the line \(h\) are critical (such lines form in \(K\) a submanifold of lower dimension).

Theorem 2. The critical points of the lines of an admissible complex form in \(\mathbf{CP}^n\) a manifold of dimension less than \(n\).

We shall call an admissible complex of lines \(K\) a complex in general position if on each line \(h \in K\) (with the possible exception of a submanifold of lines of lower dimension) there are exactly \(n-2\) pairwise distinct critical points. We describe the local structure of such complexes.

Theorem 3. Suppose that on a line \(h_0\) of an admissible complex \(K\) there are exactly \(n-2\) pairwise distinct critical points \(x_1,\ldots,x_{n-2}\). Then the critical points of the lines \(h \in K\) close to \(h_0\) describe \(n-2\) local algebraic surfaces \(M_1,\ldots,M_{n-2}\), each of which has dimension \(n-1\) or \(n-2\); moreover, if \(\dim M_i=n-1\), then the lines \(h\) are tangent to \(M_i\).

Assume further that the points \(x_i \in M_i\) are nonsingular, and let \(P_i\) be the hyperplane in \(\mathbf{CP}^n\) spanned by the tangent plane to \(M_i\) at \(x_i\) and the line \(h_0\) \((i=1,\ldots,n-2)\). Then, if the hyperplanes \(P_i\) are in general position, the set of all lines in \(\mathbf{C}^n\) tangent to each local surface \(M_i\) of dimension \(n-1\) and intersecting each local surface \(M_i\) of dimension \(n-2\) is contained in \(K\) and forms in \(K\) a neighborhood of the line \(h_0\). Conversely, if almost every line \(h_0\) of the complex \(K\) has a neighborhood of the indicated form, then \(K\) is an admissible complex in general position.

Theorem 4. For \(n=3\), every admissible complex of lines is either the complex of lines tangent to some two-dimensional algebraic submanifold, or the complex of lines intersecting some algebraic curve.

3. Plancherel theorem for the group \(GL(n,C)\).

Here we shall consider the complex of planes that arises in the derivation of the Plancherel formula on the group \(GL(n,C)\). We shall treat matrices \(x \in GL(n,C)\) as points of the space \(C^{n^2}\). Consider in \(GL(n,C)\) the family \(K\) of lower triangular unipotent matrices. It is obvious that these surfaces are surfaces given by parametric equations \(x=x_1^{-1}zx_2\), where \(x_1,x_2\) are fixed matrices, and the “parameter” \(z\) runs through lower triangular \(n(n-1)/2\)-dimensional planes in \(C^{n^2}\). It is easy to verify that the manifold \(K\) of these planes is a complex. We shall prove here that the complex \(K\) is admissible, and at the same time compute the corresponding differential form \(\varkappa_\beta\varphi\). In view of the homogeneity of the space \(GL(n,C)\), it suffices to consider the case \(\beta=e\), where \(e\) is the identity matrix.

Introduce the notation: \(x_+\) is the matrix obtained from \(x\) by replacing by zeros all elements \(x_{pq}\) lying below the main diagonal; \(x_- = x-x_+\). Any \(n(n-1)/2\)-dimensional plane in \(C^{n^2}\) (with the exception of a submanifold of planes of lower dimension) can be given by the following equation:

\[ x_+ = \sum_{p>q} a^{pq}x_{pq}+\beta, \tag{3} \]

where \(a^{pq},\beta\) are upper triangular matrices. Take the elements of the matrices \(a^{pq},\beta\) as local coordinates on the manifold of all \(n(n-1)/2\)-dimensional planes in \(C^{n^2}\). In these coordinates the form \(\varkappa_e\varphi\), defined in (2), has the form

\[ \varkappa_e\varphi = \bigwedge_{p>q} \left(\sum_{i\leq j} D_{ij}\,da^{pq}_{ij}\right) \wedge \bigwedge_{p>q} \left(\sum_{i\leq j} \overline{D}_{ij}\,d\overline{a}^{pq}_{ij}\right) \varphi\big|_{\beta=e} \]

\[ = \bigwedge_{p>q} \operatorname{tr}(D' d\alpha^{pq}) \wedge \bigwedge_{p>q} \operatorname{tr}(\bar D' d\bar\alpha^{pq})\,\varphi\big|_{\beta=e}, \]

where \(D=\|D_{ij}\|\), \(\bar D=\|\bar D_{ij}\|\); \(D_{ij}=\partial/\partial \beta_{ij}\), \(\bar D_{ij}=\partial/\partial \bar\beta_{ij}\) for \(i\le j\); \(D_{ij}=\bar D_{ij}=0\) for \(i>j\); \(D'\) denotes the transposed matrix. This form is defined on the manifold of planes passing through \(e\).

Let us find the restriction \(i^*\chi_e\varphi\) of the form \(\chi_e\varphi\) to the submanifold of planes passing through \(e\) and belonging to the complex \(K\). These planes (with the exception of a submanifold of planes of lower dimension) are given by the parametric equations \(x=\zeta^{-1}z\xi\), in which the “parameter” is the matrix \(z\), and \(\xi\) is an upper triangular unipotent matrix defining the plane. We reduce the equations of the planes to the form (3). Eliminating the “parameter” \(z\) from the equations, we obtain
\[ x_+ = \zeta^{-1}(\zeta x_-\xi^{-1})_+\xi + e. \]
Thus
\[ \alpha^{pq}=\zeta^{-1}(\zeta e^{pq}\zeta^{-1})_+\zeta, \]
where \(e^{pq}\) is the matrix whose entry at the intersection of the \(p\)-th row and the \(q\)-th column is \(1\), and whose remaining entries are zeros. Introduce the notation \(\zeta'^{-1}D\zeta'=F\), \(d\zeta\cdot\zeta^{-1}=u\). Then after elementary transformations we obtain:
\[ \bigwedge_{p>q}\operatorname{tr}(D'd\alpha^{pq}) = \bigwedge_{q<p}\left(\zeta^{-1}(F_+u'-u'F_+)_-\zeta\right)_{qp} = \bigwedge_{q<p}(uF'_+-F'_+u)_{qp}\, * . \]
Since
\[ (uF'_+-F'_+u)_{qp} = (F_{pp}-F_{qq})u_{qp} + \sum_{i>p} F_{pi}u_{qi} - \sum_{i<q} F_{iq}u_{ip}, \]
it is easy to verify from this that
\[ \bigwedge_{q<p}(uF'_+-F'_+u)_{qp} = \prod_{q<p}(F_{pp}-F_{qq})\cdot \bigwedge_{q<p}u_{qp} = \prod_{q<p}(F_{pp}-F_{qq})\cdot \bigwedge_{q<p}d\zeta_{qp}. \]
As a result we obtain
\[ i^*\chi_e\varphi = \prod_{q<p}(F_{pp}-F_{qq})(\bar F_{pp}-\bar F_{qq})\varphi\big|_{\beta=e}\cdot \bigwedge_{q<p}d\zeta_{qp}\wedge \bigwedge_{q<p}d\bar\zeta_{qp}. \tag{4} \]

It remains to verify that the operators \(F_{pp}\) belong to the tangent space to the complex \(K\), and consequently the right-hand side of equality (4) depends only on \(i^*\varphi\). For this, consider the manifold of planes of the complex \(K\) defined by the parametric equations \(x=\zeta^{-1}z\delta\zeta\), where \(\delta\) is a diagonal matrix with diagonal entries \(\delta_1,\ldots,\delta_n\). In the form (3), the equations of these planes have the form
\[ x_+=\zeta^{-1}(\zeta x_-\zeta^{-1})_+\zeta+\zeta^{-1}\delta\zeta, \]
whence \(\beta=\zeta^{-1}\delta\zeta\). Therefore,
\[ \frac{\partial}{\partial \delta_p} = \sum_{i,j}(\zeta^{-1})_{ip}\zeta_{pj}\frac{\partial}{\partial\beta_{ij}}, \]
i.e.
\[ \partial/\partial\delta_p=F_{pp}. \]
Thus, finally, we obtain
\[ i^*\chi_e\varphi = \prod_{q<p}\left(\frac{\partial}{\partial\delta_p}-\frac{\partial}{\partial\delta_q}\right) \left(\frac{\partial}{\partial\bar\delta_p}-\frac{\partial}{\partial\bar\delta_q}\right) (i^*\varphi)\big|_{\delta=e}\cdot \bigwedge_{q<p}d\zeta_{qp}\wedge \bigwedge_{q<p}d\bar\zeta_{qp}. \]

This form coincides with the differential form obtained in \((^3,{}^4)\). To find the coefficient \(c_\gamma\) in the inversion formula, one must, according to \((^2)\), take a basis \(\gamma_1,\ldots,\gamma_s\) in the homology group \(H_{2k}(G_e)\), \(k=n(n-1)/2\), consisting of Schubert submanifolds, where \(\gamma_1\) is the Euler cycle (i.e. the manifold of planes lying in a fixed \((k+1)\)-dimensional plane). Let \(\gamma=G_e\cap K=\sum a_i\gamma_i\); then \(c_\gamma=(2i)^k\pi^{2k}a_1\). By simple arguments (for example, computing the intersection index) one can show that \(a_1=n!\).

The authors express their gratitude to I. N. Bernstein, who read the manuscript and made a number of useful comments.

Received
26 XII 1967

References

1. I. M. Gel'fand, M. I. Graev, Z. Ya. Shapiro, DAN, 168, No. 6, 1236 (1966).
2. I. M. Gel'fand, M. I. Graev, Z. Ya. Shapiro, Functional Analysis and Its Applications, 1, No. 1, 15 (1967).
3. I. M. Gel'fand, M. A. Naimark, Tr. Mat. Inst. im. V. A. Steklova, 36, 3 (1950).
4. I. M. Gel'fand, M. I. Graev, Tr. Moskov. Matem. Obshch., 4, 375 (1955).

* By \(x_-'\) (\(x_+'\)) is denoted the matrix transposed to \(x_-\) (respectively, \(x_+\)).