MATHEMATICS

A. L. ONISHCHIK

COMPLEX ENVELOPES OF COMPACT HOMOGENEOUS SPACES

(Presented by Academician P. S. Aleksandrov on 16 IX 1959)

Let \(\mathfrak K\) be a connected compact Lie group, \(\Omega\) its connected closed subgroup, and \(\mathfrak o(X)\) the algebra of spherical functions on the homogeneous space \(X=\mathfrak K/\Omega\). In the present note we study homomorphisms of the algebra \(\mathfrak o(X)\) into the field \(C\) of complex numbers. It is proved that these homomorphisms are in natural correspondence with the points of a certain complex manifold \(Z\), which contains \(X\) and on which the complex envelope of the group \(\mathfrak K\) acts transitively. The manifold \(Z\) can be realized as an algebraic submanifold of a complex affine space. The results are applied to the study of inclusions between transitive complex transformation groups and to the study of homogeneous Stein manifolds.

1. Let \(\mathfrak K\) be a connected compact Lie group, \(K\) its Lie algebra, \(G=K^C\), \(i:K\to G\) the natural embedding. Suppose that a connected complex Lie group \(\mathfrak G\) and a continuous monomorphism \(\alpha:\mathfrak K\to\mathfrak G\) are given, satisfying the following conditions: 1) the Lie algebra of the group \(\mathfrak G\) is \(G\), and \(\dot\alpha=i\); 2) \(\alpha(\mathfrak K)\) is a maximal compact subgroup of the group \(\mathfrak G\). Then we say that \(\mathfrak G\) is the complex envelope of the group \(\mathfrak K\).* If the group \(\mathfrak K\) is semisimple, then condition 2) follows from 1).

Lemma 1. Every connected compact Lie group \(\mathfrak K\) has a complex envelope. If \(\mathfrak G,\widetilde{\mathfrak G}\) are two complex envelopes of the group \(\mathfrak K\), and \(\alpha,\widetilde\alpha\) the corresponding monomorphisms, then there exists a complex isomorphism \(\beta:\mathfrak G\to\widetilde{\mathfrak G}\) such that \(\widetilde\alpha=\beta\alpha\).

It follows from this that the complex envelope \(\mathfrak G\) of the group \(\mathfrak K\) is unique. We shall identify \(\mathfrak K\) with the subgroup \(\alpha(\mathfrak K)\subset\mathfrak G\).

Lemma 2. Let \(\mathfrak K,\Omega\) be connected compact Lie groups; let \(\mathfrak G,\mathfrak H\) be their complex envelopes. Every continuous homomorphism \(\varphi:\Omega\to\mathfrak K\) extends (and in a unique way) to a complex-analytic homomorphism \(\psi:\mathfrak H\to\mathfrak G\). If the group \(\operatorname{Ker}\varphi\) is finite, then \(\operatorname{Ker}\psi=\operatorname{Ker}\varphi\).

Let, in particular, \(\varphi\) be a continuous unitary representation of the group \(\Omega\), i.e. a homomorphism of the group \(\Omega\) into \(U(n)\). Obviously, the complex envelope of the group \(U(n)\) is the full linear group \(GL(n,C)\). Therefore it follows from Lemma 2 that there exists a complex-analytic representation of the group \(\mathfrak H\) extending \(\varphi\).

Let \(\mathfrak G\) be a complex envelope, and let \(\Omega\) be a connected closed subgroup of the group \(\mathfrak K\). It follows from Lemma 2 that the complex envelope \(\mathfrak H\) of the group \(\Omega\) may be identified with a complex-analytic subgroup of the group \(\mathfrak G\). It is not difficult to show that \(\mathfrak H\) is closed in \(\mathfrak G\). Therefore the subgroup \(\mathfrak H\cap\mathfrak K\) is com-

* If \(R\) is a real vector space (or Lie algebra), then its complex envelope is denoted by \(R^C\).

** For another definition of the complex envelope, see (¹).

compact. Since it contains a maximal compact subgroup \(\Omega\) of the group \(\mathfrak H\), we have \(\mathfrak H\cap\mathfrak K=\Omega\). This thereby determines an embedding \(X\to Z\), where \(X=\mathfrak K/\Omega\), \(Z=\mathfrak G/\mathfrak H\). We shall say that the homogeneous space \(Z\) is a complex envelope of the homogeneous space \(X\), and shall identify \(X\) with a submanifold of \(Z\).

Let \(M=\mathfrak G/\mathfrak H\) be some homogeneous space, and let \(V\) be a finite-dimensional vector space over the field \(C\). A mapping \(\Phi:M\to V\) is called covariant if there exists a continuous representation \(\varphi\) of the group \(\mathfrak G\) in the space \(V\) such that
\[ \Phi(gx)=\varphi(g)\Phi(x)\qquad (x\in M,\ g\in\mathfrak G). \]
A covariant mapping is always analytic; and if \(\mathfrak G\), \(M\), and \(\varphi\) are complex-analytic, then \(\Phi\) is a complex-analytic mapping. A covariant mapping \(\Phi\) is called real if there exists a real subspace \(V_R\subset V\) such that \(V_R^C=V\) and \(\Phi(M)\subset V_R\).

Lemma 3. Let \(Z\) be the complex envelope of a compact homogeneous space \(X\). Then every covariant mapping \(\Phi:X\to V\) extends (and moreover uniquely) to a complex-analytic covariant mapping \(\Psi:Z\to V\). If \(\Phi\) is real and one-to-one, then \(\Psi\) is also one-to-one.

Let us note that every homogeneous space \(\mathfrak K/\Omega\), where \(\mathfrak K\) and \(\Omega\) are compact Lie groups, admits a real one-to-one covariant mapping into some space \(V\) \((^2)\).

1. Let \(M\) be a homogeneous space of the Lie group \(\mathfrak G\). Denote by \(F(M)\) the algebra over the field \(C\) of all continuous complex-valued functions on \(M\). To each element \(g\in\mathfrak G\) there naturally corresponds an automorphism \(g^*\) of the algebra \(F(M)\). For each \(f\in F(M)\), denote by \(R_f\) the linear subspace in \(F(M)\) spanned by functions of the form \(g^*f\) \((g\in\mathfrak G)\). A function \(f\) is called spherical if the space \(R_f\) is finite-dimensional. The set of all spherical functions is a subalgebra of the algebra \(F(M)\). If \(M=\mathfrak G\), then this subalgebra coincides with the algebra of representative functions on the group \(\mathfrak G\) \((^1)\).

Let \(Z=\mathfrak G/\mathfrak H\) be the complex envelope of the compact homogeneous space \(X=\mathfrak K/\Omega\). Denote by \(\mathfrak o(X)\) the algebra of spherical functions on \(X\), and by \(\mathfrak o(Z)\) the algebra of complex-analytic spherical functions on \(Z\). Let \(f\in\mathfrak o(X)\). Consider the representation \(\varphi\) of the group \(\mathfrak K\) in the space \(R_f\), acting by the formula \(\varphi(k)h=k^{*-1}h\) \((k\in\mathfrak K,\ h\in R_f)\), and denote by \(\psi\) the representation of the group \(\mathfrak G\) in \(R_f\) extending \(\varphi\). If \(o\in X\) is the point corresponding to the subgroup \(\Omega\), then set
\[ (Pf)(go)=(\psi(g^{-1})f)(o)\qquad (g\in\mathfrak G). \]
This defines a mapping \(P:\mathfrak o(X)\to\mathfrak o(Z)\).

Lemma 4. If \(i\) is the natural embedding of the space \(X\) into \(Z\), then \(i^*\) and \(P\) are mutually inverse isomorphisms of the algebras \(\mathfrak o(X)\) and \(\mathfrak o(Z)\).

Denote by \(\mathfrak M(X)\) the set of all homomorphisms \(\omega\) of the algebra \(\mathfrak o(X)\) into \(C\) satisfying the condition \(\omega(1)=1\). To each point \(z\in Z\) assign the mapping \(\omega_z:\mathfrak o(X)\to C\), defined by the formula
\[ \omega_z(f)=(Pf)(z)\qquad (f\in\mathfrak o(X)). \]
It follows from Lemma 4 that \(\omega_z\in\mathfrak M(X)\). Our main result consists in the following.

Theorem 1. The mapping \(\Omega:Z\to\mathfrak M(X)\), defined by the formula \(\Omega(z)=\omega_z\), is a one-to-one mapping onto.

Let us outline the proof of this theorem. First of all, in the special case \(\Omega=\{e\}\), Theorem 1 is easily derived from the results of Chapter VI of the book \((^3)\). In the general case, denote by \(p\) the natural projection of the group \(\mathfrak G\) onto \(Z\). It is easy to see that \(p^*(\mathfrak o(X))\subset\mathfrak o(\mathfrak K)\). Therefore there arises a mapping ...

\[ \bar p:\mathfrak M(\mathfrak K)\to\mathfrak M(X), \]
which assigns to each \(\omega\in\mathfrak M(\mathfrak K)\) the homomorphism \(\omega p^*\). It turns out that the diagram

\[ \begin{array}{ccc} \mathfrak G & \xrightarrow{\ \widetilde{\Omega}\ } & \mathfrak M(\mathfrak K)\\ p\downarrow & & \downarrow \bar p\\ Z & \xrightarrow{\ \Omega\ } & \mathfrak M(X) \end{array} \]

is commutative (by \(\widetilde{\Omega}\) is denoted the mapping analogous to the mapping \(\Omega\)). Since \(\widetilde{\Omega}\) is onto, in order to prove that \(\Omega\) is onto it is enough to show that \(\bar p\) is onto. The latter follows from the following lemma:

Lemma 5. Let \(A\) and \(B\) be commutative algebras with identities and with a finite number of generators over the field \(C\); let \(\alpha:A\to B\) be such a homomorphism that \(\alpha(1)=1\). Suppose that there exists a linear mapping \(\beta:B\to A\) such that \(\beta(1)=1\) and \(\beta(\alpha(x)\cdot y)=x\cdot\beta(y)\) \((x\in A,\ y\in B)\). Then every homomorphism \(\omega:A\to C\) can be represented in the form \(\omega=\widetilde{\omega}\alpha\), where \(\widetilde{\omega}\) is a homomorphism of the algebra \(B\) into \(C\).

Lemma 5 must be applied to the case \(A=\mathfrak o(X)\), \(B=\mathfrak o(\mathfrak K)\), \(\alpha=p^*\). We define the mapping \(\beta\) by the formula

\[ (\beta f)(ko)=\int_{\mathfrak L} f(kl)\,dl\qquad (f\in\mathfrak o(\mathfrak K)), \]

where integration is with respect to the invariant measure on \(\mathfrak L\), normalized by the condition

\[ \int_{\mathfrak L} dl=1. \]

The one-to-one character of the mapping \(\Omega\) follows from Lemma 3.

3. Let \(\mathfrak G\) be a group, and let \(\mathfrak G'\) and \(\mathfrak G''\) be its subgroups. We say that the triple \((\mathfrak G,\mathfrak G',\mathfrak G'')\) is a decomposition if every element \(g\in\mathfrak G\) can be represented in the form \(g=g'\cdot g''\), where \(g'\in\mathfrak G'\), \(g''\in\mathfrak G''\). The study of decompositions of Lie groups is equivalent to the study of inclusions between transitive Lie groups of transformations \((^4)\). Let \(\mathfrak K'\), \(\mathfrak K''\) be connected closed subgroups of a connected compact Lie group \(\mathfrak K\); let \(\mathfrak G'\), \(\mathfrak G''\), \(\mathfrak G\) be the complex envelopes of these groups. Then \(\mathfrak G'\) and \(\mathfrak G''\) are identified with subgroups of the group \(\mathfrak G\). The triple \((\mathfrak G,\mathfrak G',\mathfrak G'')\) will be called the complex envelope of the triple \((\mathfrak K,\mathfrak K',\mathfrak K'')\).

Theorem 2. The complex envelope of a decomposition is a decomposition.

Let a decomposition \((\mathfrak G,\mathfrak G',\mathfrak G'')\) be given, where \(\mathfrak G\) is a semisimple complex Lie group, and \(\mathfrak G'\), \(\mathfrak G''\) are its connected closed complex subgroups. Let \(\mathfrak G'_0\), \(\mathfrak G''_0\) be maximal semisimple subgroups of the groups \(\mathfrak G'\), \(\mathfrak G''\). Then the triple \((\mathfrak G,\mathfrak G'_0,\mathfrak G''_0)\) is also a decomposition. The groups \(\mathfrak G'\), \(\mathfrak G''\) may be replaced by conjugate subgroups in \(\mathfrak G\) in such a way that the decomposition \((\mathfrak G,\mathfrak G'_0,\mathfrak G''_0)\) is the complex envelope of some decomposition \((\mathfrak K,\mathfrak K',\mathfrak K'')\), where \(\mathfrak K,\mathfrak K',\mathfrak K''\) are semisimple compact Lie groups.

4. From Theorem 1 the following is derived.

Theorem 3. Let \(Z\) be the complex envelope of the compact homogeneous space \(X\), and let \(\Phi:X\to V\) be a one-to-one holomorphic covariant mapping. Then the corresponding covariant mapping \(\Psi\) maps \(Z\) one-to-one and biholomorphically onto a nonsingular algebraic subvariety of the space \(V\). If one introduces on \(Z\), by means of the mapping \(\Psi\), the structure of an algebraic variety, then the algebra of regular functions on \(Z\) will coincide with \(\mathfrak o(Z)\), and every complex-analytic covariant mapping of the space \(Z\) into a finite-dimensional vector space will be a rational mapping.

We say that a complex Lie group is admissible if it is the complex envelope of some compact group. Let \(\mathfrak H\) be an admissible complex-analytic subgroup of an admissible group \(\mathfrak G\). Then \(\mathfrak H\) is closed. It follows from Theorem 3 that the manifold \(\mathfrak G/\mathfrak H\) is a Stein manifold \((^5)\). It turns out that this result admits the following converse:

Theorem 4. Let \(\mathfrak G\) be an admissible complex Lie group, and let \(\mathfrak H\) be its closed complex-analytic subgroup. If \(\mathfrak G/\mathfrak H\) is a Stein manifold, then the connected component of the identity in the group \(\mathfrak H\) is admissible.

Moscow State University
named after M. V. Lomonosov

Received
14 IX 1959

REFERENCES

\(^1\) G. Hochschild, G. D. Mostow, Ann. Math., 66, 495 (1957).
\(^2\) G. D. Mostow, Ann. Math., 65, 432 (1957).
\(^3\) K. Chevalley, Theory of Lie Groups, 1, IL, 1948.
\(^4\) A. L. Onishchik, DAN, 129, No. 2 (1959).
\(^5\) A. Cartan, in: Fibered Spaces, IL, 1958, p. 352.