MATHEMATICS

A. M. VASIL'EV

ON ORTHOGONAL SUBGROUPS OF CLASSICAL COMPACT LIE GROUPS

(Presented by Academician P. S. Aleksandrov, 25 II 1958)

1. É. Cartan introduced into the space of a Lie group a quadratic differential form invariant with respect to left and right translations. For semisimple compact groups this form is positive definite. In the present note we set forth conditions for orthogonality, in the sense of the Cartan metric, of certain subgroups of a simple compact Lie group \(G\). Knowledge of these conditions is important for certain problems in the geometry of homogeneous spaces, in particular for determining totally geodesic submanifolds of a homogeneous space in the sense of the Riemannian geometry induced on it by the Cartan metric of its group of motions \(G\) (cf. \((^1)\)).

2. Let us specify the basic concepts. Two linear subspaces \(E, E'\) of a Euclidean \(N\)-dimensional vector space \(E(N)\) intersect orthogonally if the orthogonal complement of their intersection \(E \cap E'\) in \(E\) is completely orthogonal to \(E'\), and, consequently, the orthogonal complement of \(E \cap E'\) in \(E'\) is completely orthogonal to \(E\). In particular, if \(E \subset E'\), we shall also regard them as intersecting orthogonally. Two submanifolds of a Riemannian space intersect orthogonally at a point \(M\) if their tangent spaces at \(M\) are orthogonal. Orthogonality of subgroups \(G_1, G_2\) of a group \(G\) is understood in this sense at their intersection at the identity of the group.

3. The following proposition follows from Cartan’s theory of symmetric spaces \((^2)\). Let \(G_1\) consist of all elements of \(G\) invariant with respect to an involutive automorphism \(I\) of it (\(I^2\) is the identity automorphism). \(G_2\) is orthogonal to \(G_1\) if and only if some involutive automorphism of the group \(G_2\) extends to an automorphism \(I\) of the group \(G\). Here, among the automorphisms of \(G_2\) one must include the identity automorphism and the mapping \(g \to g^{-1}\).

4. In what follows, \(O(n)\), \(U(n)\), and \(Sp(2n)\) denote, respectively, the group of all real orthogonal matrices of order \(n\), the group of all complex unitary matrices of order \(n\), and the group of all complex unitary-symplectic matrices of order \(2n\). By \(\widetilde O(n)\), \(\widetilde U(2n)\), \(\widetilde{Sp}(4n)\) we denote the simplest real representations of these groups in the spaces \(E(n)\), \(\widetilde E(2n)\), \(\widetilde E(4n)\), respectively. We shall call them \(\widetilde R\)-representations. It is known \((^3)\) that \(\widetilde U(2n)\) may be regarded as the set of all orthogonal transformations of \(\overline E(2n)\) preserving a certain decomposition of \(\overline E(2n)\) into two-dimensional subspaces (a \(C\)-structure in \(\overline E(2n)\)). Similarly, \(Sp(4n)\) for \(n>1\) may be specified by a certain decomposition of \(\widetilde E(4n)\) into four-dimensional subspaces (a \(q\)-structure). \(\widetilde{Sp}(4)\) in \(\widetilde E(4)\) may be regarded as the set of orthogonal transformations inducing, in the improper hyperplane, the group of left (or right) Clifford translations \((^4)\). In this sense one may speak of opposite (left—right) \(q\)-structures in \(\widetilde E(4)\).

A subspace \(E\) of the space \(\bar E(2n)\) shall be called a \(C\)-subspace if every plane of the \(C\)-structure having a common ray with it belongs to it entirely. Thus a \(C\)-structure is also defined in \(E\). In an analogous way one defines \(q\)-subspaces in \(\widetilde E(4n)\), on each of which a definite \(q\)-structure is induced. A subspace \(E'\) in \(\bar E(2n)\) (in \(\widetilde E(4n)\)) shall be called an \(O\)-subspace if every \(E(2)\) of the \(C\)-structure (every \(E(4)\) of the \(q\)-structure) has with it either a 0-dimensional or a one-dimensional intersection, and in the latter case this intersection is orthogonal to \(E'\). A subspace \(\bar E\) in \(\widetilde E(4n)\) shall be called a \(C\)-subspace if every \(E(4)\) of the \(q\)-structure has with it either a 0-dimensional or a 2-dimensional intersection, and in the latter case \(E(4)\) is orthogonal to \(\bar E\). These two-dimensional intersections define a \(C\)-structure in \(\bar E\).

1. By a subgroup of type \(\rho\) of the group \(\widetilde O(N)\) we shall mean a subgroup \(G_1\)—a direct product of groups \(O(l_a)\), \(U(m_k)\), \(Sp(2n_r)\), acting independently of one another [as \(\widetilde R\)-representations in a system of mutually completely orthogonal subspaces \(E(l_a)\), \(\bar E(2m_k)\), \(\widetilde E(4n_r)\) (briefly, \(E_a,\bar E_k,\widetilde E_r\)) of the space \(E(N)\) and leaving fixed all points of a maximal subspace \(E(N_0)\), completely orthogonal to all \(E_a,\bar E_k,\widetilde E_r\). Let a subgroup \(G_2\) of type \(\rho\) in the same sense be associated with a system of subspaces \({}'E(l_{a'})\), \({}'\bar E(2m_{k'})\), \({}'\widetilde E(4n_{r'})\), \({}'E(N_0')\) (briefly, \({}'E_{a'},{}'\bar E_{k'},{}'\widetilde E_{r'},{}'E(N_0')\)).

The following theorem is proved with the aid of the theory of linear representations of groups and associative algebras.

Theorem. The intersection \(G_1\cap G_2\) of two subgroups of type \(\rho\) of the group \(\widetilde O(N)\) is a direct product of a certain number of groups \(O(l_\alpha)\), \(U(m_\varkappa)\), \(Sp(2n_\rho)\), acting independently of one another in a completely orthogonal system of subspaces \(E(l_\alpha\lambda_\alpha)\), \(\bar E(2m_\varkappa\mu_\varkappa)\), \(\widetilde E(4n_\rho\nu_\rho)\) (briefly, \(E_\alpha,\bar E_\varkappa,\widetilde E_\rho\)) as, respectively, \(\lambda_\alpha\)-, \(\mu_\varkappa\)-, \(\nu_\rho\)-fold \(\widetilde R\)-representations of these groups and leaving fixed all points of a maximal subspace \(E(N_{00})\) (briefly, \(E_0\)), completely orthogonal to all \(E_\alpha,\bar E_\varkappa,\widetilde E_\rho\). The commutator \([G_1\cap G_2]\) (the totality of orthogonal transformations of \(E(N)\) commuting with every transformation from \(G_1\cap G_2\)) is a direct product of groups \(O(\lambda_\alpha)\), \(U(\mu_\varkappa)\), \(Sp(2\nu_\rho)\), \(O(N_{00})\), acting independently of one another in the same system of completely orthogonal subspaces as, respectively, \(l_\alpha\)-, \(m_\varkappa\)-, \(n_\rho\)-, 1-fold \(\widetilde R\)-representations.

In \(E(N)\) there exist two orthonormal bases \(B_1\) and \(B_2\) such that: a) every vector of \(B_1\) belongs to the intersection of one of the subspaces \(E_a,\bar E_k,\widetilde E_r,E(N_0)\) with one of the subspaces \(E_\alpha,\bar E_\varkappa,\widetilde E_\rho,E_0\); b) \(B_2\) is associated in an analogous way with the subspaces invariant with respect to \(G_2\); c) the passage from \(B_1\) to \(B_2\) is accomplished by means of a transformation from \([G_1\cap G_2]\).

Each of the intersections \(\bar E_k\) (or \({}'\bar E_{k'}\)) with \(E_\alpha,\bar E_\varkappa,\widetilde E_\rho,E_0\) is a \(C\)-subspace in \(\bar E_k\) (or \({}'\bar E_{k'}\)). The intersections \(\widetilde E_r\) and \({}'\widetilde E_{r'}\) with the same spaces are \(q\)-subspaces in \(\widetilde E_r\) or \({}'\widetilde E_{r'}\).

1. Below we give some known propositions of the geometry of a Euclidean vector space \(E(N)\) (see, for example, \((^4)\), Ch. I). Let two subspaces \(E,E'\) be given. Let the angle between the lines \(e_1\subset E\), \(e_1'\subset E'\) take, as a function of \(e_1,e_1'\), stationary values \(\varphi_\xi\leq 90^\circ\). To each \(\varphi_\xi\) correspond stationary subspaces \(E_\xi\subset E\), \(E_\xi'\subset E'\), and the \(E_\xi\) form a complete system of mutually completely orthogonal subspaces in \(E\), and the \(E_\xi'\)—in \(E'\). If \(\varphi_\xi\ne 90^\circ\), then a natural one-to-one isometric correspondence is established between the points of \(E_\xi\) and \(E_\xi'\). Subspaces \(E\) and \(E'\) having only one stationary angle \(\varphi\),

are called (for \(\varphi\ne 90^\circ\)) paratactic. Subspaces \(E,E'\) having no more than two stationary angles \(\varphi_1,\varphi_2\), with \(\varphi_2=90^\circ\), and whose stationary subspaces \(E_1,E'_1\) corresponding to \(\varphi_1\) are of dimension at most one, will be called almost orthogonal.

We shall consider \(l\)-dimensional subspaces \(E(l)\), \(2l\)-dimensional subspaces \(\bar E(2l)\), each of which carries a definite \(C\)-structure, and \(4l\)-dimensional subspaces \(\widetilde E(4l)\), each of which carries a definite \(q\)-structure. We shall say that a pair of subspaces from this set is in \(O\)-position if they have only two stationary angles \(\varphi_1\ne 90^\circ\) and \(\varphi_2=90^\circ\), and the stationary subspaces corresponding to \(\varphi_1\) are \(l\)-dimensional and are \(O\)-subspaces (see § 4) in the \(C\)- or \(q\)-structure of the subspaces forming the pair. For example, \(E(l)\) and \(\bar E(2l)\), \(\bar E(2l)\) and \(\widetilde E(4l)\), \(\widetilde E(4l)\) and \({}'\widetilde E(4l)\), etc., may be in \(O\)-position. The paratacticity of \(E(l)\) and \({}'E(l)\) will be regarded as a special case of \(O\)-position.

Consider subspaces \(\bar E(2m)\), carrying \(C\)-structures, and \(\widetilde E(4m)\), carrying \(q\)-structures. We shall say that two subspaces from this set are in \(C\)-position if they have either only one \(\varphi_1\ne 90^\circ\), or only two stationary angles \(\varphi_1\ne 90^\circ,\ \varphi_2=90^\circ\), and the stationary subspaces corresponding to \(\varphi_1\) are \(2m\)-dimensional and are \(C\)-subspaces whose \(C\)-structures naturally correspond to one another.

Let \(\widetilde E(4l)\) carry a \(q\)-structure and let \(E(l)\) be its \(O\)-subspace. The symmetry in \(\widetilde E(4l)\) with respect to \(E(l)\) takes its \(q\)-structure into some other \(q\)-structure. We shall agree to say that two subspaces \(\widetilde E(4l)\) and \({}'\widetilde E(4l)\), carrying \(q\)-structures, are in \(O'\)-position if: a) they are paratactic; b) in the natural correspondence between these subspaces, the \(q\)-structure of \({}'\widetilde E(4l)\) corresponds in \(\widetilde E(4l)\) to the \(q\)-structure obtained from its original \(q\)-structure by means of the symmetry with respect to some \(l\)-dimensional \(O\)-subspace.

7. Below are listed all necessary and sufficient conditions for the orthogonality of two subgroups of type \(\rho\).

A. The dimension of the intersection of the space \(E_\alpha\) with the spaces \(E_a,\ {}'E_{a'}\) must be either \(0\) or \(l_\alpha\), with \(\bar E_k,\ {}'\bar E_{k'}\) either \(0\) or \(2l_\alpha\), and with \(\widetilde E_r,\ {}'\widetilde E_{r'}\) either \(0\) or \(4l_\alpha\). We denote the nonzero intersections of \(E_\chi\) with \(E_a\) by \(E_{\alpha a}\), with \({}'E_{\alpha'}\) by \({}'E_{\alpha\alpha'}\), with \(\bar E_k\) by \(\bar E_{\alpha k}\), etc. Each of the spaces \(E_{\alpha a},\bar E_{\alpha k},\widetilde E_{\alpha r}\) must be in \(O\)-position with each \({}'E_{\alpha a'},{}'E_{\alpha k'}\), and each \(E_{\alpha a},\bar E_{\alpha k}\) with each \({}'\widetilde E_{\alpha r'}\). The spaces \(\widetilde E_{\alpha r}\) and \({}'\widetilde E_{\chi r'}\) are either in \(O\)-position or in \(O'\)-position (see § 6). For \(\alpha=\mathrm{const}\), the stationary angle between any two \(E_{\alpha a},{}'E_{\alpha a'}\) is one and the same and is equal to \(x_\alpha\); the stationary angle, distinct from \(90^\circ\), between any two \(E_{\alpha a},{}'\bar E_{\alpha k'}\) or \({}'E_{\alpha a'},\bar E_{\alpha k}\) is one and the same and is equal to \(u_\chi\); the same is true for any \(E_{\alpha a},{}'\widetilde E_{\alpha r'}\) and \({}'E_{\alpha a'},\widetilde E_{\alpha r}\)—their angle is \(v_\alpha\), for any \(\bar E_{\alpha k},{}'\widetilde E_{\alpha r'}\) and \({}'\bar E_{\alpha k'},\widetilde E_{\alpha r}\)—the angle is \(w_\alpha\), and for \(\widetilde E_{\alpha r}\) and \({}'\widetilde E_{\alpha r'}\) their angle is, in the case of \(O\)-position, \(z_\alpha\), and in the case of \(O'\)-position, \(z_\alpha'\). The relations must hold
\[ \cos z_\alpha=\sqrt{2}\cos w_\alpha=2\cos z_\alpha'=2\cos v_\alpha=2\cos y_\alpha=2\sqrt{2}\cos u_\alpha=4\cos x_\alpha . \]

The space \(\bar E_\chi\) may have nonzero intersections only with \(\bar E_k,{}'\bar E_{k'}\)—of dimensions \(2m_\chi\), and with \(\widetilde E_r,{}'\widetilde E_{r'}\)—of dimensions \(4m_\chi\). Denote these intersections by \(\bar E_{\chi k},{}'\bar E_{\chi k'},\widetilde E_{\chi r},{}'\widetilde E_{\chi r'}\). Each \(\bar E_{\chi k},\widetilde E_{\chi r}\) must be in \(C\)-position (see § 6) with each \({}'\bar E_{\chi k'},{}'\widetilde E_{\chi r'}\). For \(\chi=\mathrm{const}\), the stationary angle between \(\bar E_{\chi k},{}'\bar E_{\chi k'}\) is one and the same and is equal to \(y_\chi\); distinct from \(90^\circ\)

stationary angle between \(\bar E_{\chi k}\), \({}'\bar E_{\chi r'}\) or \({}'\bar E_{\chi k'}\), \(\hat E_{\chi r}\) is one and the same, equal to \(\omega_\chi\); the same is true for \(\tilde E_{\chi r}\), \({}'\bar E_{\chi r'}\)—their angle is \(z_\chi\). The relations \(\cos z_\chi=\sqrt2\cos\omega_\chi=2\cos y_\chi\) must be satisfied. Each \(\tilde E_\rho\) can have nonzero intersections only with \(\tilde E_r\), \({}'\tilde E_{r'}\) of dimensions \(4n\) (denote them by \(\hat E_{\rho r}\), \({}'\bar E_{\rho r'}\)). For \(\rho=\mathrm{const}\) each pair \(\hat E_{\rho r}\), \({}'\bar E_{\rho r'}\) is paratactic, their stationary angles are identical, and the \(q\)-structures of these subspaces, by virtue of paratacticity, correspond to one another.

B. No pair of subspaces, one of which belongs to the series \(E_a,\ \bar E_k,\ \tilde E_r\), and the other to the series \({}'E_{a'},\ {}'\bar E_{k'},\ {}'\tilde E_{r'}\), can simultaneously have nonzero intersections with two subspaces from the series \(E_\alpha,\ \bar E_\chi,\ \tilde E_\rho\). If the indicated pair of subspaces has nonzero intersections with one of the subspaces of the series \(E_\alpha,\ \bar E_\chi,\ \tilde E_\rho\), it must intersect \(E_0\) in a pair of subspaces completely orthogonal to each other.

C. Denote by \(E_{0a},\ {}'E_{0a'},\ldots,\ {}'\tilde E_{0r'}\) the intersections of the subspaces \(E_a,\ {}'E_{a'},\ldots,\ {}'E_{r'}\) with \(E_0\). Each subspace \(E_{0a},\ \bar E_{0k},\ \tilde E_{0r}\) must be almost orthogonal to each \({}'E_{0a'},\ {}'\bar E_{0k'}\), and each \(E_{0a},\ \bar E_{0k}\)—to each \({}'\tilde E_{0r'}\). The subspaces \(\tilde E_{0r},\ {}'\tilde E_{0r'}\) must either be almost orthogonal, or have two stationary angles \(\varphi_1\ne90^\circ,\ \varphi_2=90^\circ\), with the stationary subspaces corresponding to \(\varphi_1\) being 4-dimensional \(q\)-subspaces in \(\tilde E_{0r},\ {}'\tilde E_{0r'}\). By virtue of the correspondence between these paratactic subspaces, their \(q\)-structures must be opposite (see item 4).

1. All configurations of the subspaces and of their \(C\)- and \(q\)-structures that occur in item 8 are in fact realized for arbitrary \(l_a,\ m_\chi,\ n_\rho,\ N_{00}\) and for \(\lambda_\alpha,\ \mu_\chi,\ \nu_\rho\) equal to powers of the number 2 (including, in some cases, the zeroth power).

2. If two subgroups of type \(\rho\) belong to some subgroup of the group \(\tilde O(N)\), for example \(\tilde U(N)\) or \(\tilde{Sp}(N)\), the formulation of the conditions for their orthogonality is greatly simplified.

Moscow State University
named after M. V. Lomonosov

Received
11 II 1958

REFERENCES CITED

¹ P. K. Rashevskii, Proceedings of the Seminar on Vector and Tensor Analysis, 9, 49 (1952).
² É. Cartan, The Geometry of Lie Groups and Symmetric Spaces, Moscow, 1949.
³ B. A. Rozenfel’d, Mathematical Collection, 24, No. 1, 53 (1949).
⁴ B. A. Rozenfel’d, Non-Euclidean Geometries, Moscow–Leningrad, 1955.