UDC 519.45+512.86

MATHEMATICS

Yu. I. MERZLYAKOV

VERBAL AND MARGINAL SUBGROUPS OF LINEAR GROUPS

(Presented by Academician A. I. Mal'tsev, 27 II 1967)

The concept of the verbal subgroup \(v(G)\), corresponding to a word \(v\), is well known: it is the subgroup generated in the group \(G\) by the values of the word \(v\) when the variables range over all of \(G\). P. Hall \((^7)\) associated with each \(v\) also the marginal subgroup \(v^*(G)\) of the group \(G\), consisting of all possible \(x \in G\) satisfying the condition

\[ v(g_1,\ldots,g_n)=v(g_1x,\ldots,g_n)=\cdots=v(g_1,\ldots,g_nx) \quad \text{for all } g_i \in G. \]

For example, the verbal and marginal subgroups corresponding to the commutator \(x_1^{-1}x_2^{-1}x_1x_2\) are the commutator subgroup and the center of the group. It was observed long ago that the subgroups \(v(G)\) and \(v^*(G)\) (literally, “verbal” and “written in the margins”) in a certain sense complement one another—if one of them is large, then the other, as a rule, is small; for example, as I. Schur \((^8)\) showed, if the center has finite index in a group, then its commutator subgroup is finite. Refining this observation, P. Hall \((^{8,10})\) stated the following three conjectures for any word \(v\) and any group \(G\):

I. If the word \(v\) is finite-valued on \(G\), then \(v(G)\) is finite.

II. If the index \(m = G : v^*(G)\) is finite, then the order of \(v(G)\) is finite and divides some power of \(m\).

III. If \(G\) is Noetherian and \(v(G)\) is finite, then the index \(G : v^*(G)\) is finite.

Let us note a stronger version of the last conjecture (the Noetherian property of \(G\) is not assumed):

III′. If \(v\) is finite-valued on \(G\), then the index \(G : v^*(G)\) is finite.

At present none of these conjectures has been proved in full generality. The greatest progress has been made for outer commutator words (by definition, we obtain an outer commutator word if, in a finite sequence of distinct letters \(x_1 \ldots x_n\), we in some way insert square brackets denoting commutation). We mention some known results: Conjecture I has been proved for the “nilpotent” word \(\gamma_1(x)=x\), \(\gamma_{n+1}=[\gamma_n,\gamma]\) and any \(G\) (P. Hall), for the “solvable” word \(\delta_1(x)=x\), \(\delta_{n+1}=[\delta_n,\delta_n]\) and any \(G\), for an outer commutator word \(v\) and a finitely approximable \(G\), for an outer commutator word and a linear group over a field \((^{11})\); Conjecture II has been proved for an outer commutator word \(v\) and any \(G\) (R. Baer) \((^{5,8})\), for any \(v\) and an almost polycyclic \(G\) (P. Hall) \((^8)\), for any word \(v\) defining a variety of locally finitely approximable groups and any \(G\) (P. Stroud) \((^{12})\); Conjecture III has been proved for an outer commutator word \(v\) and any (Noetherian) group \(G\) (P. Hall) \((^{10})\), and a strengthening of this result is given in \((^{10})\). All three conjectures are true for any word \(v\) and any \(G\) from the class of groups all of whose homomorphic images are finitely approximable \((^{11})\).

In the present paper, Conjectures I, II, III are established for any word \(v\) and any linear group \(G\) over a field, with Conjecture III being proved in the form III′. On the way to this result, Conjecture II is proved for any \(v\) and any group \(G\) possessing the following local prop—

property \(\mathfrak X\): for every \(H \in \mathfrak X\) there exists an infinite set \(\pi_H\) of primes, depending on \(H\), such that for every \(p \in \pi_H\) almost all of \(H\) is approximated by finite \(p\)-groups. This includes the above-mentioned result of P. Hall on virtually polycyclic groups (recall that a group has a property virtually or almost entirely if it contains a normal divisor of finite index possessing this property).

Lemma 1. Let \(G\) be an algebraic linear group. If the word \(v\) is finite-valued on \(G\), then the connected component of the identity \(G_0\) is contained in the marginal subgroup \(v^*(G)\).

Proof. Let \(v = v(X_1,\ldots,X_s)\). We must prove that
\[ v(g_1,\ldots,g_s)=v(g_1x,\ldots,g_s)=\cdots=v(g_1,\ldots,g_sx) \]
for all \(g_i \in G,\ x \in G_0\). For this purpose consider the mappings
\[ f_i:X \to v(\ldots,g_{i-1},g_iX,g_{i+1},\ldots) \]
with fixed \(g_i \in G\). They are rational, and therefore take the dense set \(G_0\) into the dense sets \(f_i(G_0)\) ((2), p. 86). On the other hand, the word \(v\) is finite-valued on \(G\), hence each \(f_i(G_0)\) is finite. In view of density, each \(f_i(G_0)\) consists of a single element. This element must necessarily be \(f_i(e)=v(g_1,\ldots,g_s)\), which proves the lemma.

We shall agree, for brevity, to denote by \(G^v\) the set of values of the word \(v\) on the group \(G\).

Lemma 2. Hypothesis III′ is valid for any \(v\) and any linear group \(G\) over a field.

Proof. Let \(G \subset GL(n,\Omega)\). We shall assume that \(\Omega\) is algebraically closed and has infinite transcendence degree over the prime subfield. Let \(g_1,\ldots,g_r\) be all the distinct values of the word
\[ v=v(X_1,\ldots,X_s) \]
on \(G\). Let \(a_{ij}(x)\) denote the \((i,j)\)-th coefficient of the matrix \(x\). Obviously, the group \(G\) satisfies the identities
\[ \prod_{k=1}^{r} a_{ij}\bigl(v(X_1,\ldots,X_s)-g_k\bigr)=0,\qquad i,j=1,\ldots,n. \]

Let \(\overline G\) be the closure of \(G\) in the Zariski topology on \(GL(n,\Omega)\). The group \(\overline G\) also satisfies these identities; therefore each matrix in \(\overline G^v\) has as its \((i,j)\)-th coefficient the corresponding coefficient of some matrix \(g_k\). Hence, for each coefficient of a matrix in \(\overline G^v\) there are no more than \(r\) possibilities, whence
\[ |\overline G^v| \le r^{n^2}. \]
Since, by Lemma 1, \(\overline G_0 \subset v^*(\overline G)\) and, obviously,
\[ v^*(G)=G\cap v^*(\overline G), \]
we have
\[ G:v^*(G)=Gv^*(\overline G):v^*(\overline G)\le \overline G:\overline G_0<\infty. \]

The lemma is proved.

Lemma 3. If \(|G/G^*|=t\) and \(G^*\) is approximated by finite \(p\)-groups, then \(G\) is approximated by finite groups of orders \(tp^r\).

Lemma 4 \((^8,{}^{11})\). If a finite group \(F\) contains a normal divisor \(H\) of index \(m\), then \(F\) contains a subgroup \(K\) with the following conditions: \(F=HK,\ \pi(K)\subset \pi(m)\).

Denote by \(\mathfrak X\) the class of groups with the following property: for every group \(G\in\mathfrak X\) one can find an infinite set \(\pi_G\) of primes, depending on \(G\), such that for every \(p\in\pi_G\) almost all of \(G\) is approximated by finite \(p\)-groups. Obviously, if \(G\in\mathfrak X\), then every subgroup \(H\) of \(G\) lies in \(\mathfrak X\), and moreover \(\pi_H\supset \pi_G\).

Let, as usual, \(L\mathfrak X\) denote the class of all groups locally possessing the property \(\mathfrak X\). Obviously, the class \(L\mathfrak X\) is closed with respect to finite extensions.

Theorem 1. Hypothesis II is valid for any word \(v\) and any group \(G\) from \(L\mathfrak X\).

Proof. Let \(a_1,\ldots,a_m\) be representatives of \(G\) modulo \(v^*(G)\), and let \(H\) be the subgroup generated by them. It is easy to see that \(v(H)=v(G)\),

\[ v^*(H)=H\cap v^*(G). \]
This makes it possible to restrict ourselves to the case of finitely generated \(G\) and therefore to assume that \(G\in\mathfrak X\). Let the index \(m=G:v^*(G)\) be finite. Fix a prime number \(p\) from \(\pi_G\) not dividing \(m\). Let \(H\) be a normal divisor of finite index \(l\) in \(G\), approximable by finite \(p\)-groups. Then there exist homomorphisms \(f_\alpha:G\to G_\alpha\) onto finite groups \(G_\alpha\) of orders \(lp^{r_\alpha}\), whose kernels have trivial intersection (Lemma 3). Under the natural embedding of \(G\) into the Cartesian product of all \(G_\alpha\), the subgroup \(v(G)\) is embedded in the Cartesian product of all \(v(G_\alpha)\). Denote by \(d_\alpha\) the order of the group \(v(G_\alpha)\), and by \(l_0\) the maximal divisor of the number \(l\) all of whose prime divisors belong to \(\pi(m)\). We shall show that every \(d_\alpha\) divides \(l_0\).

Indeed, \(f_\alpha(v^*(G))\subset v^*(G_\alpha)\), and therefore the index \(m_\alpha=G_\alpha:v^*(G_\alpha)\) divides \(m\). In the group \(G_\alpha\) there is a subgroup \(K_\alpha\) satisfying
\[ G_\alpha=v^*(G_\alpha)\cdot K_\alpha,\qquad \pi(K_\alpha)\subset \pi(m_\alpha)\subset \pi(m) \]
(Lemma 4). Hence \(v(G_\alpha)=v(K_\alpha)\) and \(\pi(d_\alpha)\subset \pi(m)\). On the other hand, \(d_\alpha\) divides \(lp^{r_\alpha}\), and therefore divides \(l\), since \(p\notin\pi(m)\). Finally, \(d_\alpha\) divides \(l_0\).

Now we see that \(v(G)\) lies in the Cartesian product of finite groups of order \(l_0\), and therefore \(v(G)\) is a periodic group of exponent \(l_0\). Moreover, the word \(v\) is finite-valued on \(G\), since if \(a_1,\ldots,a_m\) are representatives of \(G\) modulo \(v^*(G)\), then every \(v(g_1,\ldots,g_s)\) is some \(v(a_{i_1},\ldots,a_{i_s})\). As R. Turner-Smith noted (\(^{11}\)), it follows from the finite-valuedness of \(v\) on \(G\) that \(v(G)\) contains an abelian subgroup \(A\) of finite index, namely its center. Since in our case \(A\) is a finitely generated abelian periodic group, it is finite, and together with it all of \(v(G)\) is finite. It is clear that the order of \(v(G)\) divides some power of \(m\). The theorem is proved.

Lemma 5. Every solvable finitely generated group of matrices over a field of characteristic zero is, for almost all primes \(p\), almost entirely approximable by finite \(p\)-groups.*

Lemma 6. Let \(v\) be a word, and let \(G\) be a linear group over a field \(\Omega\); a bar denotes closure in the Zariski topology. Then
\[ v^*(\overline G)=\overline{v^*(G)}. \]
If the index \(G:v^*(G)\) is finite and \(a_1,\ldots,a_m\) is a complete set of representatives of \(G\) modulo \(v^*(G)\), then it is also a complete set of representatives of \(\overline G\) modulo \(v^*(\overline G)\).

Theorem 2. Hypotheses I, II, III, III′ are valid for any word \(v\) and any linear group \(G\) over a field.

Proof. Hypothesis III′ has already been proved (Lemma 2); it remains to prove hypothesis II. Let \(v\) be a nontrivial word, \(G\) a subgroup of \(GL(n,\Omega)\), and let the index \(m=G:v^*(G)\) be finite. It is necessary to show that \(v(G)\) is finite and that its order divides some power of \(m\). In view of Lemma 6 it suffices to restrict ourselves to the case where \(G\) is closed in the Zariski topology.

Clearly, the identity \(v=1\) holds on \(v^*(G)\). But then, as V. P. Platonov observed (\(^{4}\)), from K. Chevalley’s results on the structure of semisimple algebraic groups (\(^{6}\)) it follows that \(v^*(G)_0\) is solvable (otherwise the factor group of \(v^*(G)_0\) by its solvable radical, being a nontrivial semisimple algebraic group, would contain, up to isogeny, \(SL(2,\Omega)\) (see (\(^{6}\), pp. 23–02), and with it free nonabelian subgroups). Thus \(G\) is almost solvable and hence almost entirely triangularizable (\(^{9,2}\)). Take in \(G\) a triangularizable normal divisor \(N\) of finite index and consider separately two cases.

First suppose that the field \(\Omega\) has characteristic zero. Since \(N\) is a solvable group of matrices over \(\Omega\), by Lemma 5 \(N\in_L\mathfrak X\), and therefore also \(G\in_L\mathfrak X\). Theorem 1 is applicable to the group \(G\), and everything is proved.

* Note added in proof. It can be shown that Lemma 5 remains true even without the word “solvable.”

Let now \(\Omega\) have positive characteristic \(p\). Since \(N\) is a triangulable group, its commutator subgroup \(N'\) is a periodic group of exponent \(p^n\). Obviously \(G/N' \in L\mathfrak X\). By Theorem 1, hypothesis II is valid for \(G/N'\), so that \(v(G)N'/N'\) is finite. The group \(v(G)\), being a finite extension of the periodic group \(v(G)\cap N'\), is periodic. Since \(v(G)\) is finitely generated, it is finite (cf. the end of the proof of Theorem 1).

Thus, in any case \(v(G)\) is finite. It remains to note that its order divides a power of \(m\). But, indeed, the subgroup generated by representatives of \(G\) modulo \(v^*(G)\) embeds in the direct product of finite groups \(G_\alpha\), while \(v(G)\) embeds under this into the product \(v(G_\alpha)\) \((^1)\). Obviously, each \(G_\alpha : v^*(G_\alpha)\) divides \(m\), and therefore the order of \(v(G_\alpha)\) divides a power of \(m\) (Lemma 4). Hence \(v(G)\) is approximated by finite \(\pi(m)\)-groups. Since \(v(G)\) is finite, its order divides a power of \(m\). The theorem is proved.

Institute of Mathematics
Siberian Branch of the Academy of Sciences of the USSR

Received
21 II 1967

CITED LITERATURE

1. A. I. Mal’cev, Matem. sborn., 8, No. 3, 405 (1940).
2. Yu. I. Merzlyakov, Rational Groups, 1, Novosibirsk, 1967.
3. Yu. I. Merzlyakov, VIII All-Union Colloquium on General Algebra, Riga, 1967, p. 83.
4. V. P. Platonov, Dokl. Akad. Nauk SSSR, 11, No. 7, 581 (1967).
5. R. Baer, Math. Ann., 124, 161 (1952).
6. C. Chevalley, Classification des groupes de Lie algébriques, Paris, 1958.
7. P. Hall, J. reine u. angew. Math., No. 182, 156 (1940).
8. P. Hall, Canadian Math. Congress, 1957.
9. E. R. Kolchin, Ann. Math., 49, No. 1, 1 (1948).
10. R. F. Turner-Smith, Proc. London Math. Soc., 14, No. 54, 321 (1964).
11. E. F. Turner-Smith, J. London Math. Soc., 41, No. 1, 166 (1966).
12. P. W. Stroud, Proc. Cambridge Phil. Soc., 61, 41 (1965).