MATHEMATICS

B. I. PLOTKIN

SOME PROPERTIES OF AUTOMORPHISMS OF NILPOTENT GROUPS

(Presented by Academician A. I. Mal’cev, 28 XI 1960)

1. The main result of the note is the following theorem.

Theorem 1. A finite set of automorphisms \(M\) of a nilpotent group \(G\) is nilpotent relative to \(G\) if and only if it is nilpotent relative to the factor group \(G/G'\) (here \(G'\) denotes the commutator subgroup of \(G\)).

Some facts adjoining this theorem are given, and, in particular, a new proof of the following theorem of P. Hall (1):

If in a group \(G\) there is a nilpotent normal divisor \(N\) such that \(G/N'\) is a nilpotent group, then \(G\) is also a nilpotent group.

Let us recall the notation and definitions. Let \(G\) be a group and \(M\) some set of its automorphisms. Along with commutators formed from elements of the group \(G\) or automorphisms, we shall consider commutators of the form \([g,\sigma]\), where \(g \in G\) and \(\sigma\) is an automorphism: \([g,\sigma]=g^{-1}\cdot\sigma(g)\). By \([A,M]\) \((A \subset G)\) we shall denote the subgroup generated by the commutators \([g,\sigma]\) for all \(g \in A,\ \sigma \in M\). Further denote:
\[ [G,M]=[G,M(1)],\ldots,[G,M(n)]=[[G,M(n-1)],M]; \]
\[ [[\ldots[[g,\sigma_{i_1}],\sigma_{i_2}],\ldots],\sigma_{i_k}] = [g;\sigma_{i_1},\sigma_{i_2},\ldots,\sigma_{i_k}]. \]

A finite set \(M\) is called nilpotent relative to the group \(G\) if for every element \(g \in G\) there is an exponent \(n=n(g)\) such that, for \(m>n\), all elements of the form \([g;\sigma_{i_1},\ldots,\sigma_{i_m}]\), where \(\sigma_{i_k}\in M\), are equal to the identity of the group \(G\). A group of automorphisms \(\Gamma\) of a group \(G\) is called externally locally nilpotent (locally nilpotent relative to \(G\)) if every finite subset of it is nilpotent relative to \(G\). \(\Gamma\) is called finitely stable if in \(G\) there is a finite normal series of \(\Gamma\)-admissible subgroups, on all factors of which the elements of \(\Gamma\) induce identical automorphisms (such a series is called \(\Gamma\)-stable). Finally, the group \(\Gamma\) is called locally finitely stable if, for every subgroup \(F\) of \(\Gamma\) having a finite number of generators, in \(G\) there is a local system of \(F\)-admissible subgroups \(G_\alpha\), in each of which \(F\) acts as a finitely stable group. Here and below, when speaking of the action of a set of automorphisms of a group \(G\) with respect to subgroups and factor groups from \(G\), we mean the action of the corresponding induced set of automorphisms.

It is known that the group of automorphisms \(\Gamma\) of a nilpotent group \(G\) is externally locally nilpotent if and only if it is locally finitely stable.

1. Everywhere below, \(\mathfrak{G}\) is some group, \(\Phi\) and \(\Sigma\) are its groups of automorphisms, with \(\Sigma\) invariant relative to \(\Phi\); \(H\) is a \(\Phi\)- and \(\Sigma\)-admissible

normal divisor in \(\mathfrak G\), and \(\Sigma\) induces the identity automorphisms in \(\mathfrak G/H\) and \(H\). By \(\mathcal E\) and \(E\) we shall denote the identities, respectively, in the automorphism group and in the abstract group.

In note \((^2)\) the following assertion is given.

If \([\mathfrak G,\Phi]\subset H\), then for any \(g\in\mathfrak G\) the relation
\([g,[\Sigma,\Phi(n)]]=[[g,\Sigma],\Phi(n)]\) \((n=1,2,\ldots)\) holds. We shall show that in this relation the element \(g\) may be replaced by the whole group \(\mathfrak G\), i.e., the following is true:

Lemma 1. If \([\mathfrak G,\Phi]\subset H\), then for every \(n\) one has
\[ [\mathfrak G,[\Sigma,\Phi(n)]]=[[\mathfrak G,\Sigma],\Phi(n)]. \]

Consider the case \(n=1\). The fact that \([\mathfrak G,[\Sigma,\Phi]]\subset[[\mathfrak G,\Sigma],\Phi]\) follows directly from \((^2)\), since the subgroup \([\mathfrak G,[\Sigma,\Phi]]\) is generated by the subgroups \([g,[\Sigma,\Phi]]\) over all \(g\in\mathfrak G\).

To prove the reverse inclusion, first note that \([\mathfrak G,\Sigma]\), and in particular \([\mathfrak G,[\Sigma,\Phi]]=A\), is contained in the center of \(H\). The generators in \([\mathfrak G,\Sigma]\) have the form \(a_{ij}=[g_i,\sigma_j]\), where \(g_i\in\mathfrak G\), \(\sigma_j\in\Sigma\). For each such \(a_{ij}\) one has \([a_{ij},\varphi]\in A\); here \(\varphi\) is an arbitrary element of \(\Phi\). Now let \(a\) and \(b\) be two elements of \(H\) such that \([a,\varphi]=x\in A\) and \([b,\varphi]=y\in A\). Then \(\varphi(a)=ax\), \(\varphi(b)=by\), and \(\varphi(ab)=ax\cdot by=ab\cdot xy\). Moreover, \(\varphi(a^{-1})=a^{-1}x^{-1}\). Hence \([a^{\varepsilon_1}\cdot b^{\varepsilon_2},\varphi]\in A\) for \(\varepsilon_i=\pm1\). Thus, if \(h\) is a product of positive and negative powers of elements of the type \(a_{ij}\), then \([h,\varphi]\in A\), which proves the reverse inclusion:
\([[\mathfrak G,\Sigma],\Phi]\subseteq[\mathfrak G,[\Sigma,\Phi]]\). For arbitrary \(n\) the formula is obtained by induction.

From the formula just given it follows, in particular, that if for some \(n\)
\([\Sigma,\Phi(n)]=\mathcal E\), then \([[\mathfrak G,\Sigma],\Phi(n)]=E\), i.e., \(\Phi\) is finitely stable relative to \([\mathfrak G,\Sigma]\).

The following lemma generalizes this fact to the case when \(H\) is a central subgroup in \(\mathfrak G\).

Lemma 2. Let \(H\) be a central subgroup in \(\mathfrak G\), and let \(\Phi\) induce a finitely stable group of automorphisms in \(\mathfrak G/H\). Then, if for some \(n\)
\[ [\Sigma,\Phi(n)]=\mathcal E, \]
then \(\Phi\) is finitely stable relative to \([\mathfrak G,\Sigma]\).

Proof. Let the normal series
\[ H=\mathfrak G_0\subset \mathfrak G_1\subset\ldots\subset \mathfrak G_{i-1}\subset \mathfrak G_i\subset\ldots\subset \mathfrak G_{n-1}\subset \mathfrak G_n=\mathfrak G \]
be a \(\Phi\)-stable series. Since all \(\mathfrak G_i\) contain \(H\), they are all \(\Sigma\)-admissible. Put \(H_i=[\mathfrak G_i,\Sigma]\). We obtain the series
\[ E=H_0\subset H_1\subset\ldots\subset H_{i-1}\subset H_i\subset\ldots\subset H_{n-1}\subset H_n=[\mathfrak G,\Sigma], \]
all of whose terms are \(\Phi\)-admissible normal divisors of the group \(\mathfrak G\). We shall show that this series can be refined to a \(\Phi\)-stable one.

Taking in Lemma 1 the subgroup \(\mathfrak G_1\) in place of \(\mathfrak G\), we see that \(\Phi\) is finitely stable relative to \(H_1\). We shall now prove that \(\Phi\) is finitely stable relative to an arbitrary factor \(H_i/H_{i-1}\). This will prove the lemma. Introduce the notation:
\[ \overline{\mathfrak G}=\mathfrak G_i/H_{i-1};\qquad \overline H=\mathfrak G_{i-1}/H_{i-1}; \]
\(\overline\Sigma\) and \(\overline\Phi\) are the groups of automorphisms induced by the groups \(\Sigma\) and \(\Phi\) in \(\overline{\mathfrak G}\).

It is clear that \([\overline\Sigma,\overline\Phi(n)]=\overline{\mathcal E}\) and that \(\overline\Sigma\) induces the identity in \(\overline H\) and in \(\overline{\mathfrak G}/\overline H\), while \(\Phi\) does so in \(\overline{\mathfrak G}/\overline H\). As before, we now obtain that
\([[\overline{\mathfrak G},\overline\Sigma],\overline\Phi(n)]=\overline E\), or
\[ [[\mathfrak G_i,\Sigma],\Phi(n)]=[H_i,\Phi(n)]\subset H_{i-1}. \]
The lemma is proved.

The following theorem is contained, in implicit form, in the work of P. Hall \((^1)\).

Theorem 2. Let \(G\) be a nilpotent group and \(\Gamma\) its group of automorphisms. Then, if \(\Gamma\) is finitely stable relative to \(G/G'\), then \(\Gamma\) is finitely stable also relative to the group \(G\).

Proof. We apply induction on the length \(s\) of the lower central series of the group \(G\). For \(s=1\) the theorem is trivial. Suppose it already...

has been proved for all \(s<k\), and let the series

\[ G \supset G^{(1)} \supset G^{(2)} \supset \cdots \supset G^{(k-2)} \supset G^{(k-1)} \supset G^{(k)} = E \tag{*} \]

be the lower central series of the group \(G\) of length \(k\). In accordance with the induction hypothesis, we shall assume that \(\Gamma\) is finitely stable relative to \(G/G^{(k-1)}\). Denote \(G^{(k-2)}=\mathfrak G,\; G^{(k-1)}=H\); \(H\) is a central subgroup in \(\mathfrak G\). Further, let \(\Sigma\) be the group of automorphisms induced by the inner automorphisms of the group \(G\) in \(\mathfrak G\); let \(\Phi\) be the group that \(\Gamma\) induces in \(\mathfrak G\). It is clear that \(\Sigma\) induces the identity in \(H\) and \(\mathfrak G/H\), that \(\Phi\) is finitely stable in \(\mathfrak G/H\), that \([\mathfrak G,\Sigma]=[\mathfrak G,G]\), and that \(\Sigma\) is invariant relative to \(\Phi\).

Since \(\Gamma\) is finitely stable relative to \(G/G^{(k-1)}\), for some \(n\) \([G,\Gamma(n)]\) belongs to the center of the group \(G\), and consequently \([\Sigma,\Phi(n)]=\mathcal E\). By Lemma 2 we immediately obtain that \(\Phi\) is finitely stable in \([\mathfrak G,\Sigma]=G^{(k-1)}\), which proves the theorem.

From this theorem there follows immediately the theorem of P. Hall mentioned in the introduction.

3. Lemma 3. Let \(H\) be a central subgroup in \(\mathfrak G\). Suppose, further, that \(\Phi\) has a finite number of generators and that in \(\Sigma\) there is a local system of subgroups \(\Sigma_\alpha\) invariant relative to \(\Phi\), such that for each \(\Sigma_\alpha\) there exists \(n=n(\Sigma_\alpha)\) such that \([\Sigma_\alpha,\Phi(n)]=\mathcal E\). Then, if \(\Phi\) is locally finitely stable relative to \(\mathfrak G/H\), then \(\Phi\) is locally finitely stable also relative to \([\Sigma,\mathfrak G]\).

Proof. The subgroups of the form \([\mathfrak G,\Sigma_\alpha]\) constitute a local system in \([\mathfrak G,\Sigma]\), and all of them are \(\Phi\)-admissible. Therefore it suffices to show that \(\Phi\) is locally finitely stable relative to each of these subgroups. Since \(\Phi\) has a finite number of generators, in \(\mathfrak G/H\) there is a local system of \(\Phi\)-admissible subgroups \(\mathfrak G_\beta/H\), in each of which \(\Phi\) induces a finitely stable group of automorphisms. Taking now in Lemma 2, as \(\mathfrak G\), the subgroup \(\mathfrak G_\beta\), and in place of \(\Sigma\), \(\Sigma_\alpha\), we obtain that \(\Phi\) induces a finitely stable group of automorphisms in \([\mathfrak G_\beta,\Sigma_\alpha]\). Since the subgroups \([\mathfrak G_\beta,\Sigma_\alpha]\), over all \(\mathfrak G_\beta\), constitute a local system of \([\mathfrak G,\Sigma_\alpha]\), the assertion is proved.

We now prove Theorem 1. We shall assume that the theorem has been proved for all nilpotent groups of class \(s<k\), and suppose that \(G\) is a nilpotent group whose lower central series \((*)\) has length \(k\). Let \(M\) be a finite set of automorphisms of the group \(G\), nilpotent relative to \(G/G'\); \(M\) is also nilpotent relative to \(G/G^{(k-1)}\). Denote by \(\Gamma\) the group of automorphisms of the group \(G\) generated by the set \(M\). Since \(G\) is a nilpotent group, it follows from the nilpotency of \(M\) relative to \(G/G^{(k-1)}\) that \(\Gamma\) is locally finitely stable relative to \(G/G^{(k-1)}\). This means that in \(G/G^{(k-1)}\) there is a local system of \(\Gamma\)-admissible subgroups \(G_\alpha/G^{(k-1)}\), relative to each of which \(\Gamma\) acts as a finitely stable group—for each \(G_\alpha\) there is an \(n=n(G_\alpha)\) such that \([G_\alpha,\Gamma(n)]\) belongs to the center of the group \(G\). In what follows we shall use the same notation as in the proof of Theorem 2. Denote also by \(\Sigma_\alpha\) the subgroup in \(\Sigma\) corresponding to \(G_\alpha\). It is clear that \(\Sigma_\alpha\) is invariant relative to \(\Gamma\) and, for \(n=n(G_\alpha)\), one has \([\Sigma_\alpha,\Gamma(n)]=\mathcal E\), so that the hypotheses of Lemma 3 are satisfied; therefore \(\Gamma\) is locally finitely stable relative to \([\mathfrak G,\Sigma]=G^{(k-1)}\), which proves the theorem.

At the same time the following theorem has been proved:

Theorem 3. Let \(G\) be a nilpotent group and \(\Gamma\) its group of automorphisms. Then \(\Gamma\) is locally nilpotent relative to \(G\) if and only if it is locally nilpotent relative to \(G/G'\).

From this theorem the following criterion for a locally nilpotent abstract group follows easily.

If in a group \(G\) there is a nilpotent normal divisor \(N\) such that \(G/N'\) is a locally nilpotent group, then \(G\) is also a locally nilpotent group.

Theorem 4. Let \(G\) be a nilpotent group, \(\Gamma\) its group of automorphisms, and \(\Sigma\) a normal divisor in \(\Gamma\). Suppose further that \(\Gamma\) induces finitely stable (outer locally nilpotent) groups of automorphisms in \(G/[G,\Sigma]\) and (through inner automorphisms) in \(\Sigma/\Sigma'\). Then, if \(\Sigma\) is finitely stable relative to \(G\), then \(\Gamma\) is also finitely stable (respectively, locally nilpotent) relative to \(G\).

Proof. We consider only the case of outer local nilpotence. From the fact that \(\Sigma\) is finitely stable relative to \(G\), it follows that \(\Sigma\) is a nilpotent group. But then, by the preceding theorem, \(\Gamma\) is locally nilpotent relative to \(\Sigma\). Restricting ourselves to the case where \(\Gamma\) has a finite number of generators, we see that in \(\Sigma\) there is a local system of subgroups \(\Sigma_\alpha\) invariant with respect to \(\Gamma\), in each of which \(\Gamma\) induces a finitely stable group of automorphisms.

Let first \(G\) be an abelian group. Denote
\[ [G,\Sigma]=G_1,\ldots,\ [G_i,\Sigma]=G_{i+1}. \]
We obtain in \(G\) a descending series of \(\Sigma\)-commutants
\[ G \supset G_1 \supset \cdots \supset G_{k-2}\supset G_{k-1}\supset G_k=E. \]
This series consists of \(\Gamma\)-admissible subgroups and is stable relative to \(\Sigma\). If the series has length \(k=2\), then the theorem is contained directly in Lemma 3. Suppose it has already been proved for series of length \(\leq k-1\). Then \(\Gamma\) is locally nilpotent relative to \(G/G_{k-1}\) and, in particular, relative to \(G_{k-2}/G_{k-1}\). Applying Lemma 3 to the group \(G_{k-2}\), we obtain that \(\Gamma\) is locally nilpotent relative to
\[ [G_{k-2},\Sigma]=G_{k-1}. \]
Consequently, \(\Gamma\) is locally nilpotent relative to \(G\).

Now consider the case where \(G\) is a nilpotent group. Since \(\Sigma\) is finitely stable relative to \(G\), \(\Sigma\) is finitely stable also relative to the factor group \(G/G'\). It is also clear that from the local nilpotence of \(\Gamma\) relative to \(G/[G,\Sigma]\) there follows the local nilpotence of \(\Gamma\) relative to the factor group of the group \(G/G'\) by its \(\Sigma\)-commutant. Hence it follows that \(\Gamma\) is locally nilpotent relative to \(G/G'\). By Theorem 3, \(\Gamma\) is locally nilpotent relative to \(G\).

Received
15 XI 1960

CITED LITERATURE

¹ P. Hall, Illinois J. Math., 2, No. 4 B, 787 (1958). ² B. I. Plotkin, V. G. Vil’yatser, DAN, 134, No. 3, 529 (1960).