UDC 519.46

MATHEMATICS

S. I. GELFAND

ANALYTIC REPRESENTATIONS OF THE FULL LINEAR GROUP OVER A FINITE FIELD

(Presented by Academician I. G. Petrovsky, 29 I 1968)

This note studies one important class of irreducible representations of the group \(G_n = GL(n, K_q)\), where \(K_q\) is the field of order \(q\).

Let \(Z_n \subset G_n\) be the subgroup of upper triangular matrices with ones on the diagonal.

Definition 1. A one-dimensional representation \(\chi\) of the subgroup \(Z_n\) is called degenerate if there exists another one-dimensional representation \(\chi'\) of \(Z_n\) for which \(\operatorname{Ker}\chi' \underset{\ne}{\subset} \operatorname{Ker}\chi\).

It is not hard to see that any one-dimensional representation \(\chi\) of the group \(Z_n\) has the form
\[ \chi(\xi)=\chi_0(a_1\xi_{12}+a_2\xi_{23}+\cdots+a_{n-1}\xi_{n-1,n}), \]
where \(\chi_0\) is a nontrivial additive character of the field \(K_q\), \(a_i \in K_q\), and the representation \(\chi\) is nondegenerate if and only if all \(a_i \ne 0\).

It is known \((^2)\) that the restriction of any irreducible representation \(T\) of the group \(G_n\) to \(Z_n\) contains at least one one-dimensional representation.

Definition 2. A representation \(T\) of the group \(G_n\) is called an analytic representation (briefly, an \(A\)-representation) if the restriction of \(T\) to \(Z_n\) contains no degenerate one-dimensional representations.

Introduce two subgroups of the group \(G_n\): \(H_n\) is the subgroup of matrices \(\|a_{ij}\|\) for which
\[ a_{n1}=a_{n2}=\cdots=a_{n,n-1}=0,\quad a_{nn}=1; \]
\(B_n \subset H_n\) is the subgroup of matrices \(\|a_{ij}\|\) for which
\[ a_{11}=a_{22}=\cdots=a_{nn}=1 \]
and all off-diagonal elements, except the elements of the last column, are equal to zero.

Proposition 1. With the exception of the special case \(q=n=2\), the group \(H_n\) has one irreducible representation \(T_0\) of dimension
\[ (q-1)(q^2-1)\cdots(q^{n-1}-1)=k_n(q); \]
the dimensions of the remaining irreducible representations of \(H_n\) are smaller than \(k_n(q)\).

Proof. \(H_n\) is the semidirect product of the commutative normal divisor \(B_n\) and the subgroup \(G_{n-1}\). Therefore \((^4)\) any irreducible representation \(T\) of the group \(H_n\) is induced by a representation \(\mu L\) of the group \(B_n A_\mu\), where \(\mu\) is a one-dimensional representation of \(B_n\), \(A_\mu \subset G_{n-1}\) is the subgroup of those \(g \in G_{n-1}\) such that \(\mu(gbg^{-1})=\mu(b)\), \(b \in B_n\), and \(L\) is an irreducible representation of \(A_\mu\). To finish the proof one must apply a simple induction on \(n\).

Proposition 2. The restriction of an irreducible \(A\)-representation of the group \(G_n\) to \(H_n\) coincides with \(T_0\).

Proof. We first show that in the restriction of any irreducible representation of the group \(H_n\) different from \(T_0\) to \(Z_n\) there is a degenerate one-dimensional representation. Suppose this has already been proved for the group \(H_{n-1}\).

Any irreducible representation \(T\) of the group \(H_n\) is induced by a representation \(\mu L\) of the group \(B_n A_\mu\). If \(\mu\) is the identity representation, then for any one-dimensional representation \(\chi\) of the group \(Z_n\) contained in the restriction of \(T\) to \(Z_n\), \(a_{n-1}=0\), and hence \(\chi\) is degenerate. Suppose now that \(\mu\) is a nonidentity representation. Then \(A_\mu\) is isomorphic to \(H_{n-1}\). If \(\dim T<k_n(q)\), then \(\dim L<k_{n-1}(q)\), and hence the restriction of \(L\) to \(Z_{n-1}\) contains a degenerate one-dimensional representation. It follows at once that the restriction of \(T\) to \(Z_n\) also contains a degenerate one-dimensional representation. Therefore the restriction of any \(A\)-representation of the group \(G_n\) to \(H_n\) is a multiple of \(T_0\).

On the other hand, every nondegenerate one-dimensional representation \(\chi\) of the group \(Z_n\) enters into the restriction of \(T\) to \(Z_n\) no more than once \((^2)\). Therefore the restriction of \(T\) to \(H_n\) coincides with \(T_0\), and the proposition is proved.

The representation \(T_0\) of the group \(H_n\) can be realized in the following way. Fix a nondegenerate character
\[ \chi(\xi)=\chi_0(\xi_{12}+\cdots \zeta_{n-1,n}) \]
of the group \(Z_n\). The representation \(T_0\) is defined in the space \(E_\chi\) of functions on \(G_{n-1}\) for which
\[ f(\zeta g)=\chi(\zeta)f(g),\qquad \zeta\in Z_{n-1},\quad g\in G_{n-1}^*, \]
with scalar product
\[ (\varphi_1,\varphi_2)_{n-1}=|G_{n-1}|^{-1}\sum_{g\in G_{n-1}}\varphi_1(g)\overline{\varphi_2(g)} \]
(\(|G|\) is the number of elements in the finite group \(G\)). The operator \(T_0(h)\), where \(h\in H_n\), \(h=ba\), \(b\in B_n\), \(a\in G_{n-1}\), acts by the formula
\[ T_0(h)f(g)=\chi(gbg^{-1})f(ga). \tag{1} \]
This representation is clearly unitary. From (1) it follows that the restriction of \(T_0\) to \(G_{n-1}\subset H_n\) coincides with the representation \(T_\chi\) of the group \(G_{n-1}\) induced by the one-dimensional representation \(\chi\) of the group \(Z_{n-1}\). From \((^2)\) follows

Proposition 3. The restriction of an irreducible \(A\)-representation \(T\) of the group \(G_n\) to \(G_{n-1}\) contains each irreducible component no more than once.

In what follows, an essential role is played by the so-called Bessel functions, introduced in \((^{1,2})\). They are defined as follows.

Consider the restriction of the regular representation of the group \(G_n\) to those functions \(f(g)\) on \(G_n\) for which
\[ f(\zeta g)=\chi(\zeta)f(g),\qquad \zeta\in Z_n,\quad g\in G_n. \]
According to \((^2)\), in the space of such functions there exists a unique invariant subspace \(E_T\), the representation in which is equivalent to \(T\). In \(E_T\) there exists, up to a factor, a unique vector \(J(g)\) for which
\[ J(g\zeta)=J(g)\chi(\zeta),\qquad \zeta\in Z_n. \]
This vector is called the Bessel function. From the definition it follows easily that

1. \[ J(\zeta_1 g\zeta_2)=\chi(\zeta_1\zeta_2)J(g). \]

2. \[ \overline{J(g)}=J(g^{-1}). \]

3. \[ J(e)=1. \]

4. \[ J*J=cJ;\quad *\text{ is convolution on the group }G_n, \]
   \[ f_1*f_2(g)=|G_n|^{-1}\sum_{g'\in G_n}f_1(g')f_2(gg'^{-1}). \]

In what follows we normalize \(J\) so that \(c=k_n(g)\). It is easy to establish the following expression for \(J(g)\) in terms of the character \(\psi\) of the representation \(T\):
\[ J(g)=|Z_n|^{-1}\sum_{\zeta\in Z_n}\chi(\zeta)\psi(g\zeta^{-1}). \]

Let us now determine where the function \(J(g)\) is concentrated. It is known \((^{3,5})\) that any element \(g\in G_n\) can be written in the form
\[ g=\zeta_1\delta s\zeta_2, \]
where \(\zeta_1,\zeta_2\in Z_n\), \(\delta\in D_n\) is a diagonal matrix, and \(s\in W_n\) is a permutation matrix. Hence, in view of property 1, it suffices to determine for which \(\delta s\) one has \(J(\delta s)\ne0\). Denote by \(s_k\) the matrix of order \(k\) in which the second diagonal consists of ones, and all other entries are zero. By
\[ s_{i_1,\ldots,i_l}(\delta_1,\ldots,\delta_l) \]
we denote the matrix
\[ s_{i_1,\ldots,i_l}(\delta_1,\ldots,\delta_l) = \begin{pmatrix} \delta_1 s_{i_1} & 0\\ & \ddots\\ 0 & \delta_l s_{i_l} \end{pmatrix}s_n, \]
where \(\delta_i\in K_q^*\), \(i_1,\ldots,i_l\) are integers, and
\[ i_1+\cdots+i_l=n. \]

\[ \text{* We assume that }G_{n-1}\text{ is embedded in }G_n\text{ so that if }g\in G_{n-1},\ \begin{pmatrix}g&0\\0&1\end{pmatrix}\in G_n. \]

Proposition 4. If \(J(\delta s)\ne 0\), then \(\delta s=s_{i_1,\ldots,i_l}(\delta_1,\ldots,\delta_l)\) for some \(i_k\) and \(\delta_k\).

The proof follows easily from property 1.

Let \(\varphi_i\) be an orthonormal basis in the space \(E_\chi\), and let \(k_{ij}(h)\) be the corresponding matrix elements of the representation \(T_0\) of the group \(H_n\). Let \(f_i(g)\) be the image of \(\varphi_i\) in \(E_T\) under the isometric mapping \(\tau:E_\chi\to E_T\), intertwining the operators of the representation \(T_0\).

Lemma. For any \(i\) and \(j\) the equality
\[ |H_n|^{-1}\sum_{h\in H_n} J(gh)\overline{k_{ij}(h)}=\gamma_i f_j(g), \tag{2} \]
holds, where \(\gamma_j\) are the coefficients in the expansion
\[ J(g)=\sum_i \gamma_i f_i(g). \]

Proof.
\[ J(gh)=\sum_i \gamma_i f_i(gh)=\sum_{i,j}\gamma_i k_{ij}(h)f_j(g), \]
\[ \sum_{h\in H_n}J(gh)\overline{k_{i_3j_3}(h)} =\sum_{i,j}\gamma_i f_j(g)\sum_{h\in H_n}k_{ij}(h)\overline{k_{i_3j_3}(h)} =|H_n|\gamma_i f_j(g). \]

Theorem. The matrix elements \(K_{ij}(g)\) of the representation \(T\) in the basis \(\{f_i\}\) are given by the formula
\[ K_{ij}(g)=c\sum_{a_1,a_2\in G_{n-1}}J(a_2ga_1^{-1})\varphi_i(a_1)\overline{\varphi_j(a_2)}, \]
where \(c=|G_{n-1}|^{-3}|Z_{n-1}|^{-1}\).

Proof. Transform formula (2):
\[ \gamma_i f_j(g)=|H_n|^{-1}\sum_{h\in H_n}J(gh)\overline{k_{ij}(h)}= \]
\[ =|H_n|^{-1}|G_{n-1}|^{-1} \sum_{\substack{a,a_0\in G_{n-1}\\ b\in B_n}} J(gba_0)\overline{\chi(aba^{-1})}\varphi_i(aa_0)\overline{\varphi_j(a)}= \]
\[ =|H_n|^{-1}|G_{n-1}|^{-1} \sum_{\substack{a,a_0\in G_{n-1}\\ b\in B_n}} J(ga_0)\overline{\varphi_i(aa_0)}\varphi_j(a)\chi(a_0^{-1}ba_0)\overline{\chi(aba^{-1})}. \]

Make the substitution \(aa_0=a'\), \(aba^{-1}=b_1\). Then
\[ \gamma_i f_j(g)=|G_{n-1}|^{-1}|H_n|^{-1} \sum_{a,a'\in G_{n-1}}J(ga^{-1}a')\overline{\varphi_i(a')}\varphi_j(a) \sum_{b_1\in B_n}\overline{\chi(b_1)}\chi(a'^{-1}b_1a'). \]

If \(\varphi^0\in E_\chi\) is such a function that \(\varphi^0(a)=0\) for \(a\in Z_{n-1}\), \(\varphi^0(e)=1\), and \(\beta_i\) are its coordinates in the basis \(\varphi_i\), then
\[ \sum_{a\in G_{n-1}}J(ga^{-1})\varphi_j(a)=c_0f_j(g),\qquad c_0=|G_{n-1}|^2\sum_i\gamma_i\overline{\beta_i}. \tag{3} \]

The matrix elements \(K_{ij}(g)\) of the representation \(T\) are computed by the formula
\[ K_{ij}(g_0)=|G_n|^{-1}\sum_{g\in G_n}f_i(gg_0)\overline{f_j(g)}. \]

Substituting (3), we obtain
\[ K_{ij}(g_0)=|G_n|^{-1}c_0^{-2} \sum_{\substack{g\in G_n\\ a_1,a_2\in G_{n-1}}} J(gg_0a_1^{-1})\overline{J(ga_2^{-1})}\varphi_i(a_1)\overline{\varphi_j(a_2)}= \]

\[ = c_0^{-2}\sum_{a_1,a_2\in G_{n-1}}' \varphi_i(a_1)\overline{\varphi_j(a_2)}\,|G_n|^{-1} \sum_{g\in G_n}' J(ga_2g_0a_1^{-1})\overline{J(g)}. \]

In view of properties 2 and 4 of the Bessel function, the inner sum is equal to

\[ |G_n|^{-1}\sum_{g\in G_n}' J(ga_2g_0a_1^{-1})\overline{J(g)} =J*J(a_2g_0a_1^{-1})=\bar k_n(q)J(a_2g_0a_1^{-1}). \]

Finally we obtain that

\[ K_{ij}(g_0)=\bar k_n(q)c_0^{-2} \sum_{a_1,a_2\in G_{n-1}}' J(a_2g_0a_1^{-1})\varphi_i(a_1)\overline{\varphi_j(a_2)}. \]

To find \(c_0\), observe that \(\sum_i \overline{\gamma_i}\bar\beta_i=(\tau^{-1}J,\varphi^0)_{n-1}\). But \(\tau^{-1}J\), as is not difficult to see, is a multiple of \(\varphi^0\), \(|\varphi^0|=\sqrt{|\bar Z_{n-1}|\,|G_{n-1}|^{-1}}\) and

\[ |J|=\left(|G_n|^{-1}\times \sum_g J(g)\overline{J(g)}\right)^{1/2} =\sqrt{\bar k_n(q)}\sqrt{J(e)}=\sqrt{\bar k_n(q)}. \]

Therefore

\[ c_0=\sqrt{|\bar Z_{n-1}|\bar k_n(q)|G_{n-1}|^3}, \]

and the theorem is proved.

The theorem just proved shows the importance of the Bessel function for the study of representations of the group \(G_n\). We shall now give an expression for the Bessel function of an irreducible \(A\)-representation of the group \(G_3\).*

Consider the extension \(K_{q^3}/K_q\). Each irreducible \(A\)-representation of the group \(G_3\) is determined by a multiplicative character \(\pi\) of the field \(K_{q^3}\), not identically equal to one on \(K_q\). For \(\sigma\in K_{q^3}\) introduce the symmetric functions

\[ P_1(\sigma)=\sigma+\sigma^q+\sigma^{q^2};\quad P_2(\sigma)=\sigma^{q+1}+\sigma^{q^2+1}+\sigma^{q^2+q};\quad P_3(\sigma)=\sigma^{q^2+q+1}. \]

It is clear that for any \(\sigma\in K_{q^3}\), \(P_i(\sigma)\in K_q\) \((i=1,2,3)\), and the equation

\[ Q_\sigma(\lambda)=\lambda^3-P_1(\sigma)\lambda^2+P_2(\sigma)\lambda-P_3(\sigma)=0 \]

has roots \(\sigma,\sigma^q,\sigma^{q^2}\).

Using the formulas for the characters of irreducible representations of the group \(G_3\), given in \((^6)\) (see also \((^7)\)), one can obtain the following expressions for the function \(J(g)\):

\[ J(s_3(\lambda))=\pi(\lambda), \]

\[ J(s_{1,2}(\lambda_1,\lambda_2)) =q^{-2}\sum_{\sigma^{q^2+q+1}=\lambda_1\lambda_2^2} \chi_0(\lambda_2^{-1}P_1(\sigma))\pi(\sigma), \]

\[ J(s_{2,1}(\lambda_1,\lambda_2)) =q^{-2}\sum_{\sigma^{q^2+q+1}=\lambda_1^2\lambda_2} \chi_0(-\lambda_1^{-1}\lambda_2^{-1}P_2(\sigma))\pi(\sigma), \]

\[ J(s_{1,1,1}(\lambda_1,\lambda_2,\lambda_3)) =q^{-3}\sum_{\sigma^{q^2+q+1}=-\lambda_1\lambda_2\lambda_3} u(\sigma)\pi(\sigma)-q^{-2}\pi(\lambda_2)\delta(\lambda_1\lambda_3+\lambda_2^2), \]

where

\[ u(\sigma)= \sum_{\substack{\zeta_1,\zeta_2\in K_q\\ \zeta_1\zeta_2=-\lambda_3^{-1}\lambda_2^{-2}Q_\sigma(\lambda_2)}} \chi_0(\zeta_1+\zeta_2), \]

\(\delta(x)\) is the Kronecker symbol.

According to Proposition 4, \(J(\delta s)=0\) for the remaining \(\delta s\). In view of property 1, the formulas written down make it possible to find \(J(g)\) for all \(g\in G_3\).

In conclusion, the author expresses his gratitude to M. I. Graev for his constant attention to the work and to A. A. Kirillov for valuable advice and suggestions.

Moscow State University
named after M. V. Lomonosov

Received
25 I 1968

CITED LITERATURE

1. I. M. Gelfand, M. I. Graev, DAN, 146, No. 4 (1962).
2. I. M. Gelfand, M. I. Graev, DAN, 147, No. 3 (1962).
3. I. M. Gelfand, M. A. Naimark, Tr. Mat. Inst. im. V. A. Steklova, AN SSSR, 36 (1950).
4. G. Mackey, Sborn. per. Matematika, 6, No. 6 (1962).
5. K. Chevalley, Sborn. per. Matematika, 2, No. 1 (1958).
6. R. Steinberg, Canad. J. Math., 3, No. 2, 225 (1951).
7. J. A. Green, Trans. Am. Math. Soc., 80, 402 (1955).

* In \((^1)\) the Bessel function was computed for the group \(G_2\).