Algebraic curves with many automorphisms

arXiv:1702.08812v2 [math.AG] 15 May 2018 Algebraic curves with many automorphisms Massimo Giulietti and Gábor Korchmáros Abstract Let X be a (projective, geometrically irreducible, non-singular) algebraic curve of genus g ≥ 2 defined over an algebraically closed field K of odd characteristic p. Let Aut(X ) be the group of all automorphisms of X which fix K element-wise. It is known that if |Aut(X )| ≥ 8g3 then the p-rank (equivalently, the Hasse-Witt invariant) of X is zero. This raises the problem of determining the (minimum-value) function f (g) such that whenever |Aut(X )| ≥ f (g) then X has zero p-rank. For even g we prove that f (g) ≤ 900g2 . The odd genus case appears to be much more difficult although, for any genus g ≥ 2, if Aut(X ) has a solvable subgroup G such that |G| > 252g2 then X has zero p-rank and G fixes a point of X . Our proofs use the Hurwitz genus formula and the Deuring Shafarevich formula together with a few deep results from Group theory characterizing finite simple groups whose Sylow 2-subgroups have a cyclic subgroup of index 2. We also point out some connections with the Abhyankar conjecture and the Katz-Gabber covers. 1 Introduction Numerous deep results on automorphism groups of algebraic curves, defined over a groundfield of characteristic zero, have been achieved in the passed 125 years following up the seminal work by Hurwitz who was the first to prove that complex curves, other than the rational and elliptic ones, can only have a finite number of automorphisms. Later, Hurwitz’s result was given a characteristic–free proof and the fundamental treatment of the theory of automorphisms from the view point of Galois coverings of curves was expanded to include groundfields of positive characteristic. Nevertheless, the theory of automorphism groups of curves present several different features in positive characteristic. This is especially apparent when the curve has many automorphisms, since the classical Hurwitz bound |G| ≤ 84(g − 1) on the order of the automorphism group Aut(X ) of a curve with genus g ≥ 2, may fail in positive characteristic when |G| is divisible by the characteristic of the groundfield K. As a matter of fact, curves in positive characteristic may happen to have much larger K-automorphism group compared to their genus. From previous works by Roquette [41], Stichtenoth [42, 47], Henn [28], and Hansen [24], we know infinite families of curves with |Aut(X )| ≈ cg3 and it is plausible that there exist many others with |Aut(X )| ≈ cg2 . Although curves with large automorphism groups may have rather different features, they seem to share a common property, namely their p-rank is equal to zero. This raises the problem of determining a function f (g) such that if a curve X defined over a field of characteristic p > 0 has an automorphism group G with |G| ≥ f (g) then X has zero p-rank. An upper bound on f (g) is 8g3 , as a corollary to Henn’s classification [28], see also [29, Theorem 11.127]. Our results stated in Theorems 1.1 and 1.2 show for odd p that if either Aut(X ) is solvable or g is even then f (g) > cg2 for some constant c independent of g and p. For a subgroup G of Aut(X ), a related question is to determine p(G) and G/p(G) where p(G) is the (normal) subgroup of G generated by its p-subgroups. The necessary part of the Abhyankar conjecture for affine curves states that if G has exactly r short orbits then the factor group G/p(G) has a generating set of 1 size at most 2ḡ + r − 1 where ḡ is the genus of the quotient curve X̄ = X /G. If X /G is rational (in particular when G is large), then G = p(G) for r = 1, and G/p(G) is cyclic for r = 2. The Abhyankar conjecture was stated in [1] and proven by Harbater [25]. Recent surveys on this conjecture are found in [39] and [26]. Theorems 1.1 and 1.2 provide a complete description of p(G) when |G| > 900g2 and either G is solvable or g is even. For a subgroup G of Aut(X ) the associated Galois cover is a Katz-Gabber G-cover if X |G is rational, G fixes a point P ∈ X , it has at most one more short orbit o and o is tame. Such covers are related with the Nottingham groups consisting of power series over a field K of positive characteristic with substitution as group operation, and are useful to understand how the Nottingham groups can be realized as automorphisms of a curve over K; see [6]. Theorem 1.1. Let G be a solvable automorphism group of an algebraic curve X of genus g ≥ 2. If |G| > p 2 p−2 84g then X has zero p-rank. Furthermore, G fixes a point of X and G = p(G) ⋊ C where p(G) is a Sylow p-subgroup of G and C is a cyclic group. In particular, the cover X |(X /G) is a Katz-Gabber G-cover. Theorem 1.2. Let G be a non-solvable automorphism group of an algebraic curve X of even genus g ≥ 2. If |G| > 900g2 then X has zero p-rank, and p(G) is isomorphic to one of the following linear groups: P SL(2, q) for q ≥ 5, P SU (3, q) for q ≡ 1 (mod 4), SL(2, q) for q ≥ 5, SU (3, q) for q ≡ 5 (mod 12), for a power q of p. In particular, G/p(G) is a cyclic group according to the Abhyankar conjecture; see Proposition 6.21 and Theorem 7.1. Certain quotient curves of the Hermitian and of the GK curves [12] provide examples covering all cases in Theorem 1.2; see Section 9. From the framework of this paper, it emerges that the automorphism groups of curves with even genus are subject to several constraints which allow to determine their structures completely; see Theorem 6.1. In some sense, this is an unexpected result since each finite group is the automorphism group of some curve defined over K. The ingredients of the proofs are the Hurwitz genus formula and the Deuring-Shafarevich formula together with a few deep results from Group theory characterizing finite simple groups whose Sylow 2-subgroups have a cyclic subgroup of index 2. We also point out that some of our lemmas are related to the Abhyankar conjecture for curves, although our proof of Theorem 1.2 is independent of it. The paper is organized as follows. Section 2 provides the necessary background on Galois coverings of algebraic curves in positive characteristic, it also collects results from Group theory which play a role in the paper. Section 3 contains the proof of Theorem 1.1 which is carried out by induction on g. In proving the first statement, two cases are distinguished according to weather G has a normal subgroup of order prime to p, or all its normal subgroups are p-groups. In either case we show that if a curve satisfies the hypothesis of Theorem 1.1 but has positive p-rank, then a quotient curve with respect to a well chosen normal subgroup of G also satisfies the hypotheses of Theorem 1.1 with genus less than g; see Propositions 3.3 and 3.5. The proof of the second statement of Theorem 1.1 is given at the end of the section as a consequence of Lemmas 3.4 and 3.9. Section 4 gives some new sufficient conditions for a curve to have p-rank zero; see Lemmas 4.1, 4.6, 4.8, and 4.9. The key achievement is Lemma 4.1, which can be viewed as an improvement on the bound of Theorem 1.1 for a large family of solvable groups. These results are used in several proofs in this paper, and appear to be of independent interest. 2 The rest of the paper is dedicated to the proof of Theorem 1.2. The main ingredient is the classification given in Theorem 6.1 where the abstract structure of G is determined without any assumption on the order of G. The key point is Lemma 6.2 which together with the classification of 2-groups containing a cyclic subgroup of index 2 allow us to use several deeper results from Group theory obtained within the classification project of finite simple groups. Theorem 6.1 is proved first for solvable groups, then for groups without non-trivial normal subgroups of odd order (see Subsection 6.1), and finally for the general case (see Subsection 6.2). From Theorem 6.1 it is apparent the central role of a few low dimensional linear and semilinear groups in the study of curves with even genus. This motivates the previous accurate analysis in Section 5 in which a series of rather technical lemmas shows that if the group is large enough with respect to the genus of the curve, then it is linear (and cannot be semilinear) and the p-rank of the curve must be zero. For large automorphism groups compared to the genera of the curves, the list in Theorem 6.1 shortens. This is showed in Subsection 6.3; see Lemma 6.19 for odd-core free groups, that is, for groups containing no non-trivial normal subgroup of odd order, and see Proposition 6.21 for the general case. The proof of the first statement of Theorem 1.2 is given in Section 7. A certain effort is made to fully understand how O(G) and the center Z(G) and are related; see Theorem 7.1. The proof is divided into three cases, according to the list given in Proposition 6.21, and also uses deeper result from Group theory. The second part of Theorem 1.2 is proved in Section 8. It essentially depends on Theorem 7.1 which allows to use Hering’s classification [30] of certain 2-transitive group spaces in order to determine the possibilities for p(G). Finally, Section 9 provides examples that illustrate Theorem 1.2. 2 Background and Preliminary Results In this paper, Aut(X ) stands for the automorphism group of a (projective, non-singular, geometrically irreducible, algebraic) curve X of genus g ≥ 2 defined over an algebraically closed field K of odd characteristic p. For a subgroup G of Aut(X ), let X̄ denote a non-singular model of K(X )G , that is, a projective nonsingular geometrically irreducible algebraic curve with function field K(X )G , where K(X )G consists of all elements of K(X ) fixed by every element in G. Usually, X¯ is called the quotient curve of X by G and denoted by X /G. The field extension K(X )|K(X )G is Galois of degree |G|. Since our approach is mostly group theoretical, we prefer to use notation and terminology from Group theory rather than from Function field theory. Let Φ be the cover of X → X̄ where X̄ = X /G is a quotient curve of X with respect to G. A point P ∈ X is a ramification point of G if the stabilizer GP of P in G is non-trivial; the ramification index eP is |GP |; a point Q̄ ∈ X̄ is a branch point of G if there is a ramification point P ∈ X such that Φ(P ) = Q̄; the ramification (branch) locus of G is the set of all ramification (branch) points. The G-orbit of P ∈ X is the subset of X o = {R | R = g(P ), g ∈ G}, and it is long if |o| = |G|, otherwise o is short. For a point Q̄, the G-orbit o lying over Q̄ consists of all points P ∈ X such that Φ(P ) = Q̄. If P ∈ o then |o| = |G|/|GP | and hence Q̄ is a branch point if and only if o is a short G-orbit. It may be that G has no short orbits. This is the case if and only if every non-trivial element in G is fixed–point-free on X , that is, the cover Φ is unramified. On the other hand, G has a finite number of short orbits. For a non-negative integer i, the i-th (i) ramification group of X at P is denoted by GP (or Gi (P ) as in [43, Chapter IV]) and defined to be (i) GP = {g | ordP (g(t) − t) ≥ i + 1, g ∈ GP }, 3 (0) where t is a uniformizing element (local parameter) at P . Here GP = GP . The structure of GP is well known; see for instance [43, Chapter IV, Corollary 4] or [29, Theorem 11.49]. Result 2.1. The stabilizer GP of a point P ∈ X in G has the following properties. (1) (i) GP is the unique Sylow p-subgroup of GP ; (i) (i) (i+1) (ii) For i ≥ 1, GP is a normal subgroup of GP and the quotient group GP /GP p-group. is an elementary abelian (1) (iii) GP = GP ⋊ U where the complement U is a cyclic group whose order is prime to p. Let ḡ be the genus of the quotient curve X̄ = X /G. The Hurwitz genus formula gives the following equation X 2g − 2 = |G|(2ḡ − 2) + dP . (1) P ∈X where dP = X i≥0 (i) (|GP | − 1). (2) P Here D(X |X¯ ) = P ∈X dP is the different . For a tame subgroup G of Aut(X ), that is for p ∤ |GP |, X P ∈X dP = m X i=1 (|G| − ℓi ) where ℓ1 , . . . , ℓm are the sizes of the short orbits of G. A subgroup of Aut(X ) is a p′ -group (or a prime to p group) if its order is prime to p. A subgroup G of Aut(X ) is tame if the 1-point stabilizer of any point in G is a p′ -group. Otherwise, G is non-tame (or wild). Obviously, every p′ -subgroup of Aut(X ) is tame, but the converse is not always true. From the classical Hurwitz’s bound, if |G| > 84(g(X ) − 1) then G is non-tame; see [47] or [29, Theorems 11.56]. An orbit o of G is tame if GP is a p′ -group for P ∈ o, otherwise o is a non-tame orbit of G. Result 2.2 (Stichtenoth’s result on the number of short orbits of large automorphism groups [47]; see also Theorems 11.56 and 11.116 in [29]). Let G be a subgroup of Aut(X ) whose order exceeds 84(g(X ) − 1). Then G has (a) exactly three short orbits, two tame and one non-tame, and |G| < 24g(X )2 ; (b) exactly two short orbits, both non-tame, and |G| < 16g(X )2 ; (c) only one short orbit which is non-tame; (d) exactly two short orbits, one tame and one non-tame. If |G| > 4(g(X ) − 1) then the quotient curve X /G is rational. Result 2.3 (Roquette’s result [41]). If the size of Aut(X ) exceeds 84(g(X ) − 1), then g(X ) ≥ p − 1. 4 The p-rank of X is defined to be the rank of the (elementary abelian) group of the p-torsion points in the Jacobian variety of X , and it coincides with the Hasse-Witt invariant of X . The p-rank of X does not exceed the genus g(X ). Let γ be the p-rank of X , and let γ̄ be the p-rank of the quotient curve X¯ = X /S for a p-subgroup S of Aut(X ). The Deuring-Shafarevich formula, see [46] or [29, Theorem 11,62], states that γ − 1 = |S|(γ̄ − 1) + k X i=1 (|S| − ℓi ) (3) where ℓ1 , . . . , ℓk are the sizes of the short orbits of S. Result 2.4. If X has zero p-rank then Aut(X ) has the following properties: (i) a Sylow p-subgroup of Aut(X ) fixes a point P ∈ X but its non-trivial elements have no fixed point other than P ; (ii) the normalizer of a Sylow p-subgroup fixes a point of X ; (iii) any two distinct Sylow p-subgroups have trivial intersection. Claim (i) is [29, Theorem 11.129]. Claim (ii) follows from Claim (i). Claim (iii) is [29, Theorem 11.133]. Result 2.5 (Nakajima’s bound [36]; see also [16] and Theorem 11.84 in [29]). If X has positive p-rank and S is a p-subgroup of Aut(X ) then  p p−2 (g(X ) − 1) for γ(X ) > 1, |S| ≤ (4) g(X ) − 1 for γ(X ) = 1. If, in addition, S fixes a point of X then |S| ≤ p p−1 g(X ). We quote a result on the orders of Sylow p-subgroups in large automorphism groups. Result 2.6 (Proposition 11.138 in [29]; see also Theorem 1.1 in [14]). Let X be a curve with p-rank 0, genus g(X ) ≥ 2, and such that G is a solvable automorphism group of X with |G| > 24g(X )(g(X ) − 1). Let S be a Sylow p-subgroup of G. If G has no fixed point then |S| < p2 . For automorphism groups with special structures or actions on X , the classical Hurwitz bound can be improved; see for instance [29, Theorems 11.108, 11.79, 11.60]. Result 2.7. (i) If G is abelian then |G| ≤ 4g(X ) + 4. (ii) If G has prime order other than p then |G| ≤ 2g(X ) + 1. (iii) If G fixes a point P of X and its order is prime to p then |G| ≤ 4g(X ) + 2. Result 2.8 (Automorphism groups of elliptic curves and genus 2 curves; see Theorem 11.94, Theorem 11.98, and Proposition 11.99 in [29], and Theorem 2 in [45]). If G is an automorphism group of an elliptic curve E over K then for every point P ∈ E the stabilizer GP has order at most 6 when p > 3 and at most 12 when p = 3. If G is an automorphism group of an hyperelliptic curve H over K with hyperelliptic involution ω and ω ∈ G then G/hωi is isomorphic to a finite subgroup of P GL(2, K). If X has genus 2 then G/hωi is not isomorphic to either P SL(2, q) with q ≥ 5, or Alt7 , or the Mathieu group M11 , or P SU (3, q) with q ≥ 5, or P GU (3, q) with q ≥ 5. 5 The results from Group theory which play a role in the paper are quoted below, where G denotes any abstract finite group. We use standard notation and terminology; see [19, 31, 27]. In particular, O(G) stands for the largest normal subgroup of G of odd order. Furthermore, q denotes a power of an odd prime d. Result 2.9 (Dickson’s classification of finite subgroups of the projective linear group P GL(2, K); see [49] and also Theorem A.8 in [29]). Any finite subgroup of the group P GL(2, K) is isomorphic to one of the following groups: (i) prime to p cyclic groups; (ii) elementary abelian p-groups; (iii) prime to p dihedral groups; (iv) the alternating group Alt4 ; (v) the symmetric group Sym4 ; (vi) the alternating group Alt5 ; (vii) the semidirect product of an elementary abelian p-group of order ph by a cyclic group of order n > 1 with n | (ph − 1); (viii) PSL(2, pf ); (ix) PGL(2, pf ). For q > 11, the minimal size of a set on which P SL(2, q) can act transitively is q + 1; see [31, Kapitel II, Satz 8.28]. The maximal subgroups of P SU (3, q) for q odd were classified by Mitchell [35]; see also [37] and [29, Theorem A.10]. In this paper we only need the following corollaries of Mitchell’s classification; see [35, Theorem 30]. Result 2.10. Let µ = gcd(3, q + 1). The subgroups of P SU (3, q) of order q 3 (q 2 − 1)/µ are maximal. For q > 5, they are the largest subgroups of P SU (3, q) for q > 5. For q = 5 the largest subgroups of PSU(3, q) have order 2520 and are isomorphic to Alt7 . Let Ω be a set of smallest size on which P SU (3, q) has a non-trivial action. Then |Ω| ≥ q 3 + 1 with an exception for q = 5 and |Ω| = 60. Result 2.11 (Classification of 2-groups with a cyclic subgroup of index 2; see Theorem 12.5.1 in [27], or Kapitel I, Satz 14.9 in [31]). The non-cyclic groups of order 2n which contain a cyclic subgroup of index 2 are generated by two elements a, b and are of the following types: n−1 • a2 = 1, b2 = 1, ba = ab (direct product of two cyclic groups); n−1 = 1, b2 = a2 n−1 = 1, b2 = 1, ba = a−1 b (dihedral group); n−1 = 1, b2 = 1, ba = a−1+2 • n ≥ 3, a2 • n ≥ 2, a2 • n ≥ 4, a2 n−1 n−2 , ba = a−1 b (generalized quaternion group); n−2 n−2 • n ≥ 4, a2 = 1, b2 = 1, ba = a1+2 notation as in [31]). b (semidihedral - also called quasidihedral - group); b (modular maximal-cyclic group, that is type (3) group with 6 Observe that the dihedral group for n = 2 is an elementary abelian group of order 4. Some elementary properties of the above 2-groups are as follows; see [31, Kapitel I, Satz 14.9]. A generalized quaternion group has rank 1, the other groups listed in Result 2.11 have rank 2 as they contain an elementary abelian group of order 4 but no elementary abelian group of order 8. Generalized quaternion groups, as well as dihedral and semidihedral groups, have center of order 2. Dihedral, semidihedral and modular maximal-cyclic groups have non-central involutions. The quotient groups of a generalized quaternion group, as well as of a semidihedral group, with respect to its center is a dihedral group. A modular maximal-cyclic group has exactly three involutions and four elements of order 4. In particular, it contains neither a dihedral group of order 2n ≥ 8 nor a generalized quaternion group. Result 2.12 (The Schur-Zassenhaus theorem; see Kapitel I, Hauptsatz 18.1 in [31]). Let N be a normal subgroup of G such that |N | and [G : N ] are coprime. Then N has a complement in G, that is, G = N ⋊ U for a subgroup U of G. Result 2.13 (The Gorenstein-Walter theorem [20, 21]; see also Chapter 16 in [19]). Let G be a group with a dihedral Sylow 2-subgroup S2 . Then, either G = O(G) ⋊ S2 , or G/O(G) ∼ = Alt7 , or G/O(G) is isomorphic to a subgroup of P ΓL(2, q) containing P SL(2, q). Result 2.14 (The Brauer-Suzuki theorem [8]; see also Chapter 12 in [19]). Let G be a group with a generalized quaternion Sylow 2-subgroup. Then the center of G/O(G) has order 2. Observe that a Sylow 2-subgroup of SL(2, q) is a generalized quaternion group. Result 2.15 (The Alperin-Brauer-Gorenstein theorem [4]). Let G be a non-abelian simple group with a semidihedral Sylow 2-subgroup S2 . Then, either G ∼ = PSL(3, q) with q ≡ 3 (mod 4), or G ∼ = PSU(3, q) with q ≡ 1 (mod 4), or G = M11 , where q is an odd prime power. Result 2.16 (The Alperin theorem [3]). A Sylow 2-subgroup of a non-abelian simple group with 2-rank equal to 2 is either dihedral, or semidihedral, or wreathed, or isomorphic to a Sylow 2-subgroup of SU (3, 2). A group G of even order is said to have a normal 2-complement if G = O(G) ⋊ S2 with a Sylow 2subgroup S2 of G. By an elementary result, see for instance [31, Kapitel I, Aufgaben I.21], the groups of even order with a cyclic Sylow 2-subgroup have a normal 2-complement. More generally, groups with a Sylow 2-subgroup containing a cyclic subgroup of index 2 have a normal 2-complement, apart from some exceptions; see [53, 52]. Result 2.17. If a group G has no 2-complement but a Sylow 2-subgroup S2 of G has a cyclic subgroup of index ≤ 2 then S2 is either a generalized quaternion, or a dihedral group, or a semidihedral group. If, in addition, G is also solvable then one of the following cases occurs. • S2 is dihedral and G/O(G) ∼ = PSL(2, 3), PGL(2, 3). • S2 is semidihedral and G/O(G) ∼ = GL(2, 3). Result 2.18. If a group G has no 2-complement but it has a semidihedral Sylow 2-subgroup, then one of the following holds. (i) G has a (normal) subgroup of index 2, which has no normal subgroup of index 2 and has a dihedral Sylow 2-subgroup. (ii) G has a (normal) subgroup of index 2, which has no normal subgroup of index 2 and has a generalized quaternion Sylow 2-subgroup. 7 (iii) G has no normal subgroup of index 2, and the involutions of G form a single conjugacy class. Let G be a group with non-trivial center Z(G). If N is a subgroup of Z(G) with quotient group U = G/N , then G is called a central extension of U by N . When the sequence splits, that is G has a subgroup isomorphic to U that meets N trivially, G is called a split extension of U by N , and G = N ⋊ U . If N ≤ G′ then the central extension is stem, and if G is perfect, that is G = G′ , the central extension is perfect. Result 2.19 (Schur’s result on perfect central extensions of P SL(2, q); see Kapitel V, Satz 25.7 in [31]). Let G be a perfect central extension of P SL(2, q) with q ≥ 5. If q 6= 9 then G = SL(2, q). This holds true for q = 9 provided that extension is by a group of order 2. Result 2.20 (Griess’ result on perfect central extensions of P SU (3, q); see Theorem 2 in [23]). The unique perfect central extension of P SU (3, q) is SU (3, q). A linear group containing SL(2, q) as an index 2 subgroups is SL± (2, q), that is, the subgroup of GL(2, q) consisting of all matrices with determinant equal to ±1. SL± (2, q) contains an involution other than the central involution of SL(2, q), and hence its Sylow 2-subgroups are not generalized quaternion groups. More precisely, for q ≡ −1 (mod 4) they are semidihedral groups. Similarly, the subgroup SU ± (2, q) of GU (2, q) consisting of all matrices with determinant equal to ±1 contains SU (2, q) as an index 2 subgroup. Furthermore, SL(2, q) ∼ = SU (2, q). The center of SU ± (2, q) has order either 2 or 4 according as q ≡ 1 (mod 4) or q ≡ −1 (mod 4), and in the former case the Sylow 2-subgroups of SU ± (2, q) are semidihedral. Result 2.21 (Schur’s classification of non-split central extension of P GL(2, q) by an involution; see p. 176 in [18]). Let G be a group with a Sylow 2-subgroup not isomorphic to a generalized quaternion group. If G is a stem non-split central extension of P GL(2, q) by a group of order 2 then a Sylow 2-subgroup of G is semidihedral, and either G ∼ = SU ± (2, q) according as q ≡ −1 (mod 4) or q ≡ 1 (mod 4). = SL± (2, q) or G ∼ For each odd q, P GL(2, q) has two non-isomorphic stem non-split central extension by a group of order 2. One has semidihedral Sylow 2-subgroups, the other has generalized quaternion and is denoted by SL(2) (2, q). Result 2.22 (Thompson’s transfer lemma; see Lemma 5.38 in [48]). Let G be a finite group of even order. Let M be an index 2 subgroup of a Sylow 2-subgroup S2 of G. If G has no subgroup of index 2, then every involution in S2 \ M is conjugate to an involution in M . Result 2.23 (The Landázuri and Seitz bound on the degree of cross-representations of P SU (3, q); see [33]). Assume that P SU (3, q) has an irreducible, faithful representation as a projective group of the projective space P G(r − 1, t). If t > 2 is an odd prime and q is not a t-power, then r − 1 ≥ q(q − 1). A group space (Ω, G) is a pair where Ω is a non-empty set and G is a group acting on Ω. This action is not supposed to be faithful, that is, the permutation representation of G on Ω may have non-trivial kernel. A group space (Ω, G) is called transitive if G induces on Ω a transitive permutation group. Result 2.24 (Hering’s classification on doubly transitive group spaces; see Theorem 2.4 in [30]). Let (Ω, G) be a finite transitive group space, where |Ω| > 2. Assume that for some α ∈ Ω the stabiliser Gα contains a normal subgroup Q which is sharply transitive on Ω \ {α}. If S is the normal closure of Q in G, then one of the following holds: (i) S ∼ = P SL(2, q), SL(2, q), Sz(q), P SU (3, q), SU (3, q) or a group of Ree type of order q 3 (q 3 + 1)(q − 1), where q is a prime power and the size of Ω is q + 1 in the linear case, q 2 + 1 in the Suzuki case and q 3 + 1 in the unitary and Ree case. 8 (ii) S ∼ = P ΓL(2, 8) and |Ω| = 28. (iii) (Ω, S) is sharply 2-transitive. (iv) |Ω| = p2 for a prime p ∈ {3, 5, 7, 11, 23, 29, 59}, Result 2.25 (The Gow-Burnside theorem [22]; see Theorem 11.134 in [29]). If a Sylow d-subgroup Sd of a solvable group G meets any other Sylow d-subgroup of G trivially then Sd is either cyclic, or a normal subgroup of G, or d = 2 and Sd is generalized quaternion group. Result 2.26 (The Feit-Thompson theorem [11]). Every group of odd order is solvable. 3 Large solvable automorphism groups of curves In this section, our goal is to provide a proof for Theorem 1.1. We begin with the first claim. Proposition 3.1. Let X be an algebraic curve of genus g ≥ 2 with a solvable subgroup G of Aut(X ) such that p 2 g . (5) |G| > 84 p−2 Then the p-rank of X is equal to zero. The proof of Proposition 3.1 is by induction on the genus of X , and is carried out by a series of lemmas and propositions. By (i) of Result 2.7, the group G is assumed to be non-abelian. Then, one the following cases occurs: (A) G has some non-trivial normal p′ -subgroup; (B) G has some non-trivial normal p-subgroup, but it does not have any non-trivial normal p′ -subgroup. We investigate Case (A). Lemma 3.2. Under the hypotheses of Proposition 3.1, if (A) holds then one of the following cases occurs. (C) For any normal p′ -subgroup N of G of maximal order the quotient curve X¯ = X /N has zero p-rank. (D) Some curve of genus less than g is a counterexample to Proposition 3.1 for an automorphism group satisfying Property (B). Proof. Claim (C) certainly holds if X̄ is rational. If X¯ is elliptic, take a Sylow p-subgroup S of G. Since (5) yields |G| > 84(g − 1), Result 2.2 shows that G has a non-tame orbit on X . Hence, some non-trivial subgroup of S fixes a point of X . Let S̄ = SN/N . Then S̄ is a Sylow p-subgroup of Ḡ = G/N , and some non-trivial subgroup T̄ of S̄ fixes a point of X¯ . From Result 2.8, |T̄ | = 3 (and p = 3). The Deuring-Shafarevich formula applied to T̄ gives γ(X̄ ) − 1 = 3(γ(Ȳ) − 1) + 2λ where Ȳ is the quotient curve of X¯ by T̄ while λ > 0 is the number of fixed points of T̄ on X̄ . Since 0 ≤ γ(Ȳ) ≤ γ(X¯ ) ≤ g(X̄ ) = 1, this equation implies γ(X¯) = 0. Then Claim (C) also holds for g(X̄ ) = 1. If g = 2, then X̄ is either rational or elliptic and both cases have already been considered. Therefore, g ≥ 3 and g(X¯ ) ≥ 2 are assumed. From the Hurwitz genus formula applied to N , g ≥ |N |(g(X¯ ) − 1) + 1. 9 This together with (|N |(g(X¯ ) − 1) + 1)2 > |N |g(X¯)2 yield |Ḡ| = |G| > |N | p g2 p−2 84 |N | > p 2 p−2 84g(X̄ ) . The group Ḡ is a non-abelian solvable group and hence it has a non-trivial minimal normal subgroup M̄ . Let M be the normal subgroup of G containing N such that M/N = M̄ . Then p divides |M | as N has been chosen to be a maximal normal p′ -subgroup of G. Since p ∤ |N |, this yields that p divides |M̄ |, as well. Since M̄ is a minimal normal subgroup, and G is solvable, M̄ is a normal (elementary abelian) p-subgroup of Ḡ. If the p-rank of X¯ is positive, X¯ turns out to be a counterexample to Proposition 3.1 for its automorphism group Ḡ satisfying property (B). Proposition 3.3. Under the hypotheses of Proposition 3.1, if (A) holds then either X has zero p-rank, or (D) holds. Proof. Among the normal p′ -subgroups of G choose one, say N , of maximal order. Let M̄ be a normal (elementary abelian) p-subgroup of Ḡ defined as in the final part of the proof of Lemma 3.2. By Lemma 3.2 we may assume that (C) holds. Then γ(X̄ ) = 0, and hence M̄ fixes a unique point P̄ ∈ X¯ , by (i) of Result 2.4. Let θ be the N -orbit consisting of all points of X lying over P̄ in the cover X |X¯ . Since p ∤ |N |, we also have p ∤ |θ|. Furthermore, as M̄ is a normal subgroup of Ḡ, P̄ is fixed by Ḡ, as well. Therefore, θ is a G-orbit. Choose a point P ∈ θ. Then |G| = |GP ||θ| and (iii) of Result 2.1 shows that the stabilizer GP of P in G is the semidirect product of a Sylow p-subgroup S of GP by a cyclic group H of order h with p ∤ h. Actually, S is a Sylow p-subgroup of G since p does not divide |θ|. If θ = {P } then G = GP = S ⋊ H. From this and (5), 24g2 < |G| = |S||H| ≤ |S|(4g + 2) by (iii) of Result 2.7 whence 24g2 g2 |S| > >6 > 4g. 4g + 2 g+1 From Result 2.5, X has zero p-rank. Therefore, |θ| ≥ 2 is assumed. Since N is a normal subgroup of G, the set L = N H is a subgroup of G whose order |L| is equal to |H||N |/|H ∩ N |. Obviously, θ is an L-orbit, and hence |L| = |LP ||θ| with LP being the stabilizer of P in L and H ≤ LP . Since |G| = |GP ||θ| = |S||H||θ|, the Lagrange theorem yields that |H| |G| = |S| |L| |Lp | is an integer. Since |S| and |LP | are coprime, this yields that |LP | must divide |H|. Hence, LP cannot contain H properly. Therefore, H = LP , and |G| = |S||LP ||θ| = |S||L|. As p does not divide |L|, Result 2.2 yields |L| ≤ 84(g − 1). It turns out that p 84 p−2 g2 < |G| = |S||L| ≤ |S|84(g − 1) < 84|S|g, whence |S| > p p−2 g. (6) From Result 2.5, X has zero p-rank. Lemma 3.4. Under the hypotheses of Proposition 3.1, if (A) holds and X has zero p-rank then G fixes a point. 10 Proof. Among the normal p′ -subgroups of G choose one, say N , of maximal order and let X̄ = X /N . Let M and M̄ be as in the final part of the proof of Lemma 3.2. We first show that M̄ fixes a unique point P̄ ∈ X¯ . Since |M | = |M̄ ||N |, from Result 2.12, there exists a p-subgroup T in G such that M = N ⋊ T . Clearly, T̄ = T /N coincides with M̄ . By (i) of Result 2.4, T fixes a point P ∈ X and no non-trivial element of T fixes a point other than P . Therefore, T̄ fixes the point P̄ lying under P in the cover X |X¯ . Assume that T̄ fixes a point Q̄ 6= P̄ . The N -orbit ω consisting of all points lying over Q̄ is left invariant by M . Since |ω| divides |N | and p is prime to |N |, p is prime to |ω| as well. Then T must fix a point Q ∈ ω. Since P 6∈ ω, this contradicts (i) of Result 2.4. Therefore, P̄ is the unique fixed point of M̄ . Since M̄ is a normal subgroup of Ḡ, P̄ is the unique fixed point of Ḡ, as well. This shows that ω is a non-tame short orbit of G. If ω = {P }, then G fixes P . Otherwise, the final part in the proof of Proposition 3.3 shows that |G| ≤ |S|84(g − 1) where p S is the Sylow p-subgroup of G fixing P . This together with our hypothesis (5) give |S| > p−1 g. On the 2 other hand, g ≥ p − 1 by Result 2.3. Therefore |S| > p and hence |S| ≥ p . By Result 2.6 G fixes a point. So far, we have proven that if a counterexample for Proposition 3.1 exists then Case (D) must hold. Therefore, we are left with Case (B). Proposition 3.5. Under the hypotheses of Proposition 3.1, if (B) holds then either X has zero p-rank, or (D) occurs. Proof. Choose a maximal normal p-subgroup N of G. Then |N | = ph with h ≥ t, and the factor group Ḡ = G/N contains no normal p-subgroup. If Ḡ is trivial then G = N , that is, G is a p-group, and γ(X ) = 0 by Result 2.5. Therefore |Ḡ| > 1 is assumed. Let S be a Sylow p-subgroup of G. Then S ≥ N . From Result 2.2, G has a unique non-tame short orbit, say θ. Since N is a normal subgroup, θ is partitioned into N -orbits, each of the same length ph−k where pk = |NP | for any point P ∈ θ. If λ counts the components of the partition, then N has exactly λ short orbits and |θ| = λph−k . Now we prove three preliminary claims that will be useful when dealing separately, as usual, with the cases g(X¯ ) = 0, 1, ≥ 2 for the quotient curve X̄ = X /N . Claim 3.6. If S = N , or |θ| = 1 then X has zero p-rank. Proof. If S = N then by Result 2.12, G = S ⋊ L with a p′ -subgroup L of G. Therefore, |G| = |S||L| ≤ g(X )2 p |S|84(g − 1) whence |S| > p−2 · g(X )−1 . Hence X has zero p-rank by Result 2.5. Therefore, S ) N is assumed. Then S̄ = S/N is a non-trivial p-subgroup of Aut(X¯). If |θ| = 1 then G would fix a point and Result 2.1 would imply S = N . Therefore, S ) N and |θ| > 1 are assumed. Claim 3.7. If λ = 1 then γ(X̄ ) ≥ 2. Proof. For λ = 1, the Deuring-Shafarevich formula applied to N gives γ(X )− 1 = ph (γ(X̄ )− 1)+ (ph − ph−k ). From this γ(X̄ ) − 1 ≥ 0, otherwise γ(X ) = 0. Furthermore, if γ(X̄ ) = 1 then γ(X ) − 1 = ph − ph−k . On the other hand, if X has positive p-rank then Result 2.5 together with |S| > |N | yield γ(X ) − 1 ≥ p−2 p |S| ≥ p−2 h+1 . Therefore, p p ph − ph−k ≥ (p − 2)ph , a contradiction with p > 2. Claim 3.8. If λ > 1, then some non-trivial subgroup of S̄ = S/N has a fixed point on X̄ . 11 (1) Proof. We show that if λ > 1 then GP
NP for any point P ∈ θ. For |NP | = 1, this is trivial. Thus k > 0 is assumed. From the Deuring-Shafarevich formula applied to N , γ(X ) − 1 = ph (γ(X¯ ) − 1) + λ(ph − ph−k ) ≥ −ph + λ(ph − ph−k ) whence  
1 1 γ(X ) − 1 ≥ λph 1 − − k ≥ λph 1 − 21 − 31 ≥ 16 λph = 61 pk |θ|. (7) λ p (1) According to (iii) of Result 2.7, for a point P ∈ θ write GP = GP ⋊ H where H is a p′ -subgroup of G (1) fixing P . Then |H| ≤ 4g + 2 by (iii) of Result 2.7. Now if GP = NP , this together with (7) would yield (1) 6(g(X ) − 1) ≥ 6(γ(X ) − 1) ≥ |GP ||θ| whence (1) 24g(X )2 > 6(4g(X ) + 2)(g(X ) − 1) ≥ |GP ||H||θ| = |G|, a contradiction. Since Case (B) occurs and G is not p-group, Ḡ has a non-trivial minimal normal subgroup T̄ which is an elementary abelian group of order dj with a prime d 6= p. Let T be the subgroup of G containing N such that T̄ = T /N . Three cases are investigated separately according to the values of g(X̄ ). If X̄ is rational, then Ḡ is isomorphic to a subgroup of P GL(2, K) of order divisible by p which has a normal p′ -subgroup isomorphic to T̄ . From Result 2.9, p = 3 and Ḡ ∼ = Alt4 or Ḡ ∼ = Sym4 . Therefore, 2 h h+1 h+1 2 24g(X ) < |G| ≤ 3 |Ḡ| ≤ 8 · 3 whence |S| = 3 ≥ 3g(X ) > 3g(X ). From Result 2.5, X has zero p-rank. If X¯ is elliptic then λ > 1 by Claim 3.7. Hence S̄ is a subgroup of X¯ with a non-trivial subgroup R̄ fixing a point of X¯. From Result 2.8, |R̄| = 3 (and p = 3). The Deuring-Shafarevich formula applied to R̄ gives γ(X¯ ) − 1 = 3(γ(Ȳ) − 1) + 2τ where Ȳ is the quotient curve of X̄ by R̄ while τ > 0 is the number of fixed points of R̄ on X¯ . This equation is only consistent with γ(X¯ ) = 0 and τ = 1. As a consequence, S̄ fixes a unique point P̄ . Also, S̄ = R̄ and hence |S| = 3|N |. Furthermore, the N -orbit ω consisting of all points of X lying over P̄ is an S-orbit. Therefore, |SP |/|NP | = |S|/|N | = 3 for P ∈ ω. Write GP = SP ⋊ H with a p′ -subgroup fixing P . If k = 0 then no non-trivial element of N fixes a point, and γ(X̄ ) = 0 gives a contradiction. Therefore k > 0 and (7) holds as in the proof of Claim 3.8. From (7), 18(γ(X ) − 1) ≥ 3 · 3k |θ| = |SP ||θ| which together with (iii) of Result 2.7 give |G| = |SP ||θ||H| ≤ 18(γ(X ) − 1)(4g(X ) + 2) ≤ 18(4g(X ) + 2)(g(X ) − 1). But this contradicts our hypothesis (5). If g(X̄ ) ≥ 2, we may argue as in the proof of Lemma 3.2 to prove p g(X̄ )2 . |Ḡ| > 84 p−2 Since T̄ is a normal p′ -subgroup of Ḡ, Proposition 3.3 applies to X¯ . Therefore, either (D) occurs, or X̄ has zero p-rank. In the latter case, from Lemma 3.4, Ḡ fixes a point P̄ ∈ X̄ . Hence, the N -orbit consisting of all points of X lying over P̄ coincides with θ. Then θ is also an S-orbit and |θ| = |N |/|NP | = |S|/|Sp | holds. (1) Write GP = GP ⋊ H with a p′ -subgroup H. Since |H| ≤ 4g(X ) + 2 by (iii) of Result 2.7. this yields |G| = |GP ||θ| ≤ |SP ||H| |S| ≤ |S|(4g(X ) + 2). |SP | 12 From this and (5), |S| > cg(X )2 > 3g(X ). 4g(X ) + 2 Therefore X has zero p-rank by Result 2.5. Proposition 3.1 is a corollary of the above lemmas and propositions. In fact, let X be a counterexample of minimal genus. From Proposition 3.3 and Lemma 3.4, case (B) occurs for X . But this is ruled out by Proposition 3.5. A further ingredient of the proof of Theorem 1.1 is the following lemma. Lemma 3.9. Let X be an algebraic curve of genus g ≥ 2 and p-rank zero. Let G be a solvable subgroup of Aut(X ). If G has no non-trivial normal p′ -subgroup then G fixes a point of X . Proof. Let N be a minimal normal subgroup of G. Since G is solvable, N is an elementary abelian group. By our hypothesis, p | |N |. Therefore N is a normal p-subgroup of G. Since X has zero p-rank, N fixes a unique point P ∈ X by (i) of Result 2.4. Therefore G fixes P . Now we are in a position to give a proof for Theorem 1.1. From Proposition 3.1 together with Lemmas 3.4 and 3.9, the p-rank of X is zero and G fixes a point of X . Then G = S ⋊ C with a Sylow p-subgroup S and a cyclic group C. From (i) of Result 2.4, no non-trivial element in S fixes a point other than P . Therefore, if G has a short orbit distinct from {P } then it is tame. Actually, it is also unique by Result 2.2 by virtue of the hypothesis |G| > 24g2 . Then the cover X |(X /G) is a Katz-Gabber G-cover. 4 Automorphism groups and p-rank In this section, we provide some sufficient conditions for a curve to have p-rank equal to zero, which will be useful for the proof of Theorem 1.2. Throughout the section d stands for an odd prime and X is an algebraic curve of genus g ≥ 2. We first show that for special solvable groups, the bound of Theorem 1.1 can be improved. Lemma 4.1. Let H be an automorphism group of an algebraic curve of genus g ≥ 2 with a normal Sylow d-subgroup Q of odd order. Suppose that a complement U of Q in H is abelian, and that NH (U ) ∩ Q = {1}. If |H| ≥ 30(g − 1), (8) then d = p and U is cyclic. Moreover, the quotient curve X̄ = X /Q is rational and either (i) X has positive p-rank, Q has exactly two (non-tame) short orbits, and they are also the only short orbits of H; or (ii) X has zero p-rank and H fixes a point. Proof. We have H = Q ⋊ U . Set |Q| = dk , |U | = u. Then u 6= d. If u = 2 then (8) reads |Q| > 15(g − 1). If d = p then X has zero p-rank by Result 2.5; then (ii) holds, since Q fixes a point by (i) of Result 2.4 and Q is normal in H. Otherwise, Q is a prime to p subgroup of H. The Hurwitz genus formula applied to Q gives |Q| < 6(g − 1) for d 6= 3 and |Q| < 12(g − 1) for d = 3, contradicting (8). Therefore we may assume that u ≥ 3. 13 Three cases are treated separately according as the quotient curve X¯ = X /Q has genus ḡ at least 2, or X̄ is elliptic, or rational. If g(X¯ ) ≥ 2, then Aut(X¯ ) has a subgroup isomorphic to U , and (i) of Result 2.7 yields 4g(X̄ ) + 4 ≥ |U |. Furthermore, from the Hurwitz genus formula applied to Q, g − 1 ≥ |Q|(g(X¯) − 1). Therefore, (4g(X̄ ) + 4)|Q| ≥ |U ||Q| = |H| ≥ 30(g − 1) ≥ 30|Q|(g(X̄ ) − 1), whence 30 ≤ 4 g(X̄ ) + 1 ≤ 12 g(X̄ ) − 1 contradicting (8). If X¯ is elliptic, then the cover X |X¯ ramifies, otherwise X itself would be elliptic. Thus, Q has some short orbits. Take one of them together with its images o1 , . . . , ou under the action of H. Since Q is a normal subgroup of H, o = o1 ∪ . . . ∪ om is a H-orbit of size u1 dv where dv = |o1 | = . . . |om |. Equivalently, the stabilizer of a point P ∈ o has order dk−m u/u1, and it is the semidirect product Q1 ⋊ U1 where |Q1 | = dk−v and |U1 | = u/u1 for a subgroup Q1 of Q and U1 of U respectively. The point P̄ lying under P in the cover X |X¯ is fixed by the factor group U¯1 = U1 Q/Q. Since X̄ is elliptic, Result 2.8 implies |Ū1 | ≤ 12 for p = 3 and |Ū1 | ≤ 6 for p > 3. As Ū1 ∼ = U1 , this yields the same bound for |U1 |, that is, u ≤ 12u1 for p = 3 and u ≤ 6u1 for p > 3. Furthermore, if p = 3 then d > 3, and dP ≥ dk−m − 1 ≥ 54 dk−m . If p > 3 then dP ≥ dk−m − 1 ≥ 32 dk−m . From the Hurwitz genus formula applied to Q, if p = 3 then 2g − 2 ≥ dm u1 dP ≥ dm u1 ( 45 dk−m ) ≥ 45 dm u1 ≥ 1 m 15 d u = 1 15 |Q||U | = 1 15 |H|, while for p > 3, 2g − 2 ≥ dm u1 dP ≥ dm u1 ( 32 dk−m ) ≥ 23 dm u1 ≥ 91 dm u = 91 |Q||U | = 19 |H|, But this contradicts (8). If X¯ is rational, then Q has at least one short orbit. Furthermore, Ū = U Q/Q is isomorphic to a subgroup of P GL(2, K) ∼ = Aut(X¯ ). Since U ∼ = Ū , Result 2.9 shows that either (I) U is an elementary abelian p-group and Ū fixes a point P̄∞ but no non-trivial element of Ū fixes a point other than P̄∞ , or (II) U is a cyclic group of order prime to p, Ū fixes two points P̄0 and P̄∞ but no non-trivial element in Ū fixes a point other than P̄0 and P̄∞ . Let o∞ and o0 be the Q-orbits lying over P̄0 and P̄∞ , respectively. Obviously, o∞ (and in case (II) also o0 ) is a short orbit of H. We show that Q has at most one or two short orbits according to cases (I) and (II), the candidates being o∞ and in case (II) also o0 . By absurd, there is a Q-orbit o of size dm with m < k which lies over a point P̄ ∈ X̄ different from P̄∞ (in case (II) from both P̄0 and P̄∞ ). Since the orbit of P̄ in Ū has length u, then the H-orbit of a point P ∈ o has length udm . If u > 3, the Hurwitz genus applied to Q gives 2g − 2 ≥ −2dk + udm (dk−m − 1) ≥ −2dk + udm 32 dk−m ≥ −2dk + 23 udk ≥ 32 (u − 3)dk > 16 udk ≥ 16 |H|, a contradiction with |H| > cg for c = 36. If u = 3 then d ≥ 3, and hence 2g − 2 ≥ −2dk + 3dm (dk−m − 1) = dk − 3dm > 31 dk , 14 whence |H| = 3dk < 9g, a contradiction with (8). In case (I), u is a power of p with p 6= d while |o∞ | = dk . Hence U must fix a point P∞ ∈ o∞ . As o∞ is a short Q-orbit, HP contains a non-trivial element v from Q. From (i) of Result 2.1, U is a normal subgroup of HP . Therefore, w ∈ NH (U ) ∩ Q, a contradiction. We are left with case (II). Since H has exactly two short orbits, (8) together with Result 2.2 yield that p-divides |H|. Therefore, d = p. Assume that Q has two short orbits. They are o∞ and o0 . If their lengths are pa and pb with a, b < k, the Deuring-Shafarevich formula applied to Q gives γ(X ) − 1 = −pk + (pk − pa ) + pk − pb whence γ(X ) = pk − (pa + pb ) + 1 > 0. The same argument shows that if Q has just one short orbit, then γ(X ) = 0 and the short orbit consists of a single point P . Since Q is a normal subgroup of H, P is also fixed by H. Remark 4.2. As a byproduct of the the above proof, if the hypotheses of Lemma 4.1 are satisfied then Case (II) occurs, that is, g(X¯) = 0, U is a cyclic group of order prime to p, Ū = U Q/Q fixes two points P̄0 and P̄∞ in X¯ but no non-trivial element in Ū fixes a point other than P̄0 and P̄∞ . In fact, the other possibilities have explicitly been ruled out in that proof for u ≥ 3. The claim is true also for u = 2 since |Q| > 15(g − 1) yields g(X¯ ) = 0 by the last statement of Result 2.2. Remark 4.3. In Lemma 4.1, if d = p is assumed then the extra-hypothesis NH (U )∩Q = {1} is unnecessary, since in the proof of Lemma 4.1 the hypothesis NH (U ) ∩ Q = {1} is only used to rule out Case (I), in which d 6= p holds. In fact, if Q is a p-subgroup then U is a p′ -subgroup, and (I) cannot occur. Furthermore, Lemma 4.1 applies whenever H is taken for a group stabilizing a point of X and satisfying (8). Remark 4.4. Both Cases (i) and (ii) in Lemma 4.1 occur as the following examples show. For Case (i), let X be a non-singular model X of the Artin-Mumford curve where C is the plane irreducible (singular) curve C of affine equation (X p − X)(Y p − Y ) − 1 = 0. The main properties of X are well known; see [9, 5, 32]: X is an ordinary curve X of genus g (and p-rank) equal to (p − 1)2 . Furthermore, C has two (ordinary) singular points X∞ and Y∞ both p-fold points. Each of them is the center of p places of K(X ). Let o∞ be the set of the p points of X associated to the p places centered at X∞ , and o0 that arising from Y∞ in the same way. For every a, b ∈ Fp , the linear map (X, Y ) → (X + a, Y + b) is an automorphism of C and hence of X . The linear maps form an (elementary abelian) group Q = Cp × Cp of order p2 , and Q has two (short) orbits of length p, namely o∞ and o0 . Also, for every c ∈ F∗p the linear map (X, Y ) → (cX, c−1 Y ) is an automorphism of C and hence of X . They form a cyclic group Cp−1 of order p − 1 that preserves both o∞ and o0 . A further (linear) automorphism is ϕ : (X, Y ) → (Y, X) which interchanges o∞ and o0 . The group generated by Cp−1 together with ϕ is a dihedral group Dp−1 . Therefore, Aut(X ) has a subgroup G = (Cp × Cp ) ⋊ Dp−1 with a dihedral group of order 2(p − 1). If K = Fp or (K, | · |) is non-Archimedean valued field of characteristic p, then Aut(X ) = G; see [5] and [9]. For p ≥ 31, |H| = p2 (p − 1) ≥ 30(g − 1) and H = (Cp × Cp ) ⋊ Cp−1 satisfies the hypotheses of Lemma 4.1, and we have Case (i). For Case (ii), let X be a non-singular model X of the the plane irreducible (singular) curve C of affine equation Y p + Y − X m = 0 with m < p. For the main properties of X see [47], and [29, Theorems 12.9 and 12.11]. If m does not divide p + 1 then X has genus 21 (p − 1)m and p-rank zero while H = Aut(X ) fixes the infinite point and H = S ⋊ C with |S| = p and |C| = m(p − 1). If p ≥ 17 then |H| > 30(g − 1), and we have Case (ii). Remark 4.5. In Lemma 4.1, if we assume d 6= p and [H : Q] ≥ 3 but drop the extra-hypothesis NH (U )∩Q = {1} then Case (I) in the proof of Lemma 4.1 occurs. Then H has a unique short orbit o∞ and the quotient curve X̃ = X/H is rational. From the Abhyankar conjecture as stated in Section 2, H is generated by U 15 together with its conjugates in H. In particular, H is not a direct product of Q and U . We do not use this fact in the forthcoming proofs. Lemma 4.6. Let T be a subgroup of Aut(X ) containing a normal subgroup H that satisfies the hypotheses of Lemma 4.1. Then T = Q ⋊ V where Q is a p-group, V is a cyclic or a dihedral group, and p is prime to |V |. If in addition X has positive p-rank and T fuses two short H-orbits together into one orbit then V is dihedral. Proof. By Remark 4.2 we may suppose that (II) in the proof of Lemma 4.1 holds. Since H is a normal subgroup of T , and o∞ and o0 are the only short orbits of H, two possibilities arise, namely either T preservers both o∞ and o0 , or o∞ ∪ o0 is a T -orbit. On the other hand, Q is the unique Sylow p-subgroup of H. Thus Q is a characteristic subgroup of H, and hence Q is a normal subgroup of T . Therefore, T̄ = G/Q is a subgroup of P GL(2, K). It turns out that either T̄ fixes both points P̄0 and P̄∞ , or T̄ interchanges them. From Result 2.9, T̄ is cyclic in the former case, and it is a dihedral group in the latter case. In both cases p is prime to |T̄ |. Therefore Q is prime to [T : Q], and hence T = Q ⋊ V where V is either cyclic or dihedral, and p is prime to |V |. Remark 4.7. In the dihedral case, some (involutory) elements of T̄ fixes a point P̄ distinct from P∞ and P0 . This agrees with the Abhyankar conjecture as stated in Section 2 which shows that the possibility for T to be dihedral in Lemma 4.6 can only occur when T has a short orbit other than the fused one. This is the case when X is a non-singular model of the Artin-Mumford curve C; see Remark 4.4. In fact, the automorphism ϕ : (X, Y ) → (Y, X) of C fixes each point P (a, a) ∈ C with ap − a ± 1 = 0 and these points are outsides o∞ and o0 . We do not use this fact in the forthcoming proofs. Lemma 4.8. Let G be a subgroup of Aut(X ) of order at least 16g2 which satisfies property (iii) of Result 2.4. If G has a Sylow d-subgroup Q of odd order such that the normalizer NG (Q) satisfies the hypotheses on H in Lemma 4.1, that is Q has an abelian complement U in NG (Q) with NNG (Q) (U ) ∩ Q = {1} and |NG (Q)| ≥ 30(g − 1), then X has zero p-rank. Proof. By Lemma 4.1, d = p. From Remark 4.2 Case (II) in the proof of Lemma 4.1 occurs. Assume by contradiction that X has positive p-rank. Then from the proof of Lemma 4.1, there exist exactly two short Q-orbits o∞ and o0 . Each is left invariant by NG (Q), as well. If they are in two different orbits of G then Case (b) in Result 2.2 with |G| < 16g2 holds, a contradiction. Therefore, if P∞ ∈ o∞ and P0 ∈ o0 then there exists w ∈ G taking P∞ to P0 . Take two non-trivial elements r∞ , r0 ∈ Q fixing P∞ and P0 , respectively. The conjugate r of r∞ by w fixes P0 . Hence both r and r0 are in the stabilizer of P0 in G. From (i) of Result 2.1, a p-subgroup R of G contains both r and r0 . Since the conjugate Q′ of Q by w is a Sylow p-subgroup of G, R has non-trivial elements from both two Sylow p-subgroups Q and Q′ . Since G is a group in which any two distinct Sylow p-subgroups have trivial intersection, it turns out that Q = Q′ , that is, w ∈ NG (Q). This is a contradiction as o∞ is an NG (Q)-orbit. Lemma 4.9. Let G be a subgroup of Aut(X ) with a non-trivial p-subgroup Q that satisfies the following properties. (i) Z(G) has a non-trivial element g of order t with t < p, (ii) The normalizer of Q in G contains a subgroup N such that N = Q ⋊ M with an abelian group M of order prime to p properly containing hgi. If X¯ = X /hgi has zero p-rank and X̃ = X /Q is rational then X has also zero p-rank. 16 Proof. Let U = hgi. The quotient group Q̄ = QU/U is a p-subgroup of Aut(X¯ ). Since γ(X¯ ) = 0, there exists a point P̄ ∈ X¯ fixed by Q̄ but no non-trivial element in Q̄ fixes a point of X̄ other than P̄ . Let o be the U -orbit in X lying over P̄ in the cover X |X¯ . Since |o| ≤ t and t < p, Q fixes o pointwise. No point R ∈ X other than those in o is fixed by Q, otherwise the point R̄ lying under R in the cover X |X¯ would be a fixed point of Q̄ distinct from P̄ . Therefore, N preserves o. From (ii), |o| ≤ t < |M |. Hence, some non-trivial element m ∈ M fixes a point in o. Since M is abelian and contains U , m fixes o pointwise. Furthermore, the cover X |X˜ totally ramifies at the points of X˜ lying under those in o while it is unramified elsewhere. Now look at the action of the quotient group Ñ = N/Q as a subgroup of Aut(X˜ ). In the natural group homomorphism N → Ñ , the image m̃ of m is a non-trivial automorphism of X̃ which fixes each of the |o| points of X˜ lying under the points of o in the cover X |X˜ . Since X˜ is rational, Ñ is isomorphic to a subgroup of P GL(2, K). Moreover, Ñ is abelian by M ∼ = Ñ . From Result 2.9, Ñ is a cyclic group of order prime to p and hence it fixes exactly two points in X˜ , and no non-trivial element of Ñ fixes a further point in X˜ ; see [49, Theorem 1]. In particular, the image g̃ of g has exactly two fixed points in X̃ , and they are also the fixed points of m̃. From this, |o| ≤ 2. If equality holds then both points in o must be fixed by N . But this is impossible since o is an U -orbit and U ≤ N . Therefore, |o| = 1, and hence Q has exactly one fixed point in X . Since γ(X˜ ) = 0, the Deuring-Shafarevich formula yields that γ(X ) = 0 also holds. 5 Some low dimensional linear groups as automorphism groups of curves For a prime power q = dk with an odd prime d, let W denote one of the following non-solvable linear groups P SL(2, q), P GL(2, q), with q ≥ 5, P SU (3, q), P GU (3, q), P SL(3, q), P GL(3, q). (9) Choose a Sylow d-subgroup Q of W . If W 6= P SL(3, q), P GL(3, q) then the normalizer H of Q is a semidirect product Q ⋊ Cm with a cyclic complement Cm of order m. Here (|Q|, m) is (q, 12 (q − 1)), or (q, q − 1), or (q 3 , (q 2 − 1)/µ), or (q 3 , q 2 − 1) according as W is P SL(2, q), or P GL(2, q), or P SU (3, q) or P GU (3, q) where µ = gcd(3, q + 1). In these cases, W satisfies property (iii) of Result 2.4. Some but not all of these facts remain true for P SL(3, q) and P GL(3, q). For P SL(3, q) with q = dk and d > 2 prime, the normalizer H of a Sylow d-subgroup Q is a semidirect product of Q by Cq−1 × C(q−1)/µ where µ is the greatest common divisor of 3 and q − 1. The same holds for P GL(3, q) with µ = 1 for any q. P SL(3, q) (and hence P GL(3, q)) does not satisfy property (iii) of Result 2.4. Throughout the section d stands for an odd prime and X is an algebraic curve of genus g ≥ 2. Lemma 5.1. Let W be one of the groups on the list (9), W 6= P SL(3, q), P GL(3, q). If W is isomorphic to a subgroup of Aut(X ) with |W | > 900g2, (10) then X has zero p-rank and q is a power of p. Proof. Let H be the normalizer of a Sylow d-subgroup Q of W . For W ∼ = P SL(2, q), we have |W | = (q+1)|H| with |H| = 12 q(q − 1) whence |H|2 > |W | > 900g(X )2 which yields |H| > 30(g(X ) − 1). As 900 > 16, the assertions follow from Lemmas 4.8 and 4.1. Similar arguments apply to the other cases. Recall that Aut(W ) = P ΓL(2, q) when W = P SL(2, q), or W = P GL(2, q) while Aut(W ) = P ΓL(3, q) when W = P SU (3, q) or W = P GU (3, q); see for instance [51, Sections 3.3.4, 3.6.3]. So, W ≤ Aut(W ) may be assumed. 17 Remark 5.2. We point out that in the natural representation of P ΓL(2, q), the 2-point stabilizer is neither abelian nor dihedral. More precisely, a semilinear subgroup of P ΓL(2, q) containing P SL(2, q) is non-abelian and, apart from the smallest value of q, namely q = 9 (and |G| = 720), it is also non-dihedral. The same result is valid for the 2-point stabilizer of P ΓU (3, q) including all cases. In its natural representation, P ΓL(2, q) is the semilinear group on the projective line P G(1, q) over Fq . Its subgroup P SL(2, q) acts on P G(1, q) as a doubly transitive permutation group such that its stabilizer M of the origin O and the infinite point ∞ has order 12 (q − 1) and consists of all permutations x 7→ ax with a non-zero quadratic element a in Fq . The stabilizer of ∞ is H = Q ⋊ M where Q is the Sylow d-subgroup of P SL(2, q) consisting of all translations x 7→ x + c with c ∈ Fq . If a subgroup G of P ΓL(2, q) contains P SL(2, q) then the stabilizer of ∞ in G is G∞,O ⋊ Q where G∞,O is the stabilizer of ∞ and O in G, and M is a normal subgroup of G∞,O . Suppose that G contains P SL(2, q) properly. Then G∞,O has an element v other than those in M . If v ∈ P GL(2, q) then G also contains P GL(2, q) = hP SL(2, q), vi. Since P GL(2, q) is the only linear subgroup of P ΓL(2, q) containing P SL(2, q) properly, it turns out that if G is not linear then G∞,O contains a permutation ϕ : x 7→ αxσ where α ∈ Fq and σ is a non-trivial automorphism of Fq . Then ϕψa 6= ψa ϕ for ψa : x 7→ ax with a ∈ Fq such that σ(a) 6= a. Hence G∞,O is not abelian. We show that G∞,O is neither dihedral for |G| > 720. Since M is a cyclic subgroup of order 1 2 (q − 1)/2 ≥ 4, the index 2 cyclic subgroup of any dihedral group containing M also contains M . As we have already showed the centralizer of M does not contain ϕ. Therefore, if G∞,O is assumed to be dihedral then ϕ is an element in the coset of the index 2 cyclic subgroup of G∞,O . Thus ϕ2 = 1, that is, σ is involutory and ασ+1 = 1. Furthermore, ϕψa ϕ = (ψa )−1 = ψa−1 holds for every a ∈ D. Therefore, aσ+1 = 1 for every non-zero quadratic element a ∈ Fq . Since the number of such elements is 12 (q − 1), this occurs if and only if d = 3, q = 9. Hence, either G = P ΓL(2, 9), and |G| = 1440, or G has order 720. The former case G = P ΓL(2, 9), as P ΓL(2, 9)∞,O is a semi-dihedral group of order 16. In the latter case, G is one of the two subgroups of P ΓL(2, 9) of order 720, other than P GL(2, 9). Therefore, if G∞,O is dihedral then |G| = 720. In its natural representation, P ΓU (3, q) is the semilinear group preserving the set of all Fq2 -rational points of the Hermitian curve H with equation y q + y = xq+1 . Its subgroup P SU (3, q) acts as a doubly transitive permutation group. Let D be the subgroup of index µ in the multiplicative group of Fq2 . Then a Sylow d-subgroup Q of P SU (3, q) consists of all maps (x, y) 7→ (x + a, y + aq x + b) with a, b ∈ Fq2 and bq + b = aq+1 . The subgroup P SU (3, q) acts on H as a doubly transitive permutation group such that its stabilizer M of the origin O = (0, 0) and the infinite point ∞ of the y-axis has order (q 2 − 1)/µ and it consists of all maps ψa,b : (x, y) 7→ (ax, by) with a ∈ D and b = aq+1 . Suppose that a subgroup G of P ΓU (3, q) contains P SU (3, q) properly. Then the stabilizer G∞,O of G contains an element v other than those in M . If v ∈ P GU (3, q) (and µ = 3) then G also contains P GU (3, q) = hP SU (3, q), vi. Since P GU (3, q) is the only linear subgroup of P ΓU (3, q) containing P SU (3, q) properly, it turns out that if G is not linear then G∞,O contains a map ϕ : (x, y) 7→ (αxσ , βy σ ) where α ∈ D, β = αq+1 and σ is a non-trivial automorphism of Fq . Then ϕψa,b 6= ψa,b ϕ for a ∈ Fq such that σ(a) 6= a. Hence G∞,O is not abelian. We show that G∞,O is neither dihedral. Since M is a cyclic subgroup of order (q 2 − 1)/µ ≥ 3, the index 2 cyclic subgroup of any dihedral group containing M also contains M . The centralizer of M does not contain ϕ. Therefore, if G∞,O is dihedral by absurd, then ϕ is in the coset of the index 2 cyclic subgroup of G∞,O . Hence σ is involutory, ασ+1 = 1, and ϕψa,b ϕ = (ψa,b )−1 = ψa−1 ,b−1 holds for every ψa,b ∈ M . From this, aσ+1 = 1 follows for every a ∈ D. Since |D| = (q 2 − 1)/µ this yields dh + 1 = (d2k − 1)/µ with σ = dh . As µ = 1 or µ = 3, this is impossible. Therefore G∞,O is not dihedral. Lemma 5.3. Let G be a subgroup of Aut(W ) containing W where W is one of the groups on the list (9). If G is also a subgroup of Aut(X ) up to isomorphism, and |G| > c · 900g2 18 (11) with c=                              2k(q+1) (q−1)q if W = P SL(2, q), k(q+1) (q−1)q if W = P GL(2, q), 3 2kµ(q +1) (q2 −1)q3 if W = P SU (3, q), 3 2k(q +1) (q2 −1)q3 if W = P GU (3, q), 2 kµ(q +q+1)(q+1) (q2 −1)q3 if W = P SL(3, q), 2 k(q +q+1)(q+1) (q2 −1)q3 if W = P GL(3, q), then G is linear, and W 6= P SL(3, q), P GL(3, q). Proof. We begin with the case W = P SL(2, q). Let Q be a Sylow d-subgroup of P SL(2, q) and H its normalizer in P SL(2, q). We may assume that Q and H are taken as in Remark 5.2. Then T = G∞ is a subgroup of G containing H as a normal subgroup such that T = Q ⋊ G∞,O . Assume that G is nonlinear. From Remark 5.2, G∞,O is not cyclic and it is dihedral only for d = 3 and |G| = 720. On the other hand, k ≥ 2 and |G| = 12 h(q − 1)q(q + 1) where h > 1 is a divisor of k. Assumption |G| > 900cg2 yields |H| = 1 2 (q − 1)q > 30 s c(q − 1)q g > 30g. 2h(q + 1) (12) By (11), H satisfies the hypotheses of Lemma 4.1. By Lemma 4.6, G∞,O is either cyclic or dihedral. Since |G| > 720 this is impossible. Now, let W = P SU (3, q). By the above argument applied to a subgroup G of P ΓL(3, q) containing P SU (3, q) properly, Remark 5.2 shows that T = G∞ is a subgroup of G containing H as a normal subgroup such that T = Q ⋊ G∞,O where G∞,O is neither cyclic nor dihedral when G is nonlinear. On the other hand, |G| = µ1 h(q 2 − 1)q 3 (q 3 + 1) where h is a divisor of 2k. Assumption |G| > 900cg2 yields s c(q 2 − 1)q 3 |H| = µ1 (q 2 − 1)q 3 > 30 g > 30g. (13) µh(q 3 + 1) By (11), H satisfies the hypotheses of Lemma 4.1. From Lemma 4.6, T = Q ⋊ G∞,O where V is either cyclic or dihedral, a contradiction. Finally, let W = P SL(3, q). Then |W | = µ1 h(q − 1)2 q 3 (q 2 + q + 1)(q + 1) where h is a divisor of k. Then |H| = 1 µ (q − 1)2 q 3 > 30 s c(q − 1)2 q 3 g > 30g. µh(q 2 + q + 1)(q + 1) (14) By (10), the hypotheses on H in Lemma 4.1 are satisfied. From Lemma 4.6, T = Q ⋊ V where V is either cyclic or dihedral. However, H contains the direct product of two cyclic groups, and hence V is neither cyclic, nor a dihedral group. This contradiction shows that W 6= P SL(3, q). A similar argument shows that W = P GL(3, q) cannot occur. Remark 5.4. In Lemma 5.3 c ≤ 1 holds in each case. 19 Remark 5.5. In some but not all cases, Lemma 5.3 follows from the Abhyankar conjecture under a bit weaker condition, namely |G| > 24g2 . This condition yields one of the Cases (c) and (d) in Result 2.2. Furthermore, the quotient curve X /G is rational, and the Abhyankar conjecture stated in Introduction applies to G. If G has one short orbit, that is in Case (c), then G is generated by its Sylow p-subgroups and hence G ≥ W implies G = W ; thus G is linear, and Lemma 5.3 follows. If G has two short orbits, that is in Case (d), then G/W is a cyclic group. For instance, this occurs when G ∼ = P GL(2, q) or G ∼ = P GU (3, q). The Abhyankar conjecture rules out the possibility that in Case (d) G properly contains P GL(2, q) or P GU (3, q), as G/W is not cyclic in these cases. The following lemma shows that the hypothesis W 6= P SL(3, q), P GL(3, q) in Lemma 5.1 is meaningful. Lemma 5.6. If W = P SL(3, q) with q > 3 is a subgroup of Aut(X ) then |G| < 72g2 . Proof. As we have already pointed out H = Q ⋊ (Cq−1 × C(q−1)/µ ). Let M be the subgroup of C(q−1)/µ of index 2. Then the factor group T̄ = (Cq−1 × C(q−1)/µ )/M has order 2(q − 1). Here T̄ is a non-cyclic abelian group of order greater than 4. Obviously, the subgroup U = Q ⋊ M is a normal subgroup of H. Therefore, H̄ = H/U is an automorphism group of the quotient curve X¯ = X /U . Since H̄ ∼ = T̄ , no subgroup of P GL(2, K) is isomorphic to H̄ by Result 2.9. Therefore X̄ is not rational. If X¯ is elliptic then U has a short orbit o, and for a point P ∈ o, the contribution of o to the different D(X |X¯ ) is at least |o|(|UP | − 1) ≥ 12 o|UP | = 21 |U |. Assume that o is the unique short orbit of U . Then o is also an H-orbit. Thus |HP |/|UP | = |H|/|U | whence |HP U/U | = |HP U | |HP ||U | 1 |HP | |H| = · = = = |T̄ | = 2(q − 1). |U | |HP ∩ U | |U | |UP | |U | This shows that HP U/U has order 2(q − 1). Furthermore, HP U/U viewed as an automorphism group of X¯ fixes the point P̄ ∈ X¯ lying under o. But then q < 5, as X̄ is elliptic, see Result 2.8. Therefore, there is another such orbit, and the Hurwitz genus formula gives 2(g − 1) ≥ |U | whence g−1≥ 1 2 |U | = 1 q−1 4 µ q3 . This holds true for g(X¯ ) ≥ 2, by the Hurwitz genus formula. Therefore, 72g2 > 72(g − 1)2 ≥ 2 24 (q−1) 6 q 16µ µ ≥ 3 2 2 2 q 3 q 3 (q−1) > (q 2 + q + 1)(q + 1)q 3 (q−1) = |G|, µ µ a contradiction with the hypothesis of the lemma. A variant of the proof of Lemma 5.3 can be used to prove a similar result for the group SL(2, q) with q = dk and d an odd prime. In the natural representation of SL(2, q) as a linear group of the vector space V (2, Fq ), a Sylow d-subgroup is the subgroup Q consisting of all maps (x, y) 7→ (x + ry, y) with r ∈ Fq , and the normalizer of Q in SL(2, q) is the semidirect product Q ⋊ M where M ∼ = Cq−1 consists of all transformations (x, y) 7→ (ax, a−1 y) with a ∈ F∗q . Lemma 5.7. Let G be a group with commutator subgroup G′ = SL(2, q) such that the centralizer of G′ in G has order 2. If G is also a subgroup of Aut(X ), and |G| > 900g2 then X has zero p-rank and q is a power of p. 20 (15) Proof. For every g ∈ G, let ϕg denote the automorphism of G′ taking the element h ∈ G′ to its conjugate hg . The map from G to Aut(G′ ) which takes g to ϕg is a group homomorphism whose kernel consists of the elements in G which centralize G′ . Hence, |G| = 2|L| with a subgroup L of Aut(G′ ). From Aut(SL(2, q)) ∼ = P ΓL(2, q), we have that |G| ≤ 2k(q + 1)q(q − 1) where q = dk and d is an odd prime. Observe that if 4k(q + 1) > (q − 1)q then q = 3, 5, 9, but for these values of q, |G| ≤ 2880 < 3600 ≤ 900g2. Hence 4k(q + 1) ≤ 1. (q − 1)q (16) For the normalizer H = Q ⋊ U of Q in SL(2, q), assumption |G| > 900g2 yields |H| = (q − 1)q > 30 g, (17) which shows that H satisfies the hypotheses in Lemma 4.1. Therefore, d = p and the quotient curve X˜ = X /Q is rational. Take a point P from one of the short orbits Ω1 of H. Then |HP | = |H||QP | = (q − 1)pr with a positive integer r. By absurd, X has positive p-rank and hence (i) of Lemma 4.1 occurs. We prove that Lemma 4.9 applies to G where g is the central involution of G′ . Obviously, g ∈ Z(G) as G′ is a characteristic subgroup of G. It remains to show that the quotient curve X¯ = X /V with V = hgi has zero p-rank. For this purpose, we need three facts: (i) Ḡ = G/V is isomorphic to a subgroup of P ΓL(2, q) containing P SL(2, q). (ii) Ḡ is a subgroup of Aut(X¯ ). (iii) Let H̄ = H/V . If P̄ is a point of X̄ lying under P ∈ Ω1 in the cover X |X̄ , then |H̄P̄ | = 21 (q − 1)pr with a positive integer r. By (16), Lemma 5.3 for W = P SL(2, q) applies to Ḡ. Therefore, either Ḡ = P SL(2, q), or Ḡ = P GL(2, q). If ḡ = g(X¯) ≥ 2 then g − 1 ≥ 2(ḡ − 1) and hence |Ḡ| = 12 |G| ≥ 900g2 ≥ 900 · 21 (2ḡ − 1)2 > 900ḡ2 , which yields ḡ = 0 by Lemma 5.1 applied to Ḡ. If X̄ is elliptic, (iii) together with Result 2.8 yield p = 3 and 14 (q − 1) = |H̄P | ∈ {6, 12} whence q = 9. On the other hand |P GL(2, 9)| = 2|P SL(2, 9)| = 720. Hence |G| ≤ 1440 < 900g2 as g ≥ 2. Therefore, X¯ is not elliptic. 6 Structure of automorphism groups of curves of even genus In this section we prove a classification theorem on the abstract structure of automorphism groups of algebraic curves with even genus. It is worth mentioning that the list of the possible structures is quite short. As a consequence, the inverse Galois problem restricted on curves with even genus is is not solvable in general. Theorem 6.1. If G is a subgroup of the automorphism group of some non-rational algebraic curves with even genus, defined over an algebraically closed field of odd characteristic p, then one of the following cases occurs up to isomorphism: (i) G has odd order; 21 (ii) G = O(G) ⋊ S2 where S2 is a 2-group described in Result 2.11; (iii) the commutator subgroup of G/O(G) is isomorphic to SL(2, q) with q ≥ 5; (iv) P SL(2, q) ≤ G/O(G) ≤ P ΓL(2, q) with q ≥ 3; (v) P SL(3, q) ≤ G/O(G) ≤ P ΓL(3, q) with q ≡ 3 (mod 4); (vi) P SU (3, q) ≤ G/O(G) ≤ P ΓU (3, q) with q ≡ 1 (mod 4); (vii) G/O(G) = Alt7 ; (viii) G/O(G) = M11 ; (ix) G/O(G) = GL(2, 3); (x) G/O(G) is the unique perfect group of order 5040 and (G/O(G))/Z(G) ∼ = Alt7 ; (xi) G/O(G) is the group of order 48 named SmallGroup(48,28) in the GAP-database. In Subsection 6.3 refinements of Theorem 6.1 are obtained under the additional hypothesis that the order of G is large enough compared to genus g of the curve; more precisely when |G| is assumed to exceed 900g(X )2 ; see Proposition 6.21. Throughout the section, X stands for an algebraic curve defined over an algebraically closed field K of odd characteristic p, whose genus g is a (nonzero) even integer, and G denotes a subgroup of Aut(X ). For G of even oder, a key property is given in the following lemma. Lemma 6.2. If G has even order, then any 2-subgroup of G has a cyclic subgroup of index 2. Proof. Let U be a subgroup of Aut(X ) of order 2u ≥ 2. Since p is odd, U is tame. Therefore, the Hurwitz genus formula applied to U gives 2g − 2 = 2u (2ḡ − 2) + m X i=1 (2u − ℓi ) where ḡ is the genus of the quotient curve X̄ = X /U and ℓ1 , . . . , ℓm are the short orbits of U on X . Since 2(g(X )−1) ≡ 2 (mod 4) while 2u (2ḡ−2) ≡ 0 (mod 4), some ℓi (1 ≤ i ≤ m) must be either 1 or 2. Therefore, U or a subgroup of U of index 2 fixes a point of X and hence is cyclic. Remark 6.3. Lemma 6.2 shows that Result 2.11 applies to any 2-subgroup of Aut(X ). For solvable G, Theorem 6.1 is obtained as a corollary of Lemma 6.2 together with Results 2.11 and 2.17. Proposition 6.4. If G is solvable then one of the following cases occurs. (i) G = O(G) ⋊ S2 where S2 is a Sylow 2-subgroup of G. (ii) G/O(G) ∼ = P SL(2, 3), or G/O(G) ∼ = P GL(2, 3). (iii) G/O(G) ∼ = GL(2, 3). If G contains no non-trivial normal subgroup of odd order, then G is an odd core-free group. We first prove Theorem 6.1 in this case. 22 6.1 The case |O(G)| = 1 We show that the hypothesis of g to be even gives a heavy restriction on the structure of the minimal normal subgroups of an odd-core free subgroup of Aut(X ). Lemma 6.5. If G is odd core-free, then a minimal normal subgroup N of G is either a 2-group with |N | ≤ 4, or a non-abelian simple group. Proof. Since N is characteristically simple, if N is solvable then N is an elementary abelian group, otherwise it is a direct product of pairwise isomorphic non-abelian simple groups; see, for instance, [31, Kapitel I, Satz 9.13]. In the former case, N ∼ = Zh2 since G is odd core-free, and h ≤ 2 otherwise N would be a 2-group without a cyclic subgroup of index 2. If N is the direct product of pairwise isomorphic simple groups, let M one of these simple subgroups of N . From Result 2.14, M is neither cyclic nor a generalized quaternion group. By the remarks made after Result 2.11, a Sylow 2-subgroup of N contains an elementary abelian group of order 4, say V . Then M = N , otherwise V and any involution from another factor in that direct product would generate an elementary abelian subgroup of N of order 8, which is a 2-group without a cyclic subgroup of index 2. Three cases are now treated separately, according as a minimal normal subgroup N has order 2, 4, or is a simple group. Lemma 6.6. If G has a (minimal) normal subgroup N of order 2 then G has a 2-complement, unless a Sylow 2-subgroup of G is either a generalized quaternion group, or a semidihedral group. In the semidihedral case, G has an index 2 subgroup M such that either M has a 2-complement, or a Sylow 2-subgroup of M is a generalized quaternion group. If, in addition, G is solvable and has no 2-complement then G/O(G) ∼ = GL(2, 3) and G has a semidihedral Sylow 2-subgroup of order 16. Proof. If a Sylow 2-subgroup S2 of G is dihedral then Result 2.13 yields that G has a normal 2-complement. In fact, N O(G)/O(G) is a non-trivial subgroup of G/O(G) contained in the center of G/O(G) whereas any subgroup of P ΓL(2, q) containing P SL(2, q), as well as Alt7 , have trivial center. By the first claim in Result 2.17, we are left with the case where S2 is a semidihedral group. Then S2 has a cyclic subgroup U of index 2 such that S2 \ U contains an involution v. From Result 2.22, either G has a a subgroup of index 2 containing U , or v is conjugate to an involution in U . The latter case cannot actually occur in our situation, since the normal subgroup N consists of the unique involution of U together with the identity. Therefore, G has a subgroup of index 2, and either (i) or (ii) in Result 2.18 holds. In case (i), G has an index 2 subgroup M with a dihedral Sylow 2-subgroup, and hence M has a 2-complement. In case (ii), G has an index 2 subgroup M with a generalized quaternion Sylow 2-subgroup. If G is solvable, the second claim in Result 2.17 applies. Remark 6.7. Lemma 6.6 applies to SL± (2, q) and SU ± (2, q) and shows that Aut(X ) has no subgroup isomorphic to SL± (2, q) for q ≡ 1 (mod 4) and to SU ± (2, q) for q ≡ −1 (mod 4). Lemma 6.8. If G has a minimal normal subgroup N of order 4, then either G = O(G) ⋊ S2 where S2 is the direct product of a cyclic group by a group of order 2, or G/O(G) ∼ = P SL(2, 3), P GL(2, 3). Proof. Let V be a Sylow 2-subgroup of the centralizer T = CG (N ) of N in G. Then N ≤ V . Since N ∼ = C2 × C2 , Result 2.11 shows that V must be abelian. Hence V ∼ = C2m × C2 with m ≥ 1. From Result 2.17, T has a normal 2-complement. We investigate the case where T is a proper subgroup of G. In this case, G/T is isomorphic to subgroup of Sym3 and hence it is solvable. Since T is also solvable, this yields 23 that G is itself solvable. Since GL(2, 3) has no normal subgroup of order 4, Result 2.17 shows that either G/O(G) ∼ = P GL(2, 3). = P SL(2, 3) or G/O(G) ∼ Lemma 6.9. If G is an odd core-free group and has a non-abelian minimal normal subgroup N then one the following cases occurs. (i) N ∼ = P SL(2, q) with q ≥ 5 odd, and a Sylow 2-subgroup of N is dihedral; (ii) N ∼ = P SL(3, q) with q ≡ 3 (mod 4), and a Sylow 2-subgroup of N is semidihedral; (iii) N ∼ = P SU (3, q) with q ≡ 1 (mod 4), and a Sylow 2-subgroup of N is semidihedral; (iv) N ∼ = Alt7 , and a Sylow 2-subgroup of N is dihedral; (v) N ∼ = M11 , the Mathieu group on 11 letters, and a Sylow 2-subgroup of N is semidihedral. Proof. By Lemma 6.5 N is simple. From Result 2.16, a Sylow 2-subgroup of N is either dihedral, or semidihedral, or wreathed, or isomorphic to a Sylow 2-subgroup of SU (3, 2). Actually, by Lemma 6.2 and Result 2.11, the latter two cases cannot occur in our situation. Now, the claim follows from Results 2.13 and 2.15. Theorem 6.1 for odd core-free groups is a corollary of Lemmas 6.10, 6.11, and 6.13. Lemma 6.10. Let G be an odd core-free group with generalized quaternion Sylow 2-subgroups. Then its commutator subgroup G′ is isomorphic to SL(2, q) with q ≥ 5, apart from three cases: (i) G is a generalized quaternion group, (ii) G is isomorphic to the unique perfect group of order 5040 and G/Z(G) ∼ = Alt7 , (iii) G is isomorphic to the group of order 48 named SmallGroup(48, 28) in the GAP-database. Proof. If G is a 2-group, Case (i) holds. From now on |G| is assumed to have some odd divisor. From Result 2.14, |Z(G)| = 2. We point out that the factor group Ḡ = G/Z(G) is also odd core-free. By absurd, let N̄ be a non-trivial normal subgroup of Ḡ of odd order. Then there exists a normal subgroup N of G such that ϕ(N ) = N̄ where ϕ is the natural homomorphism G → Ḡ. Since G is odd core-free, N has even order. Since the involution z generating Z(G) is the unique involution in G, N must contain z. Since |N̄ | is odd, |N | is twice an odd number bigger than 1. By z ∈ N , this yields N = Z(G) × O(N ). Therefore, O(N ) is non-trivial. Since N is a normal subgroup of G and O(N ) is a characteristic subgroup of N , this yields that O(N ) is a normal subgroup of G, a contradiction as G is odd core-free. Hence Ḡ is odd core-free, as well. Furthermore, a Sylow 2-subgroup of Ḡ is S̄2 = S2 /Z(S) which is a dihedral group as S2 a generalized quaternion group. From Result 2.13, Ḡ is either a dihedral group whose order is a power of 2 in which Case (i) holds, or Ḡ ∼ = Alt7 , or P SL(2, q) ≤ Ḡ ≤ P ΓL(2, q) with q odd, up to isomorphism. For q ≥ 5, P SL(2, q) is a perfect group, while the commutator subgroup of P SL(2, 3) is an elementary abelian group of order 4. If q = 3 then either Ḡ ∼ = SL(2, 3) = P GL(2, 3). By direct GAP-aided computation, G ∼ = P SL(2, 3), or Ḡ ∼ ∼ in the former case while if Ḡ = P GL(2, 3), the only possibility for G is the group SmallGroup(48, 28) in the GAP database. This gives (iii). Suppose q ≥ 5. As G′ is a normal subgroup of G, and G is odd core-free, G′ has even order. Since z is the unique involution of G, z is contained in G′ , that is, Z(G) ≤ G′ . Thus, G′ /Z(G) = G′ Z(G)/Z(G). On 24 the other hand, G′ Z(G)/Z(G) is the commutator subgroup of Ḡ, see [31, Kapitel I, Hilfssatz 8.4], and, since q ≥ 5, either G′ /Z(G) ∼ = Alt7 . = P SL(2, q), or G′ /Z(G) ∼ The commutator subgroup H of G′ is a characteristic subgroup of G′ , and hence a normal subgroup of G. Therefore, H has even order, and hence contains Z(G). From this, either H/Z(G) ∼ = P SL(2, q), or H/Z(G) ∼ = Alt7 . Since H ≤ G′ , this yields H = G′ showing that G′ is a perfect group, and hence G′ is perfect non-split extension of either P SL(2, q) or Alt7 . Now, the first claim in Lemma 6.10 follows from Result 2.19. Finally, Alt7 has a unique non-split central extension by a cyclic group of order 2 which is the group of order 5040 given in (ii). Lemma 6.11. Let G be an odd core-free group with a semidihedral Sylow 2-subgroup S2 . If G has a (minimal) normal subgroup of order 2 then either G = S2 , or its commutator group G′ is isomorphic to SL(2, q). Proof. If G is solvable the assertion follows from Lemma 6.6. Assume that G has even order, and take a minimal normal subgroup N of G such that |N | = 2. Since S2 is semidihedral, the quotient group S2 /N is dihedral. Therefore, the quotient group Ḡ = G/N has a dihedral Sylow 2-subgroup, and Result 2.13 applies. First the case where Ḡ is isomorphic to a subgroup of P ΓL(2, q) containing P SL(2, q) is investigated. In this case, the commutator group Ḡ′ of Ḡ is isomorphic to P SL(2, q) with q ≥ 5. From Lemma 6.6, G has a (normal) subgroup M of index 2, and we may assume that its commutator group M ′ is isomorphic to SL(2, q). Then N ≥ M ′ , and the factor group M̄ = M/N is a subgroup of Ḡ. In particular, the commutator subgroup M̄ ′ is contained in Ḡ′ . Since M̄ is normal in Ḡ and M̄ ′ is a characteristic subgroup of M̄ , we have that M̄ ′ is normal in Ḡ. Therefore, M̄ ′ is a normal subgroup of Ḡ′ . As q ≥ 5, Ḡ′ is simple, and hence M̄ ′ = Ḡ′ . From [31, Kapitel I, Hilfssatz 8.4], Ḡ′ = G′ /N and M̄ ′ = M ′ /N whence G′ = M ′ ∼ = SL(2, q). Finally, since [G : M ] = 2 together with N ≤ M ≤ G yield [Ḡ : M̄ ] = 2, the case Ḡ ∼ = Alt7 does not occur. Lemma 6.12. If G is an odd core-free group with commutator subgroup G′ ∼ = SL(2, q) then the centralizer of G′ in G has order 2. Proof. Since G′ is a normal subgroup of G, the centralizer CG (G′ ) of G′ in G is a normal subgroup of G. As G is odd core-free, CG (G′ ) has even order. Assume by contradiction that |CG (G′ )| > 2. Therefore |CG (G′ )| must be divisible by four, otherwise |CG (G′ )| is twice an odd number and hence CG (G′ ) = Z(G′ )×O(CG (G′ )) where O(CG (G′ )) is a characteristic subgroup of CG (G′ ) of odd order contradicting the hypothesis that G is odd core-free. Therefore, CG (G′ ) has a Sylow 2-subgroup R of order at least 4. Take a Sylow 2-subgroup V of G′ , and let U = V R. Then |Z(U )| ≥ 4. Since V is a generalized quaternion group, we also have |Z(V )| = 2. Therefore, U
V . From |Z(U )| ≥ 4, U is neither a generalized quaternion, nor a dihedral nor a semidihedral group. As we pointed out after Result 2.11, a modular maximal-cyclic group contains no generalized quaternion group. Therefore U
V cannot actually occur, and the assertion follows by absurd. Lemma 6.13. If G is an odd core-free group with a non-abelian simple minimal normal subgroup, then one of the following cases occurs, up to isomorphisms, where q is an odd prime power dk : (i) P SL(2, q) ≤ G ≤ P ΓL(2, q) with q ≥ 5; (ii) P SL(3, q) ≤ G ≤ P ΓL(3, q) with q ≡ 3 (mod 4); (iii) P SU (3, q) ≤ G ≤ P ΓU (3, q) with q ≡ 1 (mod 4); (iv) G = Alt7 ; 25 (v) G = M11 . Proof. Let N be a non-abelian simple minimal normal subgroup of G. We prove that the centralizer CG (N ) of N in G is trivial. Obviously, CG (N ) ∩ N is trivial. If CG (N ) has an involution u then u 6∈ N and hence the 2-subgroup of G generated by u and a Sylow 2-subgroup of N has rank at least 3, a contradiction. Therefore CG (N ) has odd order. Then CG (N ) is trivial, since CG (N ) is a normal subgroup of G, and G is odd core-free. Therefore, G is isomorphic to an automorphism group of N . By Lemma 6.9 all possibilities for N are determined. The cases N ∼ = P SL(2, q) with q ≥ 5, N ∼ = P SL(3, q) and N ∼ = P SU (3, q) give (i), (ii) and (iii), respectively. If N = Alt7 then either G = N , or G ∼ = Sym7 . The latter case cannot actually occur as a Sylow 2-subgroup of Sym7 has 2-rank 3. Hence (iv) holds. Finally, M11 is isomorphic to its automorphism group, and hence (v) holds. Remark 6.14. The subgroups of P ΓL(2, q) containing P SL(2, q) whose Sylow 2-subgroups have 2-rank 2 are P SL(2, q), P GL(2, q) and, when q = dh with an odd prime d and h ≥ 2, the semidirect product of P SL(2, q) or P GL(2, q) with a cyclic group whose order is an odd divisor of h. Analog results are valid for P SL(3, q) and P SU (3, q). These groups are all the candidates for G in (i),(ii),(iii) of Lemma 6.13. It is not true that d must be equal to p. For instance, the Hermitian curve of equation X q+1 + Y q+1 + Z q+1 = 0 √ 1 defined over Fq has genus 2 q(q−1) and if either p = 7 or −7 ∈ Fq it has an automorphism group isomorphic to P SL(2, 7). Remark 6.15. The Hermitian curve of equation X 6 + Y 6 + 1 = 0 defined over F5 has genus 10 and an automorphism group isomorphic to Alt7 . Over an algebraically closed field of characteristic 3, the modular curve X(11) has genus 26, and its automorphism group is isomorphic to M11 ; see [2, 40]. 6.2 The case |O(G)| > 1 To prove Theorem 6.1 for any automorphism groups the following lemma is useful. Lemma 6.16. Let N be any subgroup of G of odd order. If a Sylow 2-subgroup of G fixes no point of X then the quotient curve Y = X /N has also even genus. Proof. If the cover X |Y is unramified then the claim follows from the Hurwitz genus formula applied to N . For a point P̄ ∈ Y where X |Y ramifies, take a point P ∈ X lying over P̄ . Since |N | is odd, all ramification groups of NP at P have odd order. By (2), dP is even. Let θ be the G-orbit containing P . Then, |GP ||θ| = |G|. Take a Sylow 2-subgroup S of G containing a Sylow 2-subgroup SP of GP . Then S 6= SP , as S does not fix P . Therefore, |S| does not divide |GP | showing that |θ| must be even. This yields that dP |θ| is divisible by four. Since N is a normal subgroup of G, the different D(X |Y) is also divisible by four, that is, D(X |Y)/2 is even. Therefore, the Hurwitz genus formula applied to N yields g(X ) − 1 = |V |(g(X¯ ) − 1) + (D(X |Y)/2) whence g(Y) − 1 is odd. Remark 6.17. If a Sylow 2-subgroup of G is not cyclic, then (iii) of Result 2.1 shows that the hypothesis of Lemma 6.16 is satisfied. Lemma 6.18. If |O(G)| > 1 and G/O(G) is a subgroup of P GL(2, K), then Theorem 6.1 holds for G. Proof. By Proposition 6.4, G/O(G) may be supposed to be non-solvable. Then Result 2.9 yields that either G/O(G) ∼ = P SL(2, q), or G ∼ = P GL(2, q) with q ≥ 5. 26 We are now in a position to prove Theorem 6.1 for the case |O(G)| > 1. If G is any subgroup of Aut(X ) then the factor group G/O(G) is an odd core-free automorphism group of the quotient curve X¯ = X /O(G). If a Sylow 2-subgroup fixes a point of X then S2 is cyclic, and hence G = O(G) ⋊ S2 . Dismissing this case allows us to use Lemma 6.16. Doing so, either X̄ is rational and Ḡ = G/O(G) is a subgroup of P GL(2, K), and Lemma 6.18 applies, or g(X¯) ≥ 2 and Lemmas 6.6, 6.8, 6.10, 6.11 and 6.13 apply. 6.3 Refinements of Theorem 6.1 for large G For large automorphism groups compared to the genera of the curves, the list in Theorem 6.1 shortens. For the odd core-free case this is shown in the following lemma. Lemma 6.19. If G be an odd core-free group of automorphisms of a curve X of even genus g(X ) ≥ 2 such that |G| > 900g(X )2 , (18) then X has zero p-rank, and one of the following three cases occurs for G, up to group isomorphisms (i) P SL(2, q), P GL(2, q), q = pk ≥ 5; (ii) P SU (3, q), q = pk and q ≡ 1 (mod 4), P GU (3, q), q = pk and q ≡ 5 (mod 12); (iii) SL(2, q), SL(2) (2, q), SU ± (2, q), q ≡ 1 (mod 4), SL± (2, q), q ≡ −1 (mod 4), q = pk ≥ 5 . Proof. Theorem 6.1 with O(G) = {1} applies. Obviously, |G| is even, otherwise G is not odd-core free, because it is solvable by Result 2.26. If G is solvable, Theorem 6.1 shows that G is a 2-group apart from four sporadic cases, namely when G ∼ = P SL(2, 3), P GL(2, 3), GL(2, 3), SmallGroup(48, 28) whose orders are 12, 24, 48, and 48 respectively. If G is a 2-group then Lemma 6.2 yields that G has a cyclic subgroup N of index 2. From (i) of Result 2.7 applied to N , |G| = 2|N | ≤ 8(g(X ) + 1) whence |G| < 900g(X )2 . Therefore, G is assumed to be non-solvable. The sporadic cases G ∼ = Alt7 , (18) = Alt7 , G ∼ = M11 and G/Z(G) ∼ = Alt7 do not occur. In fact, for G ∼ 2 ∼ yields g(X ) < 4 which contradicts g(X ) ≥ 2. For G = M11 , we have |G| = 7920. From (18), it follows g(X )2 < 9 whence g(X ) = 2. In this case the hyperelliptic involution ω does not belong to G, since G is simple; hence, Ghωi/hωi ∼ = M11 , which contradicts the last claim in Result 2.8. For G/Z(G) ∼ = Alt7 , we have |G| = 5040, and (18) yields g(X ) = 2 which shows that the quotient curve X /Z(G) is rational by Remark 6.3. But this contradicts Result 2.9 as no subgroup of P GL(2, K) is isomorphic to Alt7 . Furthermore, the case P SL(3, q) ≤ G ≤ P ΓL(3, q) is ruled out by Lemma 5.3 and Remark 5.4. Now, from Theorem 6.1 with |O(G)| = 1, either G′ ∼ = SL(2, q), or one of the Cases (iv) or (vi) of Theorem 6.1 hold. In the latter two cases, X has zero p-rank and q is a power of p from Lemma 5.1. Also, by Lemma 5.3 and Remark 5.4, either (i) or (ii) of Lemma 6.19 holds. Hence G′ = SL(2, q) can be assumed. In this case a Sylow 2-subgroup of G is either a generalized quaternion group or a semidihedral group; in fact, the subgroup N = Z(G′ ) generated by the central involution in G′ = SL(2, q) is normal in G since G′ is a characteristic subgroup of G, and hence Lemma 6.6 applies. From Lemmas 6.12 and 5.7, X has zero p-rank. Also, since N is normal in G, the central involution in G′ commutes with each element in G and hence Z(G′ ) ⊆ Z(G). Actually, Z(G′ ) = Z(G) by Lemma 6.12. From Lemma 5.7 d = p, that is, q is a power of p. To prove that Case (iii) must occur, consider the quotient group Ḡ = G/Z(G) regarded as an automorphism group of the quotient curve X¯ = X /Z(G). Since the Sylow 2-subgroups of G are either semidihedral groups or generalized quaternion groups, those of Ḡ are dihedral groups. Furthermore, as G′ ∼ = SL(2, q), 27 ∼ P SL(2, q). Since G′ /Z(G) = G′ Z(G)/Z(G) and, by [31, Kapitel I, Hilfssatz 8.4], we have G′ /Z(G) = ′ ∼ ′ Ḡ = G Z(G)/Z(G), this yields Ḡ′ ∼ = P SL(2, q). First we prove that Ḡ is also odd core-free. By absurd, let N̄ be a non-trivial odd order normal subgroup of Ḡ. The subgroup N of G containing Z(G) such that N/Z(G) = N̄ has order twice an odd number. Hence, its maximal normal subgroup of odd order is non-trivial. Since such a maximal subgroup is a characteristic subgroup of N , it must be a normal subgroup of G, a contradiction as G is odd core-free. Next we prove that Ḡ is isomorphic to a subgroup of P ΓL(2, q) containing P SL(2, q). Since Ḡ is odd core-free and its Sylow 2-subgroups are dihedral, Result 2.13 applies to Ḡ showing that either Ḡ ∼ = Alt7 , or, up to an isomorphism, P SL(2, n) ≤ Ḡ ≤ P ΓL(2, n) with an odd prime power n ≥ 5. As we have already shown, Ḡ′ ∼ = P SL(2, q) is the commutator group of Ḡ. Therefore n = q, since P SL(2, n) is the commutator subgroup of any subgroup of P ΓL(2, n) containing P SL(2, n), The case Ḡ ∼ = Alt7 does not occur. In fact, from 2|Alt7 | = 5040 > 900g(X )2 , we would have g(X ) = 2 which contradicts Result 2.8. To finish the proof, three cases are distinguished according to the value g(X̄ ). If g(X¯ ) ≥ 2 then the Hurwitz genus formula together with (18) yield |Ḡ| > 900g(X̄ )2 . From Lemma 5.3 and Remark 5.4 applied to Ḡ, either Ḡ ∼ = P SL(2, q) or Ḡ ∼ = P GL(2, q). In the former case, |G| = 2|Ḡ| = ′ 2|P SL(2, q)| = |SL(2, q)| = |G | whence G = G′ follows. This gives the first possibility in (iii). In the latter case, G is isomorphic to a central extension of P GL(2, q) by a group of order 2. If this is a split extension then G contains a subgroup T ∼ = P GL(2, q) such that G = Z(G) × T . Then a Sylow 2-subgroup of G is the direct product of a dihedral group and a cyclic group of order 2. But such a direct product contains neither a generalized quaternion group nor a semidihedral group. Therefore, Result 2.21 applies, and either the second, or the third, or the fourth case in (iii) must occur. If g(X̄ ) = 0, Result 2.9 gives the same possibilities for Ḡ, namely Ḡ = P SL(2, q) or Ḡ = P GL(2, q), and the same conclusion can be made. If g(X̄ ) = 1 then the Hurwitz genus formula shows that the set θ of all points fixed by Z(G) has size 2g(X ) − 2. Further, Ḡ leaves θ invariant. For a point P̄ ∈ θ, the stabilizer ḠP of P has order at most 12 by Result 2.8. Therefore, |Ḡ| ≤ 12|θ| whence |Ḡ| ≤ 24(g(X ) − 1). Thus |G| ≤ 48(g(X ) − 1). Comparing with (18) shows that this is impossible. Remark 6.20. The Hermitian curve of equation Y q + Y = X q+1 with q ≡ 1 (mod 4) is an example for (ii). If q ≥ 125, the Roquette curve X of equation Y 2 = X q − X provides an example for (iii) with G ∼ = SL(2, q) where q ≡ 1 (mod 4). The general case |O(G)| ≥ 1 is described in the following proposition. Proposition 6.21. Let G be a non-solvable group of automorphisms of a curve X of even genus g(X ) ≥ 2 such that (18) holds. Then either G/O(G) has a non-abelian simple minimal normal subgroup, and one of the following two cases occurs, up to group isomorphisms (A) G/O(G) ∼ = P SL(2, q), G/O(G) ∼ = P GL(2, q), q = pk ≥ 5; (B) G/O(G) ∼ = P GU (3, q), q = pk and q ≡ 5 (mod 12); = P SU (3, q), q = pk and q ≡ 1 (mod 4), G/O(G) ∼ or (C) G/O(G) ∼ = SL(2, q), SL(2) (2, q), G/O(G) ∼ = SU ± (2, q), q ≡ 1 (mod 4), G/O(G) ∼ = SL± (2, q), q ≡ −1 k (mod 4), where, in all cases, q = p ≥ 5. Proof. G/O(G) is an odd core-free automorphism group of the quotient curve X̄ = X /O(G). If g(X̄ ) ≥ 2, the assertion follows from Lemma 6.19. Furthermore, from Lemma 6.16 and Remark 6.17, X̄ is not elliptic. 28 If X¯ is rational then G/O(G) is a subgroup of P GL(2, q) with q = pk . Since G/O(G) is supposed to be non-solvable, Result 2.9 yields that case (A) occurs with only two possible exceptions for p 6= 5, namely G/O(G) ∼ = P SL(2, 5) and G/O(G) ∼ = P GL(2, 5). We rule out both exceptions. Both P SL(2, 5) and P GL(2, 5) contain a maximal, solvable non-abelian subgroup of index 6. As O(G) is solvable by Result 2.26, G also has a solvable non-abelian subgroup N of index 6. This together with (18) give |N | > 150g(X )2 . As N is solvable, Theorem 1.1 yields that N fixes a point P ∈ X . Since G does not fix P , and N is a maximal subgroup of G, the G-orbit o of P has length 6. By Theorem 1.1, X has zero p-rank. If N has some elements of order p, then a p-subgroup of N fixes P and acts transitively on the remaining 5 points in o. But this yields p = 5, a contradiction. Otherwise, N is a prime to p subgroup, and the classical Hurwitz bound yields |N | < 84(g − 1) whence |G| < 504(g − 1) contradicting (18). Finally, a useful bound on the size of O(G) is given in the lemma below. Lemma 6.22. Let G be a non-solvable group of automorphisms of a curve X of even genus g ≥ 2 such that (18) holds. Let X̄ = X /O(G). If g(X¯) > 0 then 900|O(G)| < |G/O(G)|. Proof. From Lemma 6.16 and Remark 6.17, g(X̄ ) ≥ 2. The Hurwitz genus formula applied to O(G) gives g(X ) − 1 ≥ |O(G)|. This together with (18) show |G/O(G)||O(G)| > 900g(X )2 > 900|O(G)|2 whence the claim follows. 7 Large automorphism groups of curves with even genus An important step toward our second main result is the following theorem. Theorem 7.1. Let G be a non-solvable automorphism group of an algebraic curve X of even genus g(X ) ≥ 2 such that |G| > 900g(X )2 . Then X has zero p-rank. Furthermore, O(G) is a cyclic prime to p subgroup, and either O(G) = Z(G) or [Z(G) : O(G)] = 2 according as Cases (A),(B), or (C) in Proposition 6.21 holds. The proof of Theorem 7.1 requires a careful analysis of the Cases (A),(B),(C) in Proposition 6.21. The basic idea is to find an appropriate upper bound on |O(G)| depending on q so that the linear bound given in Lemma 4.1 may be applied to show that X has zero p-rank. Once this is done, the relationship between O(G) and Z(G) can thoroughly been worked out. 7.1 Proof of Theorem 7.1 in Case (A) of Proposition 6.21 Let L = G′ O(G) where G′ is the commutator subgroup of G. Since G′ O(G)/O(G) ∼ = (G/O(G))′ by [31, ′ Kapitel I, Hilfssatz 8.4], we have that L̄ = L/O(G) is the commutator subgroup Ḡ of Ḡ. In particular L̄ = Ḡ′ ∼ = P SL(2, q), and either Ḡ′ = Ḡ, or Ḡ′ is the unique normal subgroup of Ḡ and Ḡ ∼ = P GL(2, q). Furthermore, either L = G or the index of L in G is equal to 2. Lemma 7.2. If Case (A) in Proposition 6.21 holds then 25 (q − 1)q > . |O(G)|(q + 1) 4 29 (19) In particular, |O(G)| < 4 q 25 (20) and q > 11. Proof. Consider L̄ = L/O(G) as an automorphism group of the quotient curve X¯ = X /O(G). Since L̄ ∼ = P SL(2, q) its Sylow 2-subgroups are dihedral and hence not cyclic. Result 2.1 shows that L fixes no point of X . From Lemma 6.16, g(X¯ ) is even. Therefore, either X¯ is rational, or g(X¯ ) ≥ 2. In the latter case, since L̄ contains an abelian subgroup of odd order q, see Result 2.9, we have 4g(X̄ ) + 3 ≥ q by (i) of Result 2.7. Thus 4g(X̄ ) + 4 ≥ q + 1. If X̄ is not rational, the Hurwitz genus formula applied to O(G) gives g(X ) − 1 ≥ |O(G)|(g(X¯ ) − 1). From (18), g(X )2 (q − 1)q > 900 . |O(G)|(q + 1) Since g(X ) > g(X ) − 1, (q − 1)q > 900 |O(G)|(q + 1)  g(X¯ ) − 1 q+1 2 > 900  g(X̄ ) − 1 4(g(X̄ ) + 1) 2 > 900 25 = , 144 4 whence (19) follows. A direct computation shows that this yields (20) and q > 11. If X¯ is rational, then L̄ acts on X¯ as P SL(2, q) on the projective line P G(1, K). Take any point P̄ ∈ X¯ . As P SL(2, q) is non-solvable, L̄ does not fix P̄ by (iii) of Result 2.1. Hence, the L̄-orbit of P̄ has length at least q. In fact, |L̄P̄ | ≤ 21 q(q − 1) as the order of the largest proper subgroup of P SL(2, q) is 12 q(q − 1) by Result 2.9. Now take a point P ∈ X where the cover X |X¯ ramifies. Then the G-orbit o of P has length at least qu where u = |O(G)|/|O(G)P |. The Hurwitz genus formula applied to O(G) yields 2g − 2 ≥ −2|O(G)| + |o|(|O(G)P | − 1) ≥ −2|O(G)| + |o| 23 |O(G)P | ≥ |O(G)|( 23 q − 2). Hence g(X ) > 31 (q − 3)|O(G)|. From this and (q + 1)q(q − 1)|O(G)| > 900g2 , |O(G)| < 3 9 (q + 1)(q − 1) q< q 900 (q − 3)2 50 Therefore, both (19) and (20) hold for any X̄ . Also, q > 11 must hold. Lemma 7.3. If Case (A) in Proposition 6.21 holds then the quotient curve X¯ = X /O(G) has zero p-rank. Proof. We may assume that X̄ is not rational. As we have already observed in the proof of Lemma 7.2, g(X̄ ) is even by Lemma 6.16. Since L̄ contains a subgroup Q̄ of odd order q ≥ 13, see Result 2.9, if X̄ has positive p-rank then 13 ¯ g(X¯) − 1 ≥ p−2 p q ≥ 3 by Result 2.5 applied to Q̄. Thus we can assume g(X ) ≥ 6 and hence g(X ) > g(X ) − 1 ≥ |O(G)|(g(X¯ ) − 1) ≥ 56 |O(G)|g(X¯ ). As |L̄| ≥ |G| 2|O(G)| holds, taking into account (18) we then have |L̄| ≥ 25 450g(X )2 > 450 · |O(G)|g(X¯ )2 > 900g(X̄ )2 . |O(G)| 36 From Lemma 6.19 applied to L̄, the p-rank of X̄ is equal to zero. 30 (21) Lemma 7.4. If Case (A) in Proposition 6.21 holds then X has zero p-rank. Proof. Lemma 7.3 together with Result 2.4 applied to X¯ show that a Sylow p-subgroup Q̄ of L̄, as well as its normalizer in L̄, fixes a unique point P̄ ∈ X̄ . Here |Q̄| = q, and the normalizer of Q̄ in L̄ is the semidirect product Q̄ ⋊ T̄ with a cyclic group T̄ of order 12 (q − 1) by Result 2.9. Since this normalizer is a maximal subgroup of Ḡ, see again Result 2.9, it is the stabilizer of P̄ in L̄. Let o be the O(G)-orbit lying over P̄ in the cover X |X¯ . The counter-image of Q̄ ⋊ T̄ in the homomorphism L → L̄ is a subgroup N of order 1 2 (q − 1)q|O(G)| where N is the subgroup of G which preserves o. For a point P ∈ o, the stabilizer H of P in N has order 12 (q − 1)q|O(G)P |. Therefore, the (unique) Sylow p-subgroup Q1 of H has order q1 = qpa and a complement of Q1 in H is a cyclic subgroup Ct of order t = 21 (q − 1)b with p ∤ b by (iii) of Result 2.1. Hence H = Q1 ⋊ Ct , and |o|pa b = |O(G)|. Now, since by (18) q(q − 1)(q + 1)|O(G)| > 900g(X )2 , |H|2 > (pa b)2 1 4 (q − 1)q 900g(X )2 . |O(G)|(q + 1) This together with (19) give |H|2 > 900g(X )2 whence |H| = q1 t > 30g(X ). From Lemma 4.1 and Remark 4.3, either X has zero p-rank, or Q1 has just one short orbit θ other than its fixed point P . Observe that θ is contained in o. In fact, each Q1 -orbit in o is short as |o| ≤ |O(G)| < q < q1 by (20). Therefore o = θ ∪ {P }, and hence |o| = 1 + ph for some h ≥ 0. Since |o| divides |O(G)| this implies that |O(G)| is even, a contradiction. Thus X has zero p-rank. To complete our investigation of Case (A) in Proposition 6.21, we only need the following result. Lemma 7.5. If Case (A) in Proposition 6.21 holds then O(G) is a prime to p cyclic group, and Z(G) = O(G). Proof. For every h ∈ G the map ϕh taking u ∈ O(G) to h−1 uh is an automorphism of O(G), and hence the map Φ : h 7→ ϕh is a homomorphism from G into Aut(O(G)). Take a Sylow p-subgroup Q of G. From Lemma 7.4 and Result 2.4, Q fixes a unique point P ∈ X but no non-trivial element of Q fixes a point other than P . If g is any element in O(G), then (20) yields that the Q-orbit of g in the action of Φ has length smaller than |Q|. Hence, g commutes with a non-trivial element of Q. Therefore, g fixes P , and hence O(G) does, as well. Repeating this argument with another Sylow p-subgroup of G, it turns out that O(G) fixes a point P ′ ∈ X other than P . From (i) of Result 2.4, O(G) contains no element of order p. Therefore, from (iii) of Result 2.1, O(G) is a cyclic prime to p group. In particular, ker(Φ) ≥ O(G). Since Ḡ is either simple, or L̄ ∼ = P SL(2, q) is the only non-trivial normal subgroup of Ḡ, two cases arise namely ker(Φ) = O(G) and ker(Φ) ≥ L. In the former case, Φ induces a faithful action of L̄ on O(G). If Λ is a non-trivial orbit then |Λ| < q by (20). On the other hand, since q > 11, Result 2.9 yields |Λ| ≥ q + 1, a contradiction. In the latter case, ker(Φ) ≥ L and hence O(G) ⊆ Z(L); actually, O(G) = Z(L) since Z(L̄) is trivial. As we have shown, O(G) fixes a point P ∈ X . Since γ(X ) = 0, (ii) of Result 2.4 shows that GP coincides with the normalizer NG (Q) of Q in G. From Result 2.12, a complement C of Q contains O(G). If L  G then C contains an element of G \ L. Therefore, some element in G \ L centralizes O(G). This shows that Z(G) = O(G), since Z(Ḡ) is trivial. 31 7.2 Proof of Theorem 7.1 in Case (B) in Proposition 6.21 The quotient curve X̄ = X /O(G) is not rational as Ḡ = G/O(G) is an automorphism group of the quotient curve X¯ = X /O(G) but P GL(2, K) has no subgroup isomorphic to Ḡ ∼ = P SU (3, q) by Result 2.9. Also, X¯ ¯ is neither elliptic by Lemma 6.16. Then g(X ) > 2 by Result 2.8. In particular, g(X̄ ) > 2. From |O(G)| ≥ 3 and g(X ) > g(X ) − 1 ≥ |O(G)|(g(X¯ ) − 1), we have by (18) |Ḡ| = |G| 900 |O(G)| ¯ 2 > g(X )2 > 900|O(G)|(g(X¯ ) − 1)2 ≥ 900 g(X ) > 900g(X̄ )2 |O(G) |O(G)| 3 which together with Lemma 5.1 show that X¯ has zero p-rank. Let L = G′ O(G) where G′ is the commutator subgroup of G. By [31, Kapitel I, Hilfssatz 8.4], L̄ = L/O(G) is the commutator subgroup Ḡ′ of Ḡ. In particular L̄ = Ḡ′ ∼ = P SU (3, q), and either Ḡ′ = Ḡ, or Ḡ′ is the ∼ unique proper normal subgroup of Ḡ and Ḡ = P GU (3, q). First we show how the arguments used in Section 7.1 can be adapted as far as (q 2 − 1)q 3 > µ2 (22) |O(G)|(q 3 + 1) where, as in Result 2.10, µ = gcd(3, q + 1). Lemma 7.6. If (22) is assumed then Theorem 7.1 holds. Proof. Since the p-rank of X̄ is equal to zero, (i) of Result 2.4 shows that a Sylow p-subgroup Q̄ of L̄ fixes a unique point P̄ ∈ X¯ . The normalizer of Q̄ in L̄ is the semidirect product Q̄ ⋊ T̄ where Q̄ is a non-abelian p-group of order q 3 while T̄ is a cyclic group of order (q 2 − 1)/µ. From Result 2.10, this normalizer is a maximal subgroup of L̄. Hence it coincides with the stabilizer of P̄ in L̄. Let o be the O(G)-orbit lying over P̄ in the cover X |X¯ . The counter-image of Q̄ ⋊ T̄ in the homomorphism L → L̄ is a subgroup N of order (q 2 −1)q 3 |O(G)|/µ. For a point P ∈ o, the stabilizer of P in N is a subgroup H of order (q 2 −1)q 3 |O(G)P |/µ. Therefore, the (unique) Sylow p-subgroup Q1 of H has order q1 = q 3 pa and a complement of Q1 in H is a cyclic subgroup Ct of order t = b(q 2 − 1)/µ with p ∤ b. Hence H = Q1 ⋊ Ct , and |o|pa b = |O(L)|. From (18), (q 2 − 1)q 3 > Therefore, |H|2 = (q1 t)2 > (pa b)2 This together with (22) give 900g(X )2 . + 1)|O(G)| (q 3 (q 2 − 1)q 3 900g(X )2 µ2 |O(G)|(q 3 + 1) |H| > 30g(X ). From Lemma 4.1 and Remark 4.3, either X has zero p-rank, or Q1 has just one short orbit θ other than its fixed point P . Observe that θ is contained in o. In fact, each Q1 -orbit in o is short as |o| ≤ |O(G)| < (q 2 − 1)/µ2 < q 3 < q1 by (22). Therefore o = θ ∪ {P }, and hence |o| = 1 + ph for some h ≥ 0. Since |o| divides |O(G)| this implies that |O(G)| is even, a contradiction. Thus X has zero p-rank. Finally, O(G) = Z(G) and O(G) is a cyclic prime to p group. We point out that this can be shown by the arguments used in the final part of the proof of Lemma 7.5. It is enough to deal with the case where Φ induces a faithful action of L̄ ∼ = P SU (3, q) on O(G). If Λ is a non-trivial orbit then |Λ| < |O(G)| < (q 2 − 1)/µ2 by (22). On the other hand, |Λ| is at least as large as the index of the largest subgroup of PSU(3, q), and, from Result 2.10, |Λ| ≥ q 3 + 1 for q 6= 5 while |Λ| ≥ 60 for q = 5. In both cases, this gives a contradiction. Next we show that (22) is not heavily restrictive. Throughout this section, let X¯ = X /O(G). 32 7.2.1 The case of ramified cover X |X¯ Lemma 7.7. If the cover X |X¯ ramifies then (22) holds. Proof. Take a ramification point P̄ ∈ X¯. As P SU (3, q) is non-solvable, Ḡ does not fix P̄ by (iii) of Result 2.1. Let ō be the Ḡ-orbit of P̄ . Note that q 6= 3 by q ≡ 1 (mod 4). From Result 2.10, either |ō| ≥ q 3 + 1, or |ō| ≥ 60 and q = 5. Assume q 6= 5. Then the cover X |X¯ has at least q 3 + 1 ramification points with the same ramification index. If the O(G)-orbit lying over P̄ has length ℓ then |O(G)| − ℓ ≥ 32 |O(G)|. Recall that g(X̄ ) > 2 . Therefore, if q 6= 5 then 2g(X ) − 2 > (q 3 + 1)(|O(G)| − ℓ) ≥ |O(G)| 23 (q 3 + 1), (23) whence g(X ) > g(X ) − 1 > 49 (q 3 + 1)|O(G)|. This together with (18) give 900((q 3 + 1)|O(G)| 49 )2 < 900g(X )2 < |O(G)|(q 3 + 1)(q 2 − 1)q 3 and hence (22) holds. Similarly, if q = 5 we have g(X ) > g(X ) − 1 > 20|O(G)|, and 900(20|O(G)|)2 < 900g(X )2 < |O(G)| · |P GU (3, 5)| gives (22) for q = 5. By Lemmas 7.6 and 7.7, Theorem 7.1 holds for ramified covers X |X̄ . We will show below that the unramified case cannot actually occur. Unfortunately, this case appears to be more involved and our proof requires deeper results from Group theory. 7.2.2 The case of unramified cover X |X¯ It has alredy been noticed that X¯ has zero p-rank and that g(X̄ ) > 2. Also, by Lemma 6.16 the genus of X¯ is even. Lemma 7.8. If the cover X |X¯ is unramified then O(G) is not charactersitically simple. Proof. Assume on the contrary that O(G) is characteristically simple. From Result 2.22 O(G) is solvable. Hence O(G) is the direct product of r ≥ 1 cyclic groups of odd prime order v; see [31, Kapitel I, Satz 9.12 ]. We show that v 6= p. Let Q̄ be a Sylow p-subgroup of Ḡ. From (i) of Result 2.4, there is a point P̄ ∈ X¯ which is fixed by Q̄. Assume on the contrary that v = p. Then |O(G)| = pr . Let Q be a Sylow p-subgroup of G containing O(G) such that Q̄ = Q/O(G). Let o be the O(G)-orbit lying over P̄ in the cover X |X¯ . Since |o| = pr , for each P ∈ O the stabilizer of P in Q has order q 3 = |Q̄|. On the other hand, no non-trivial element of Q fixes a point in X \ o. In fact, if h ∈ Q fixes a point P ′ in X \ o, then h∈ / O(G) since X |X¯ is unramified. Let h̄ be the image of h under the homomorphism G → Ḡ. Then h̄ is not the trivial automorphism of X¯ , and fixes two distinct points, namely P̄ and the point P̄ ′ lying under P ′ in the cover X |X̄ . Since h̄ is a p-element, the p-rank of X¯ is not zero by (i) of Result 2.4, a contradiction. 33 Note that X¯ = X /O(G) is an unramified p-cover of X¯/Q, and hence the p-rank of X /Q is zero. From the Deuring-Shafarevich formula applied to Q, γ(X ) − 1 = −|Q| + |O(G)|(q 3 − 1) = −pr q 3 + pr (q 3 − 1) = −pr , a contradiction. Thus, v 6= p. Let Φ denote the homomorphism from G to the automorphism group of O(G) arising from conjugacy. Since O(G) is abelian, ker(Φ) contains O(G). Two cases are distinguished according as ker(Φ) = O(G) or ker(Φ) contains O(G) properly. In the former case, Φ gives rise to a faithful representation of Ḡ by a matrix group of the vector space V = Frv . As we have shown at the beginning of the proof, no non-trivial normal subgroup of G is contained in O(G). Therefore, Φ is irreducible. Let P G(r − 1, v) be the projective space arising from V . Since L̄ ∼ = PSU(3, q) is simple, L̄ also has an irreducible, faithful representation as a projective group of PG(r − 1, v). Since v 6= p, this yields r − 1 ≥ q(q − 1) by Result 2.23. Hence |O(G)| ≥ v q(q−1)+1 . But this contradicts Lemma 6.22. In the latter case, ker(Φ)/O(G) is a proper normal subgroup of Ḡ. Since Ḡ ∼ = P SU (3, q), or Ḡ ∼ = P GU (3, q), this yields that ker(Φ) contains L, that is, O(G) ⊆ Z(L); actually O(G) = Z(L) holds since the center of L̄ is trivial. Two cases arise according as O(G) ≤ G′′ or O(G)  G′′ . If G′′ contains O(G), then Z(G′ ) = O(G) and G′ /O(G) = L̄ ∼ = P SU (3, q). Therefore, G′ is a perfect ′ central extension of P SU (3, q). From Result 2.20, G = SU (3, q). In particular, |Z(G′ )| = 3 whence |O(G)| = 3. This shows that (22) holds. From Lemma 7.6, X has zero p-rank. As any p-subgroup of G is contained in G′ , we have that a Sylow p-subgrop of G, say Sp is contained in G′ . Since X has zero p-rank, Sp fixes a point P in X and no non-trivial element of Sp fixes another point; see Result 2.4. Each h ∈ O(G) commutes with every element in Sp , and hence fixes P , which contradicts the hypothesis that the cover X |X¯ is unramified. If G′′ does not contain O(G) then they intersect trivially as O(G) contains no proper normal subgroup of G. Therefore, G′′ × O(G) is a normal subgroup of G. Since P SU (3, q) ≤ G/O(G) ≤ P GU (3, q) up to an isomorphism, this implies G′′ ∼ = P SU (3, q). In particular, any p-subgroup of G is contained in G′′ . Take a point P̄ ∈ X¯. Since X¯ has zero p-rank, (i) of Result 2.4 yields that P̄ is fixed by a Sylow p-subgroup Q̄ of Ḡ. The O(G)-orbit o lying over P̄ is left invariant by a subgroup Q of G such that Q/O(G) = Q̄. Since |O(G)| is prime to |Q̄| = q 3 , Result 2.12 shows that Q = O(G) ⋊ R where |R| = |Q̄| with R ≤ G′′ , and that R fixes a point P ∈ o. Since O(G) is contained in the centralizer of G′ in G, this yields that R fixes o pointwise. From the Deuring-Shafarevich formula applied to R, γ(X ) ≥ −q 3 + |O(G)|(q 3 − 1) + 1 > 13 (q 3 + 1)|O(G)|. (24) This together with g(X ) ≥ γ(X ) yields (22). From Lemma 7.6, γ(X ) = 0, but this contradicts (24). Lemma 7.9. The cover X |X¯ is ramified. Proof. Assume that X is a counter-example with minimal genus. Since by Lemma 7.8 O(G) is not characteristically simple, G has a normal subgroup N properly contained in O(G). Consider the quotient curve X̃ = X /N . From N < O(G), we have 4 ≤ g(X̄ ) < g(X˜ ). From the Hurwitz genus formula applied to N , g(X ) > g(X ) − 1 = |N |(g(X˜ ) − 1) ≥ 54 |N |g(X̃ ). 34 Since G̃ = G/N is a subgroup of Aut(X̃ ), and |N | ≥ 3, |G̃| > 900 · 16 2 146 |N |g(X̃ )2 ≥ · 900g(X̃ )2 . 25 25 Note that G̃ is a non-solvable group and that the quotient group O(G)/N is the largest normal subgroup O(G̃) of G̃ of odd order. Then X̄ = X̃ /O(G̃) and the cover X̃ |X¯ is unramified. Therefore, X˜ with respect to G̃ satisfies Condition (B) in Proposition 6.21. As X is assumed to be a counter-example with minimal genus, we obtain a contradiction. 7.3 Proof of Theorem 7.1 in Case (C) in Proposition 6.21 We use the idea worked out in Section 7.1. Note that in this case the commutator subgroup Ḡ′ contains SL(2, q). Let L be the subgroup of G such that L̄ = SL(2, q). Lemma 7.10. In Case (C) of Proposition 6.21, X has zero p-rank. Proof. As L̄ ∼ = SL(2, q) is a subgroup of Aut(X¯ ), Result 2.9 yields that X̄ is not rational. Furthermore, since a Sylow 2-subgroup of SL(2, q) is not cyclic, Lemma 6.16 and Remark 6.3 yield that X̄ is neither elliptic. Therefore, g(X¯ ) ≥ 2. If equality holds then Result 2.5 applied to g(X¯ ) = 2 yields γ(X¯ ) = 0, with just one exception, namely when p = 3, and a Sylow p-subgroup of L̄ has order 3. This exception does not actually occur here as q ≥ 5 in Case (C). Therefore, X¯ has zero p-rank. We show that this holds true for g(X¯) > 2. From the Hurwitz genus formula applied to O(G), g(X ) − 1 ≥ |O(G)|(g(X¯) − 1). This and (18) give |Ḡ||O(G)| > 900(g(X̄ ) − 1)2 |O(G)|2 ≥ 900g(X̄ )2 49 |O(G)|2 ≥ 900g(X̄ )2 |O(G)| whence |Ḡ| > 900g(X̄ )2 . From Lemma 6.19, X̄ has zero p-rank. Furthermore, since |Ḡ| ≤ 2(q + 1)q(q − 1) and SL(2, q) has an abelian subgroup of order q + 1, 2(q − 1)q |G| > 900 ≥ |O(G)|(q + 1) (|O(G)|(q + 1))2  g(X ) − 1 |O(G)|(q + 1) This yields |O(G)| < 2 ≥ 900  g(X¯ ) − 1 4(g(X̄ ) + 1) 2 ≥ 32 q. 225 225 900 = . 64 16 (25) (26) Also, q ≥ 23 holds. From (i) of Result 2.4, a Sylow p-subgroup Q̄ of L̄ ∼ = SL(2, q) fixes a unique point P̄ ∈ X¯ . Here |Q̄| = q, and the normalizer of Q̄ in L̄ is the semidirect product Q̄ ⋊ T̄ with a cyclic group T̄ of order q − 1. Since this normalizer is a maximal subgroup of L̄, it is the stabilizer of P̄ in L̄ otherwise Ḡ would fix P̄ contradicting (iii) of Result 2.1. Let o be the O(G)-orbit lying over P̄ in the cover X |X¯ . The counter-image of Q̄⋊ T̄ in the homomorphism G → Ḡ is a subgroup N of order (q − 1)q|O(G)| where N is the subgroup of G which preserves o. For a point P ∈ o, the stabilizer H of P in N has order (q − 1)q|O(G)P |. Therefore, the (unique) Sylow p-subgroup Q1 of H has order q1 = qpa and a complement of Q1 in H is a cyclic subgroup Ct of order t = (q − 1)b with p ∤ b. Hence H = Q1 ⋊ Ct , and |o|pa b = |O(G)|. Now, (18) yields q(q − 1)pa b(q + 1)|o| > 450g(X )2 whence |H|2 = (q1 t)2 > 2(q − 1)q (q − 1)qpa b 450g2 = 41 (pa b)2 900g(X )2 . (q + 1)|o| |O(G)|(q + 1) 35 This together with (25) give |H| = q1 t > 30g(X ). From Lemma 4.1 and Remark 4.3, either X has zero p-rank, or Q1 has just one short orbit θ other than its fixed point P . Observe that θ is contained in o. In fact, each Q1 -orbit in o is short as |o| ≤ |O(G)| < q < q1 by (26). Therefore o = θ ∪ {P }, and hence |o| = 1 + ph for some h ≥ 0. Since |o| divides |O(G)| this implies that |O(G)| is even, a contradiction. Thus X has zero p-rank. Lemma 7.11. In Case (C) of Proposition 6.21, Z(G) is a cyclic prime to p group and contains O(G) as an index 2 subgroup. Proof. We adapt the arguments used in Section 7.1. For this purpose, let M be the normal subgroup of G containing O(G) such that M/O(G) = Z(G/O(G)). Since Ḡ = G/O(G) is isomorphic to either SL(2, q), or SL(2) (2, q), or SL± (2, q), or SU ± (2, q), we have |Z(Ḡ)| = 2. Thus |M | = 2|O(G)|. Since L̄ ∼ = SL(2, q), we have that L̄ contains Z(Ḡ). Hence M ≤ L, and L̂ = L/M ∼ = P SL(2, q). From = (L/O(G))/(M/O(G)) ∼ Lemma 6.19 applied to X /M , we have that Ĝ = G/M is isomorphic to either P SL(2, q) or P GL(2, q) according as Ĝ = L̂ or [Ĝ : L̂] = 2. For every h ∈ G the map ϕh taking u ∈ M to h−1 uh is an automorphism of M. Thus the map Φ : h 7→ ϕh is a homomorphism from G into Aut(M ). Now, after replacing O(G) by M and Lemma 7.5 by Lemma 7.10, the analog argument used in the proof of Lemma 7.5 shows that M is a cyclic prime to p group. In particular, ker(Φ) ≥ M . Since either Ĝ = L̂, or L̂ is the only non-trivial normal subgroup of Ĝ, two cases arise namely ker(Φ) = M and ker(Φ) ≥ L. In the former case, Φ induces a faithful action of L̂ on M . If Λ is a non-trivial orbit then |Λ| < 2|O(G)|, and hence |Λ| < q by (26). On the other hand, |Λ| ≤ q + 1 by the analog argument in the proof of Lemma 7.5; a contradiction. In the latter case, ker(Φ) ≥ L and hence Z(L) = M . Let Q be a Sylow p-subgroup of G. Then Q ≤ L as either G = L or [G : L] = 2. From Lemma 7.10 and (i) of Result 2.4, Q fixes a unique point P . Since Z(L) = M , M fixes P . From Lemma 7.10 and (ii) of Result 2.4, GP coincides with the normalizer NG (Q) of Q in G. From Result 2.12, a complement C of Q contains M . If L  G then C contains an element of G \ L. Therefore, some element in G \ L centralizes O(G). This shows that Z(G) = M . 8 Proof of Theorem 1.2 From Theorem 7.1, γ(X ) = 0. If O(G) = {1}, Theorem 1.2 is a corollary of Lemma 6.19, taking into account that for the groups G listed in Lemma 6.19 p(G) is P GL(2, q), P SL(2, q) or SL(2, q). Therefore, O(G) is assumed to be non-trivial. From (i) of Result 2.4, each Sylow p-subgroup fixes a point of X . Therefore, the set Ω consisting of the fixed points of the Sylow p-subgroups of G is non-empty. There is a bijection between Ω and the set of all Sylow p-subgroups of G. Since the Sylow p-subgroups of G are pairwise conjugate in G, Ω is a G-orbit. Therefore, G has a transitive permutation representation Φ on Ω. Let G̃ = G/ ker(Φ). Then Z(G) ≤ ker(Φ) as Z(G) fixes Ω pointwise. Also, from Theorem 7.1, Z(G) is a prime to p subgroup, and hence Sylow p-subgroups of G are isomorphic to those of G/Z(G). Note that ker(Φ) is a proper subgroup of G as otherwise G would have just one Sylow p-subgroup. If Case (A) or Case (B) in Proposition 6.21 holds, then Ḡ = G/O(G) has trivial center and Z(G) = O(G) by Theorem 7.1. Therefore, Z(G) = ker Φ since ker(Φ)/O(G) is a normal subgroup of Ḡ. Then G̃ = Ḡ acts (by conjugation) on Ω faithfully in the same way as either P SL(2, q), or P GL(2, q), or P SU (3, q), or P GU (3, q), on the set of their Sylow p-subgroups, that is, in their usual doubly transitive permutation 36 representation. In particular, q = |Ω| − 1 for P SL(2, q) and P GL(2, q) while q 3 = |Ω| − 1 for P SU (3, q) and P GU (3, q). If case (C) in Proposition 6.21 occurs, then either G/Z(G) ∼ = P SL(2, q), or G/Z(G) ∼ = P GL(2, q), and [Z(G) : O(G)] = 2 by Theorem 7.1. In the former case, G/Z(G) is simple, in the latter one G/Z(G) has a unique proper normal subgroup, and this subgroup is isomorphic to P SL(2, q). Therefore, ker(Φ) = Z(G), and G̃ acts on Ω faithfully as either P SL(2, q), or P GL(2, q), on the set of their Sylow p-subgroups, that is, in their usual doubly transitive permutation representation. Let Sp be a Sylow p-subgroup of G, and look at the action of Sp on Ω. Obviously Sp fixes the point P ∈ Ω represented by Sp itself. From (i) of Result 2.4, no non-trivial element of Sp fixes another point, that is, Sp acts semiregularly on the points of Ω other than P . Actually, Sp is regular on Ω \ {P }, as |Sp | = |Ω| − 1. Furthermore, from (i) of Result 2.1, Sp is a normal subgroup of the stabilizer GP of P in G. Therefore, the pair (Ω, G) is a group space satisfying the hypothesis of Result 2.24 where the normal closure S of Q is the subgroup p(G) of G generated by all Sylow p-subgroups of G. In Result 2.24, there are several possibilities for S and hence for our p(G), but those consistent with Proposition 6.21 are only four, namely P SL(2, q), SL(2, q), P SU (3, q) with q ≡ 1 (mod 4) and SU (3, q) with q ≡ 5 (mod 12). This completes the proof of Theorem 1.2. 9 Examples for Theorem 1.2 In Remark 6.20 we have already pointed out that Hermitian curves and Roquette curves provide example for the cases p(G) = P SU (3, q), P SL(2, q), SL(2, q). We show that the GK curves [12] have quotient curves that provide more examples for Theorem 1.2. Let q be a power of p such that q ≡ 1 (mod 4). From [12], the GK curve Z has genus g(Z) = 21 (q 3 + 1)(q 2 − 2) + 1 which is even for q ≡ 1 (mod 4), and Aut(Z) has order (q 3 + 1)q 3 (q 2 − 1)(q 2 − q + 1). More precisely, Aut(Z) ∼ = P SU (3, q) × Cq2 −q+1 for q 6≡ −1 (mod 3) while Aut(Z) has an index 3 subgroup G ∼ = SU (3, q) × C(q2 −q+1)/3 for q ≡ −1 (mod 3); see [12]. In particular, |Aut(Z)| ≈ 8g(X )2 . Furthermore, Aut(Z) has exactly two short orbits. One of them has size q 3 + 1 and the kernel of the permutation representation of Aut(Z) on it is the cyclic subgroup Cq2 −q+1 of Aut(Z) while no non-trivial element of Cq2 −q+1 fixes a point in the other short orbit. The quotient curves of Z have been under investigations; see [10, 17]. Here we limit ourselves to a special case, also discussed in [12]. For a divisor u of q 2 − q + 1, the group Cq2 −q+1 contains a subgroup Cu of order u. Let Xu = Z/Cu be the quotient curve of X with respect to Cu . Since Cu is tame, the Hurwitz genus formula gives (q 3 + 1)(q 2 − 2) = 2g(Y) − 2 = u(2g(Xd ) − 2) + (d − 1)(q 3 + 1), whence g(Xd ) = 1 2   (q 3 + 1)(q 2 − u − 1) +2 . u In particular, g(Xu ) is even. Furthermore, since Cu is a normal subgroup of Aut(X ), Aut(X )/Cu is a subgroup Gu of Aut(Xu ) such that |Gu | = q 3 (q 3 + 1)(q 2 − 1)(q 2 − q + 1) . u Comparison |Gu | with g(Xd ) shows that if u ≥ 450 then |Aut(Xu )| ≥ |Gu | > 900g(Xu )2 . 37 ∼ C(q2 −q+1)/u are ∼ P SU (3, q) and Cq2 −q+1 /Cu = If q 6≡ −1 (mod 3) then both (P SU (3, q) × Cu )/Cu = subgroups of Gu , and their intersection is trivial. Therefore, they generate a group isomorphic to P SU (3, q)× C(q2 −q+1)/u whose order is equal to that of Gu . Thus Gd ∼ = P SU (3, q) × C(q2 −q+1)/u . This gives an example, not isomorphic to the Hermitian curve, for the case p(G) ∼ = P SU (3, q) with q ≡ 1 (mod 3) in Theorem 1.2. If q ≡ −1 (mod 3), take the index 3 subgroup G ∼ = SU (3, q) × C(q2 −q+1)/3 together with a subgroup Cu of C(q2 −q+1)/3 . Then u is not divisible by 3 and we may replace Gu = Aut(X )/Cu with G/Cu in the above argument. This time we obtain G/Cu ∼ = SU (3, q) × C(q2 −q+1)/3u which gives an example for the case p(G) ∼ = SU (3, q) with q ≡ −1 (mod 3) in Theorem 1.2. References [1] S. Abhyankar, Coverings of algebraic curves, Amer. J. Math., 79 (1957), 825-856. [2] A. Adler, The Mathieu group M (11) and the modular curve X(11), Proc. London Math. Soc. 74 (1997), 1-28. [3] L. Alperin, R. Brauer and D. Gorenstein, Finite simple groups of 2-rank two, Scripta Math. 29 (1973), 191-214. [4] J.L. Alperin, R. Brauer and D. Gorenstein, Finite groups with quasi-dihedral and wreathed Sylow 2-subgroups, Trans. Amer. Math. Soc. 151 (1970), 1-261. [5] N. Arakelian and G. Korchmáros, A characterization of the Artin-Mumford curve, J. Number Theory 154 (2015), 278-291. [6] F.M. Bleher, T. Chinburg, B. Poonen and P. Symonds, Automorphisms of Harbater-Katz-Gabber curves, arXiv:1509.02139. [7] I.I. Bouw, The p-rank of ramified covers, Compositio Math. 126 (2001), 209–322. [8] R. Brauer and M. Suzuki, On finite groups of even order whose 2-Sylow group is a quaternion group, Proceedings of the National Academy of Sciences of the United States of America 45 (1959), 1757-1759. [9] G. Cornelissen, F. Kato and A. Kontogeorgis, Discontinuous groups in positive characteristic and automorphisms of Mumford curves, Math. Ann. 320 (2001), 55-85. [10] S. Fanali and M. Giulietti, Quotient curves of the GK curve, Adv. Geom. 12 (2012), 239-268. [11] W. Feit and J.G. Thompson, Solvability of groups of odd order, Pac. Jour. Math. 13 (1963), 775-1029. [12] M. Giulietti and G. Korchmáros, A new family of maximal curves, Math. Ann. (2009) 343, 229-245. [13] M. Giulietti and G. Korchmáros, Large 2-groups of automorphisms of curves with positive 2-rank, J. Algebra, 427 (2015), 264-294. [14] M. Giulietti and G. Korchmáros, Algebraic curves with a large non-tame automorphism group fixing no point, Trans. Amer. Math. Soc. 362 (2010), 5983–6001. [15] M. Giulietti and G. Korchmáros, Automorphism groups of algebraic curves with p-rank zero, J. London Math. Soc., (2) 81 (2010) 277-296. 38 [16] M. Giulietti and G. Korchmáros, Large p-groups of automorphisms of algebraic curves in characteristic p, J. Algebra 481 (2017), 215-249. [17] M. Giulietti, L. Quoos, Luciane and G. Zini, Maximal curves from subcovers of the GK-curve, J. Pure Appl. Algebra 220 (2016), 3372-3383. [18] D. Gorenstein and K. Harada, Finite Groups Whose 2-Subgroups Are Generated by at Most 4 Elements, Memoirs of the American Mathematical Society, No. 147, American Mathematical Society, Providence, R.I., 1974. vii+464 pp. [19] D. Gorenstein, Finite groups, Second edition. Chelsea Publishing Co., New York, 1980. xvii+519 pp. ISBN: 0-8284-0301-5 20-02 (20Dxx). [20] D. Gorenstein and J.H. Walter, The characterization of finite groups with dihedral Sylow 2-subgroups. I. J. Algebra 2 (1965), 85-151. [21] D. Gorenstein and J.H. Walter, The characterization of finite groups with dihedral Sylow 2-subgroups. II. J. Algebra 2 (1965), 218-270. [22] R. Gow, Some T I subgroups of solvable groups, J. London Math. Soc. 12 (1976), 285–286. [23] R. L. Griess, Jr., Schur multipliers of finite simple groups of Lie type, Trans. Amer. Math. Soc., 183 (1973), 355–421. [24] J.P. Hansen and J.P. Pedersen, Automorphism group of Ree type, Deligne-Lusztig curves and function fields, J. Reine Angew. Math. 440 (1993), 99–109. [25] D. Harbater, Abhyankar’s conjecture on Galois groups over curves, Invent. Math., 117 (1994), 1-25. [26] D. Harbater, A. Obus, R. Pries, and K. Stevenson, Abhyankar’s conjectures in galois theory: current status and future directions, arXiv:1408.0859 [math.AG]. [27] M. Hall, Jr. The theory of groups, The Macmillan Co., New York, N.Y. 1959 xiii+434 pp. [28] H.W. Henn, Funktionenkörper mit groβer Automorphismengruppe, J. Reine Angew. Math. 302 (1978), 96–115. [29] J.W.P. Hirschfeld, G. Korchmáros and F. Torres, Algebraic Curves Over a Finite Field, Princeton Univ. Press, Princeton and Oxford, 2008, xx+696 pp. [30] C. Hering, A theorem on group space, Hokkaido Math. J. 8 (1979), 115-120. [31] B. Huppert, Endliche Gruppen. I, Grundlehren der Mathematischen wissenschaften 134, Springer, Berlin, 1967, xii+793 pp. [32] G. Korchmáros and M. Montanucci, The geometry of the Artin-Schreier-Mumford curves over an algebraically closed field, Acta Sci. Math. (Szeged) 83 (2017), 673-681. [33] V. Landázuri and G.M. Seitz, On the minimal degrees of projective representations of the finite Chevalley groups, J. Algebra 32 (1974), 418–443. 39 [34] L.M. Isaacs, Character theory of finite groups, Corrected reprint of the 1976 original [Academic Press, New York; MR0460423]. Dover Publications, Inc., New York, 1994. xii+303 pp. [35] H.H. Mitchell, Determination of the ordinary and modular ternary linear groups. Trans. Amer. Math. Soc. 12 (1911), 207-242. [36] S. Nakajima, p-ranks and automorphism groups of algebraic curves, Trans. Amer. Math. Soc. 303 (1987), 595–607. [37] M.E. O’Nan, On unitary collineation group, J. Algebra 22 (1972), 211-218. [38] R.J. Pries, Wildly ramified covers with large genus J. Number Theory 119 (2006), 194-209. [39] R. Pries and K. Stevenson, A survey of Galois theory of curves in characteristic p, WIN–women in numbers, 169-191, Fields Inst. Commun., 60, Amer. Math. Soc., Providence, RI, 2011. [40] C.S. Rajan, Automorphisms of X(11) over characteristic 3 and the Mathieu group M (11), Ramanujan Math. Soc. 13, (1998), 63–72 [41] P. Roquette, Abschätzung der Automorphismenanzahl von Funktionenkörpern bei Primzahlcharakteristik, Math. Z. 117 (1970), 157–163. [42] H. Stichtenoth, Über die Automorphismengruppe eines algebraischen Funktionenkörpers von Primzahlcharakteristik. I. Eine Abschätzung der Ordnung der Automorphismengruppe, Arch. Math. 24 (1973), 527–544. [43] J.-P. Serre, Local Fields, Graduate Texts in Mathematics 67, Springer, New York, 1979. viii+241 pp. [44] I.R. Shafarevich, On p-extensions, Amer. Math. Soc. Transl. 4 (1954), 59–71. [45] T. Shaska and H. Völklein, Elliptic subfields and automorphisms of genus 2 function fields, in Algebra, arithmetic and geometry with applications (West Lafayette, IN, 2000), 703-723, Springer, Berlin, 2004. [46] F. Sullivan, p-torsion in the class group of curves with many automorphisms, Arch. Math. 26 (1975), 253–261. [47] H. Stichtenoth, Über die Automorphismengruppe eines algebraischen Funktionenkörpers von Primzahlcharakteristik. II. Ein spezieller Typ von Funktionenkörpern, Arch. Math. 24 (1973), 615– 631. [48] J.G. Thompson, Nonsolvable finite groups all of whose local subgroups are solvable. Bull. Amer. Math. Soc. 74 (1968), 383-437. [49] R.C. Valentini and M.L. Madan, A Hauptsatz of L.E. Dickson and Artin–Schreier extensions, J. Reine Angew. Math. 318 (1980), 156–177. [50] J.H. Walter, The characterization of finite groups with abelian Sylow 2-subgroups, Ann. of Math. 89 (1969), 405-514. [51] R.A. Wilson, The finite simple groups, Graduate Texts in Mathematics, 251, Springer-Verlag London, Ltd., London, 2009. xvi+298 pp. 40 [52] W.J. Wong, On finite groups with semi-dihedral Sylow 2-subgroups. J. Algebra 4 (1966), 52-63. [53] W.J. Wong, On finite groups whose 2-sylow subgroups have cyclic subgroups of index 2, J. Austral. Math. Soc. 4 (1964), 90-112. Authors’ addresses: Massimo GIULIETTI Dipartimento di Matematica e Informatica Università degli Studi di Perugia Via Vanvitelli, 1 06123 Perugia (Italy). E–mail: [email protected] Gábor KORCHMÁROS Dipartimento di Matematica Università della Basilicata Contrada Macchia Romana 85100 Potenza (Italy). E–mail: [email protected] 41