In group theory, a topic in abstract algebra, the Mathieu groups are the five sporadic simple groups M11, M12, M22, M23 and M24 introduced by Mathieu (1861, 1873). They are multiply transitive permutation groups on 11, 12, 22, 23 or 24 objects. They were the first sporadic groups to be discovered.
Sometimes the notation M9, M10, M20 and M21 is used for related groups (which act on sets of 9, 10, 20, and 21 points, respectively), namely the stabilizers of points in the larger groups. While these are not sporadic simple groups, they are subgroups of the larger groups and can be used to construct the larger ones. John Conway has shown that one can also extend this sequence up, obtaining the Mathieu groupoid M13 acting on 13 points. M21 is simple, but is not a sporadic group, being isomorphic to PSL(3,4).
Mathieu (1861, p.271) introduced the group M12 as part of an investigation of multiply transitive permutation groups, and briefly mentioned (on page 274) the group M24, giving its order. In Mathieu (1873) he gave further details, including explicit generating sets for his groups, but it was not easy to see from his arguments that the groups generated are not just alternating groups, and for several years the existence of his groups was controversial. Miller (1898) even published a paper mistakenly claiming to prove that M24 does not exist, though shortly afterwards in (Miller 1900) he pointed out that his proof was wrong, and gave a proof that the Mathieu groups are simple. Witt (1938a, 1938b) finally removed the doubts about the existence of these groups, by constructing them as successive transitive extensions of permutation groups, as well as automorphism groups of Steiner systems.
After the Mathieu groups no new sporadic groups were found until 1965, when the group J1 was discovered.
Mathieu was interested in finding multiply transitive permutation groups, which will now be defined. For a natural number k, a permutation group G acting on n points is k-transitive if, given two sets of points a1, ... ak and b1, ... bk with the property that all the ai are distinct and all the bi are distinct, there is a group element g in G which maps ai to bi for each i between 1 and k. Such a group is called sharply k-transitive if the element g is unique (i.e. the action on k-tuples is regular, rather than just transitive).
M24 is 5-transitive, and M12 is sharply 5-transitive, with the other Mathieu groups (simple or not) being the subgroups corresponding to stabilizers of m points, and accordingly of lower transitivity (M23 is 4-transitive, etc.).
The only 4-transitive groups are the symmetric groups Sk for k at least 4, the alternating groups Ak for k at least 6, and the Mathieu groups M24, M23, M12 and M11. (Cameron 1999, p. 110) The full proof requires the classification of finite simple groups, but some special cases have been known for much longer.
It is a classical result of Jordan that the symmetric and alternating groups (of degree k and k + 2 respectively), and M12 and M11 are the only sharply k-transitive permutation groups for k at least 4.
M12 has a simple subgroup of order 660, a maximal subgroup. That subgroup is isomorphic to the projective special linear group PSL2(F11) over the field of 11 elements. With −1 written as a and infinity as b, two standard generators are (0123456789a) and (0b)(1a)(25)(37)(48)(69). A third generator giving M12 sends an element x of F11 to 4x2 − 3x7; as a permutation that is (26a7)(3945).
This group turns out not to be isomorphic to any member of the infinite families of finite simple groups and is called sporadic. M11 is the stabilizer of a point in M12, and turns out also to be a sporadic simple group. M10, the stabilizer of two points, is not sporadic, but is an almost simple group whose commutator subgroup is the alternating group A6. It is thus related to the exceptional outer automorphism of A6. The stabilizer of 3 points is the projective special unitary group PSU(3,22), which is solvable. The stabilizer of 4 points is the quaternion group.
The stabilizers of 1 and 2 points, M23 and M22 also turn out to be sporadic simple groups. The stabilizer of 3 points is simple and isomorphic to the projective special linear group PSL3(4).
There exists up to equivalence a unique S(5,8,24) Steiner system W24 (the Witt design). The group M24 is the automorphism group of this Steiner system; that is, the set of permutations which map every block to some other block. The subgroups M23 and M22 are defined to be the stabilizers of a single point and two points respectively.
Similarly, there exists up to equivalence a unique S(5,6,12) Steiner system W12, and the group M12 is its automorphism group. The subgroup M11 is the stabilizer of a point.
The group M24 is the permutation automorphism group of the extended binary Golay code W, i.e., the group of permutations on the 24 coordinates that map W to itself. All the Mathieu groups can be constructed as groups of permutations on the binary Golay code.
M12 has index 2 in its automorphism group, and M12:2 happens to be isomorphic to a subgroup of M24. M12 is the stabilizer of a dodecad, a codeword of 12 1's; M12:2 stabilizes a partition into 2 complementary dodecads.
There is a natural connection between the Mathieu groups and the larger Conway groups, because the Leech lattice was constructed on the binary Golay code and in fact both lie in spaces of dimension 24. The Conway groups in turn are found in the Monster group. Robert Griess refers to the 20 sporadic groups found in the Monster as the Happy Family, and to the Mathieu groups as the first generation.