Especially in geometric contexts, an automorphism group is also called a symmetry group. A subgroup of an automorphism group is sometimes called a transformation group.
Automorphism groups are studied in a general way in the field of category theory.
If X is a set with no additional structure, then any bijection from X to itself is an automorphism, and hence the automorphism group of X in this case is precisely the symmetric group of X. If the set X has additional structure, then it may be the case that not all bijections on the set preserve this structure, in which case the automorphism group will be a subgroup of the symmetric group on X. Some examples of this include the following:
If X is an object in a category, then the automorphism group of X is the group consisting of all the invertible morphisms from X to itself. It is the unit group of the endomorphism monoid of X. (For some examples, see PROP.)
In general, however, an automorphism group functor may not be represented by a scheme.