Outgroup (cladistics)

Choice of outgroup

The chosen outgroup is hypothesized to be less closely related to the ingroup than the ingroup is related to itself. The evolutionary assumption from these relationships is that the outgroup species has a common ancestor with the ingroup that is older than the common ancestor of the ingroup. Choice of outgroup can change the topology of a phylogeny. Therefore, phylogeneticists typically use more than one outgroup in cladistic analysis. The use of multiple outgroups is preferable because it provides a more robust phylogeny, buffering against poor outgroup candidates. If one outgroup taxon is designated as the root and the others are included simply as additional taxa in the analysis, this provides a test of the ingroup's hypothesized monophyly.

To qualify as an outgroup, a taxon must satisfy the following two characteristics:

• It must not be a member of the ingroup.
• It must be related to the ingroup, closely enough so that its character states are comparable to those of the ingroup.

Therefore, an appropriate outgroup must be unambiguously outside the clade of interest in the phylogenetic study. An outgroup that is nested within the ingroup will, when used to root the phylogeny, result in incorrect inference of phylogenetic relationships and trait evolution. However, the optimal level of relatedness of the outgroup to the ingroup depends on the depth of phylogenetic analysis. Choosing a closely related outgroup relative to the ingroup is more useful when looking at subtle differences, while choosing an unduly distant outgroup can result in mistaking convergent evolution for a direct evolutionary relationship due to a common ancestor. For shallow phylogenetics—for example, resolving the evolutionary relationships of a clade within a genus—an appropriate outgroup would be a member of the sister clade. However, for deeper phylogenetic analysis, less closely related taxa can be used. For example, Jarvis et al. (2014) used humans and crocodiles as outgroups while resolving the early branches of the avian phylogeny. In molecular phylogenetics, satisfying the second requirement typically means that DNA or protein sequences from the outgroup can be successfully aligned to sequences from the ingroup. Although there are algorithmic approaches to identify the outgroups with maximum global parsimony, they are often limited by failing to reflect the continuous, quantitative nature of certain character states. Character states are traits, either ancestral or derived, that affect the construction of branching patterns in a phylogenetic tree.

Examples

In each example, a phylogeny of organisms in the ingroup may be rooted by scoring the same character states for one or more members of the outgroup.