Multiple comparison procedures for the effect means in a linear model

Methods of controlling the Type I error and dependencies between point estimates when making multiple comparisons between effect means in a linear model.

Procedure Purpose / Assumptions
Student's t (Fisher's LSD) Compare the means of each pair of groups using the Student's t method.

When making all pairwise comparisons this procedure is also known as unprotected Fisher's LSD, or when only performed following significant ANOVA F -test known as protected Fisher's LSD.

Controls the Type I error rate individually for each contrast.

Tukey-Kramer Compare the means of all pairs of groups using the Tukey-Kramer method.

Controls the error rate simultaneously for all k(k+1)/2 contrasts.

Hsu Compare the means of all groups against the best of the other groups using the Hsu method.

Controls the error rate simultaneously for all k contrasts.

Dunnett Compare the means of all groups against a control using the Dunnett method.

Controls the error rate simultaneously for all k-1 contrasts.

Scheffé Compare the means of all groups against all other groups using the Scheffé F method.

Controls the error rate simultaneously for all possible contrasts.

Bonferroni Not a multiple comparisons method. It is an inequality useful in producing easy to compute multiple comparisons. In most scenarios, there are more powerful procedures such as Tukey, Dunnett, Hsu.

A useful application of Bonferroni inequality is when there are a small number of pre-planned comparisons. In this case, use the Student's t (LSD) method with the significance level (alpha) set to the Bonferroni inequality (alpha divided by the number of comparisons). In this scenario, it is usually less conservative than using Scheffé all contrast comparisons.