Tukey's range test

Assumptions

1. The observations being tested are independent within and among the groups.
2. The subgroups associated with each mean in the test are normally distributed.
3. There is equal within-subgroup variance across the subgroups associated with each mean in the test (homogeneity of variance).

The test statistic

Tukey's test is based on a formula very similar to that of the t-test. In fact, Tukey's test is essentially a t-test, except that it corrects for family-wise error rate.

The formula for Tukey's test is

${\displaystyle q_{\mathsf{s}}=\frac{\left|Y_{\mathsf{A}}-Y_{\mathsf{B}}\right|}{\mathsf{S}\mathsf{E}},}$

where Y_A and Y_B are the two means being compared, and SE is the standard error for the sum of the means. The value q_s is the sample's test statistic. (The notation |x| means the absolute value of x; the magnitude of x with the sign set to +, regardless of the original sign of x.)

This q_s test statistic can then be compared to a q value for the chosen significance level α from a table of the studentized range distribution. If the q_s value is larger than the critical value q_α obtained from the distribution, the two means are said to be significantly different at level ${\displaystyle \alpha :0\le \alpha \le 1.}$

Since the null hypothesis for Tukey's test states that all means being compared are from the same population (i.e. μ₁ = μ₂ = μ₃ = ... = μ_k ), the means should be normally distributed (according to the central limit theorem) with the same model standard deviation σ, estimated by the merged standard error, ${\displaystyle \mathsf{S}\mathsf{E},}$ for all the samples; its calculation is discussed in the following sections. This gives rise to the normality assumption of Tukey's test.

The studentized range (q) distribution

The Tukey method uses the studentized range distribution. Suppose that we take a sample of size n from each of k populations with the same normal distribution N(μ, σ²) and suppose that ${\displaystyle {\overline{y}}_{\mathsf{m}\mathsf{i}\mathsf{n}}}$ is the smallest of these sample means and ${\displaystyle {\overline{y}}_{\mathsf{m}\mathsf{a}\mathsf{x}}}$ is the largest of these sample means, and suppose S² is the pooled sample variance from these samples. Then the following random variable has a Studentized range distribution:

${\displaystyle q\equiv \frac{{\overline{y}}_{\mathsf{m}\mathsf{a}\mathsf{x}}-{\overline{y}}_{\mathsf{m}\mathsf{i}\mathsf{n}}}{S\sqrt{2/n}}}$

This definition of the statistic q given above is the basis of the critically significant value for q_α discussed below, and is based on these three factors:

${\displaystyle \alpha }$ the Type I error rate, or the probability of rejecting a true null hypothesis;

${\displaystyle k}$ the number of sub-populations being compared;

${\displaystyle \mathsf{d}\mathsf{f}}$ the number of degrees of freedom for each mean

( df = N − k ) where N is the total number of observations.)

The distribution of q has been tabulated and appears in many textbooks on statistics. In some tables the distribution of q has been tabulated without the ${\displaystyle \sqrt{2}}$ factor. To understand which table it is, we can compute the result for k = 2 and compare it to the result of the Student's t-distribution with the same degrees of freedom and the same α . In addition, R offers a cumulative distribution function (ptukey) and a quantile function (qtukey) for q .

Confidence limits

The Tukey confidence limits for all pairwise comparisons with confidence coefficient of at least 1 − α are

${\displaystyle {\overline{y}}_{i\bullet }-{\overline{y}}_{j\bullet }\pm \frac{q_{\alpha ;k;N-k}}{\sqrt{2}}{\hat{\sigma }}_{\epsilon }\sqrt{\frac{2}{n}}:i,j=1,\dots ,ki\ne j.}$

Notice that the point estimator and the estimated variance are the same as those for a single pairwise comparison. The only difference between the confidence limits for simultaneous comparisons and those for a single comparison is the multiple of the estimated standard deviation.

Also note that the sample sizes must be equal when using the studentized range approach. ${\displaystyle {\hat{\sigma }}_{\epsilon }}$ is the standard deviation of the entire design, not just that of the two groups being compared. It is possible to work with unequal sample sizes. In this case, one has to calculate the estimated standard deviation for each pairwise comparison as formalized by Clyde Kramer in 1956, so the procedure for unequal sample sizes is sometimes referred to as the Tukey–Kramer method which is as follows:

${\displaystyle {\overline{y}}_{i\bullet }-{\overline{y}}_{j\bullet }\pm \frac{q_{\alpha ;k;N-k}}{\sqrt{2}}{\hat{\sigma }}_{\epsilon }\sqrt{\frac{1}{n_i}+\frac{1}{n_j}}}$

where n _i and n _j are the sizes of groups i and j respectively. The degrees of freedom for the whole design is also applied.

Comparing ANOVA and Tukey–Kramer tests

Both ANOVA and Tukey–Kramer tests are based on the same assumptions. However, these two tests for k groups (i.e. μ₁ = μ₂ = ... = μ_k ) may result in logical contradictions when k > 2 , even if the assumptions do hold.

It is possible to generate a set of pseudorandom samples of strictly negative measure such that hypothesis μ₁ = μ₂ is rejected at significance level ${\displaystyle 1-\alpha >0.95}$ while μ₁ = μ₂ = μ₃ is not rejected even at ${\displaystyle 1-\alpha =0.975.}$