Lably
Skip to content

McNemar's test

Statistical test used on paired nominal data

  • 9 min read
  • 4 chapters

Related on Lably

Through Statistical hypothesis test, Homogeneity and heterogeneity and Chi-squared distribution

Lab · Class 12Differential EquationsLab · Class 8–10Periodic ClassificationLab · Class 11–12Introduction to CalculusLab · Class 10Heredity & Genetics

The commonly used parameters to assess a diagnostic test in medical sciences are sensitivity and specificity. Sensitivity (or recall) is the ability of a test to correctly identify the people with disease. Specificity is the ability of the test to correctly identify those without the disease.

Now presume two tests are performed on the same group of patients. And also presume that these tests have identical sensitivity and specificity. In this situation one is carried away by these findings and presume that both the tests are equivalent. However this may not be the case. For this we have to study the patients with disease and patients without disease (by a reference test). We also have to find out where these two tests disagree with each other. This is precisely the basis of McNemar's test. This test compares the sensitivity and specificity of two diagnostic tests on the same group of patients.

Definition

The test is applied to a 2 × 2 contingency table, which tabulates the outcomes of two tests on a sample of N subjects, as follows.

The null hypothesis of marginal homogeneity states that the two marginal probabilities for each outcome are the same, i.e. pa + pb = pa + pc and pc + pd = pb + pd.

Thus the null and alternative hypotheses are

H0:pb=pcH1:pb≠pc

Here pa, etc., denote the theoretical probability of occurrences in cells with the corresponding label.

The McNemar test statistic is:

χ2=(b−c)2b+c.

Under the null hypothesis, with a sufficiently large number of discordants (cells b and c), χ2 has a chi-squared distribution with 1 degree of freedom. If the χ2 result is significant, this provides sufficient evidence to reject the null hypothesis, in favour of the alternative hypothesis that pb ≠ pc, which would mean that the marginal proportions are significantly different from each other.

Variations

If either b or c is small (b + c < 25) then χ2 is not well-approximated by the chi-squared distribution. An exact binomial test can then be used, where b is compared to a binomial distribution with size parameter n = b + c and p = 0.5. Effectively, the exact binomial test evaluates the imbalance in the discordants b and c. To achieve a two-sided P-value, the P-value of the extreme tail should be multiplied by 2. For b ≥ c:

exact-P-value=2∑i=bn(ni)0.5i(1−0.5)n−i,

which is simply twice the binomial distribution cumulative distribution function with p = 0.5 and n = b + c.

Edwards proposed the following continuity corrected version of the McNemar test to approximate the binomial exact-P-value:

χ2=(|b−c|−1)2b+c.

The mid-P McNemar test (mid-p binomial test) is calculated by subtracting half the probability of the observed b from the exact one-sided P-value, then double it to obtain the two-sided mid-P-value:

mid-p-value=2(∑i=bn(ni)0.5i(1−0.5)n−i−0.5(nb)0.5b(1−0.5)n−b)

This is equivalent to:

mid-p-value=exact-p-value−(nb)0.5b(1−0.5)n−b

where the second term is the binomial distribution probability mass function and n = b + c. Binomial distribution functions are readily available in common software packages and the McNemar mid-P test can easily be calculated.

The traditional advice has been to use the exact binomial test when b + c < 25. However, simulations have shown both the exact binomial test and the McNemar test with continuity correction to be overly conservative. When b + c < 6, the exact-P-value always exceeds the common significance level 0.05. The original McNemar test was most powerful, but often slightly liberal. The mid-P version was almost as powerful as the asymptotic McNemar test and was not found to exceed the nominal significance level.

Sign in to keep reading

3 more chapters are free with a Lably account.

More in Statistics

Mortality rateDeaths per 1,000 individuals per yearSystematic reviewComprehensive review of research literature using systematic methodsSensitivity and specificityStatistical measure of a binary classificationLongitudinal studyStudy with repeated observations over timePrevalenceNumber of disease cases in a given population at a specific timeStochasticRandomly determined process