G-test

Distribution and use

Given the null hypothesis that the observed frequencies result from random sampling from a distribution with the given expected frequencies, the distribution of the test statistics ${\displaystyle G}$ is approximately a chi-squared distribution, with the same number of degrees of freedom as in the corresponding chi-squared test.

For very small samples the multinomial test for goodness of fit, and Fisher's exact test for contingency tables, or even Bayesian hypothesis selection are preferable to the G-test. McDonald recommends to always use an exact test (exact test of goodness-of-fit, Fisher's exact test) if the total sample size is less than 1 000 .

There is nothing magical about a sample size of 1 000, it's just a nice round number that is well within the range where an exact test, chi-square test, and G–test will give almost identical p values. Spreadsheets, web-page calculators, and SAS shouldn't have any problem doing an exact test on a sample size of 1 000 .

— John H. McDonald (2014)

G-tests have been recommended at least since the 1981 edition of Biometry, a statistics textbook by Robert R. Sokal and F. James Rohlf.

Relation to other metrics

Relation to the chi-squared test

The commonly used chi-squared tests for goodness of fit to a distribution and for independence in contingency tables are in fact approximations of the log-likelihood ratio on which the G-tests are based.

The general formula for Pearson's chi-squared test statistic is

${\displaystyle {\chi }^2=\sum _i\frac{{\left(O_i-E_i\right)}^2}{E_i}.}$

The approximation of the G-test statistics by chi-squared test statistics is obtained by a second order Taylor expansion of the natural logarithm around 1 (see the derivation below). We have ${\displaystyle G\approx {\chi }^2}$ when the observed counts ${\displaystyle O_i}$ are close to the expected counts ${\displaystyle E_i}$. When this difference is large, however, the approximation by the chi-squared test statistics begins to break down. Here, the effects of outliers in data will be more pronounced, and this explains the why chi-squared tests fail in situations with little data.

For samples of a reasonable size, the G-test and the chi-squared test will lead to the same conclusions. However, the approximation to the theoretical chi-squared distribution for the G-test is better than for the Pearson's chi-squared test. In cases where ${\displaystyle O_i>2\cdot E_i}$ for some cell case the G-test is always better than the chi-squared test.

For testing goodness-of-fit the G-test is infinitely more efficient than the chi-squared test in the sense of Bahadur, but the two tests are equally efficient in the sense of Pitman or in the sense of Hodges and Lehmann.

Derivation (chi-squared)

Consider

${\displaystyle G=2\sum _i{O}_i\ln \left(\frac{O_i}{E_i}\right),}$

and let ${\displaystyle O_i=E_i+{\delta }_i}$ with ${\displaystyle \sum _i{\delta }_i=0}$, so that the total number of counts remains the same. Assume that ${\displaystyle {\delta }_i=O_i-E_i}$ is small in comparison to ${\displaystyle E_i}$ for all ${\displaystyle i}$. To be more precise, notice that ${\displaystyle E_i=\mathrm{\Theta }(n)}$ using big Θ notation. If ${\displaystyle O_i=E_i+\mathcal{O}(n^{1/2})}$ using big O notation for large ${\displaystyle n}$, which should be true under the null hypothesis because of the central limit theorem, then ${\displaystyle {\delta }_i=\mathcal{O}(n^{1/2})}$ and

${\displaystyle \frac{{\delta }_i^3}{E_i^2}=\mathcal{O}\left(\frac{n^{3/2}}{n^2}\right)=\mathcal{O}(n^{-1/2})}$

follow, which will be used later.

Upon substitution we find,

${\displaystyle G=2\sum _i(E_i+{\delta }_i)\ln \left(1+\frac{{\delta }_i}{E_i}\right).}$

Using the Taylor expansion ${\displaystyle \ln (1+x)=x-\frac{1}{2}{x}^2+\mathcal{O}(x^3)}$ yields

${\displaystyle G=2\sum _i(E_i+{\delta }_i)\left(\frac{{\delta }_i}{E_i}-\frac{1}{2}\frac{{\delta }_i^2}{E_i^2}+\mathcal{O}\left(\frac{{\delta }_i^3}{E_i^3}\right)\right),}$

and distributing terms we find,

${\displaystyle G=2\sum _i\left({\delta }_i+\frac{1}{2}\frac{{\delta }_i^2}{E_i}+\mathcal{O}\left(\frac{{\delta }_i^3}{E_i^2}\right)\right).}$

Now, using ${\displaystyle \sum _i{\delta }_i=0}$ and ${\displaystyle {\delta }_i=O_i-E_i}$ and ${\displaystyle \mathcal{O}({\delta }_i^3/E_i^2)=\mathcal{O}(n^{-1/2})}$ for large ${\displaystyle n}$, we can write the result,

${\displaystyle G\approx \sum _i\frac{{\left(O_i-E_i\right)}^2}{E_i}.}$

Relation to Kullback–Leibler divergence

The G-test statistic is proportional to the Kullback–Leibler divergence of the theoretical distribution ${\displaystyle \tilde{p}=({\tilde{p}}_1,\dots ,{\tilde{p}}_m)}$ of the null hypothesis from the empirical distribution ${\displaystyle \hat{p}=({\hat{p}}_1,\dots ,{\hat{p}}_m)}$ of the observed data:

${\displaystyle \begin{array}{cc}G& =2\sum _i{O}_i\cdot \ln \left(\frac{O_i}{E_i}\right)=2N\sum _i{\hat{p}}_i\cdot \ln \left(\frac{{\hat{p}}_i}{{\tilde{p}}_i}\right)\\ & =2N{D}_{\mathrm{K}\mathrm{L}}(\hat{p}\Vert \tilde{p}),\end{array}}$

where ${\displaystyle N}$ is the total number of observations and ${\displaystyle {\tilde{p}}_i=\frac{E_i}{N}}$ and ${\displaystyle {\hat{p}}_i=\frac{O_i}{N}}$ are the theoretical and empirical probabilities of objects of type ${\displaystyle i}$, respectively.

Relation to mutual information

For analysis of contingency tables the value of the G-test statistics can also be expressed in terms of mutual information.

In this case objects with two-dimensional types ${\displaystyle (i,j)}$ are considered. Let ${\displaystyle O_{ij}}$ be the count of objects of type ${\displaystyle (i,j)}$, i.e., ${\displaystyle O_{ij}}$ is the entry in the contingency table in row ${\displaystyle i}$ and column ${\displaystyle j}$. Set

${\displaystyle N=\sum _{ij}{O}_{ij},{\hat{p}}_{ij}=\frac{O_{ij}}{N},{\hat{p}}_{i\bullet }=\frac{\sum _j{O}_{ij}}{N},{\hat{p}}_{\bullet j}=\frac{\sum _i{O}_{ij}}{N}.}$

Then the estimated expected count of objects of type ${\displaystyle (i,j)}$ assuming independence is given by

${\displaystyle E_{ij}=N{\hat{p}}_{i\bullet }{\hat{p}}_{\bullet j}.}$

Finally, the G-test statistics in this case is given by

${\displaystyle G=2\sum _{ij}{O}_{ij}\ln \left(\frac{O_{ij}}{E_{ij}}\right)}$

Let ${\displaystyle X,Y}$ be random variables with joint distribution given by the empirical distribution ${\displaystyle {\hat{p}}_{ij}}$ of the contingency table, i.e.,

${\displaystyle P(X=i,Y=j)={\hat{p}}_{ij},P(X=i)={\hat{p}}_{i\bullet },P(Y=j)={\hat{p}}_{\bullet j}.}$

Then the G-test statistics can be expressed in several alternative forms:

${\displaystyle \begin{array}{cc}G& =2N\cdot \sum _{ij}{\hat{p}}_{ij}\left(\ln ({\hat{p}}_{ij})-\ln ({\hat{p}}_{i\bullet })-\ln ({\hat{p}}_{\bullet j})\right)\\ & =2N\cdot (H(X)+H(Y)-H(X,Y))\\ & =2N\cdot \mathrm{MI}(X,Y),\end{array}}$

where the entropies ${\displaystyle H(X)}$ and ${\displaystyle H(Y)}$ are given

${\displaystyle H(X)=-\sum _i{\hat{p}}_{i\bullet }\ln ({\hat{p}}_{i\bullet }),H(Y)=-\sum _j{\hat{p}}_{\bullet j}\ln ({\hat{p}}_{\bullet j})}$

and the joint entropy ${\displaystyle H(X,Y)}$ is given by

${\displaystyle H(X,Y)=-\sum _{ij}{\hat{p}}_{ij}\ln ({\hat{p}}_{ij})}$

and the mutual information of ${\displaystyle X}$ and ${\displaystyle Y}$ is

${\displaystyle \mathrm{MI}(X,Y)=H(X)+H(Y)-H(X,Y).}$

It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G-test statistic.

Application

• The McDonald–Kreitman test in statistical genetics is an application of the G-test.
• Dunning introduced the test to the computational linguistics community where it is now widely used.
• The R-scape program (used by Rfam) uses G-test to detect co-variation between RNA sequence alignment positions.

Statistical software

• In R fast implementations can be found in the AMR and Rfast packages. For the AMR package, the command is g.test which works exactly like chisq.test from base R. R also has the likelihood.test Archived 2013-12-16 at the Wayback Machine function in the Deducer Archived 2012-03-09 at the Wayback Machine package. Note: Fisher's G-test in the GeneCycle Package of the R programming language (fisher.g.test) does not implement the G-test as described in this article, but rather Fisher's exact test of Gaussian white-noise in a time series.
• Another R implementation to compute the G-test statistic and corresponding p-values is provided by the R package entropy. The commands are Gstat for the standard G statistic and the associated p-value and Gstatindep for the G statistic applied to comparing joint and product distributions to test independence.
• In SAS, one can conduct G-test by applying the /chisq option after the proc freq.
• In Stata, one can conduct a G-test by applying the lr option after the tabulate command.
• In Java, use org.apache.commons.math3.stat.inference.GTest.
• In Python, use scipy.stats.power_divergence with lambda_=0.