Exact statistics

When exact p-values and confidence intervals are computed under a certain distribution, such as the normal distribution, then the underlying methods are referred to as exact parametric methods. The exact methods that do not make any distributional assumptions are referred to as exact nonparametric methods. The latter has the advantage of making fewer assumptions whereas, the former tend to yield more powerful tests when the distributional assumption is reasonable. For advanced methods such as higher-way ANOVA regression analysis, and mixed models, only exact parametric methods are available.

When the sample size is small, asymptotic results given by some traditional methods may not be valid. In such situations, the asymptotic p-values may differ substantially from the exact p-values. Hence asymptotic and other approximate results may lead to unreliable and misleading conclusions.

The approach

All classical statistical procedures are constructed using statistics which depend only on observable random vectors, whereas generalized estimators, tests, and confidence intervals used in exact statistics take advantage of the observable random vectors and the observed values both, as in the Bayesian approach but without having to treat constant parameters as random variables. For example, in sampling from a normal population with mean ${\displaystyle \mu }$ and variance ${\displaystyle {\sigma }^2}$, suppose ${\displaystyle \overline{X}}$ and ${\displaystyle S^2}$ are the sample mean and the sample variance. Then, defining Z and U thus:

${\displaystyle Z=\sqrt{n}(\overline{X}-\mu )/\sigma \sim N(0,1)}$

and that

${\displaystyle U=n{S}^2/{\sigma }^2\sim {\chi }_{n-1}^2}$.

Now suppose the parameter of interest is the coefficient of variation, ${\displaystyle \rho =\mu /\sigma }$. Then, we can easily perform exact tests and exact confidence intervals for ${\displaystyle \rho }$ based on the generalized statistic

${\displaystyle R=\frac{\overline{x}S}{s\sigma }-\frac{\overline{X}-\mu }{\sigma }=\frac{\overline{x}}{s}\frac{\sqrt{U}}{\sqrt{n}}-\frac{Z}{\sqrt{n}}}$,

where ${\displaystyle \overline{x}}$ is the observed value of ${\displaystyle \overline{X}}$ and ${\displaystyle S}$ is the observed value of ${\displaystyle s}$. Exact inferences on ${\displaystyle \rho }$ based on probabilities and expected values of ${\displaystyle R}$ are possible because its distribution and the observed value are both free of nuisance parameters.