Empirical distribution function

Asymptotic properties

By the strong law of large numbers, the estimator ${\displaystyle {\hat{F}}_n(t)}$ converges to F(t) as n → ∞ almost surely, for every value of t: $${\displaystyle {\hat{F}}_n(t)\stackrel{\text{a.s.}}{\to }F(t);}$$ thus the estimator ${\displaystyle {\hat{F}}_n(t)}$ is consistent. This expression asserts the pointwise convergence of the empirical distribution function to the true cumulative distribution function. There is a stronger result, called the Glivenko–Cantelli theorem, which states that the convergence in fact happens uniformly over t: $${\displaystyle {\Vert {\hat{F}}_n-F\Vert }_{\mathrm{\infty }}\equiv \underset{t\in \mathbb{R}}{sup}\left|{\hat{F}}_n(t)-F(t)\right|\stackrel{}{\to }0.}$$ The sup-norm in this expression is called the Kolmogorov–Smirnov statistic for testing the goodness-of-fit between the empirical distribution ${\hat{F}}_n(t)$ and the assumed true cumulative distribution function F. Other norm functions may be reasonably used here instead of the sup-norm. For example, the L²-norm gives rise to the Cramér–von Mises statistic.

The asymptotic distribution can be further characterized in several different ways. First, the central limit theorem states that pointwise, ${\hat{F}}_n(t)$ has asymptotically normal distribution with the standard $\sqrt{n}$ rate of convergence: $${\displaystyle \sqrt{n}({\hat{F}}_n(t)-F(t))\stackrel{d}{\to }\mathcal{N}(0,F(t)(1-F(t))).}$$ This result is extended by the Donsker’s theorem, which asserts that the empirical process $\sqrt{n}({\hat{F}}_n-F)$, viewed as a function indexed by $t\in \mathbb{R}$, converges in distribution in the Skorokhod space ${\displaystyle D[-\mathrm{\infty },+\mathrm{\infty }]}$ to the mean-zero Gaussian process $G_F=B\circ F$, where B is the standard Brownian bridge. The covariance structure of this Gaussian process is $${\displaystyle \mathrm{E}[G_F(t_1)G_F(t_2)]=F(t_1\wedge t_2)-F(t_1)F(t_2).}$$ The uniform rate of convergence in Donsker’s theorem can be quantified by the result known as the Hungarian embedding: $${\displaystyle \underset{n\to \mathrm{\infty }}{lim sup}\frac{\sqrt{n}}{{\ln }^{2}n}{\Vert \sqrt{n}({\hat{F}}_n-F)-G_{F,n}\Vert }_{\mathrm{\infty }}<\mathrm{\infty },\text{a.s.}}$$

Alternatively, the rate of convergence of ${\displaystyle \sqrt{n}({\hat{F}}_n-F)}$ can also be quantified in terms of the asymptotic behavior of the sup-norm of this expression. Number of results exist in this venue, for example the Dvoretzky–Kiefer–Wolfowitz inequality provides bound on the tail probabilities of $\sqrt{n}{\Vert {\hat{F}}_n-F\Vert }_{\mathrm{\infty }}$: $${\displaystyle Pr(\sqrt{n}\Vert {\hat{F}}_n-F{\Vert }_{\mathrm{\infty }}>z)\le 2e^{-2z^2}.}$$ In fact, Kolmogorov has shown that if the cumulative distribution function F is continuous, then the expression $\sqrt{n}{\Vert {\hat{F}}_n-F\Vert }_{\mathrm{\infty }}$ converges in distribution to ${\Vert B\Vert }_{\mathrm{\infty }}$, which has the Kolmogorov distribution that does not depend on the form of F.

Another result, which follows from the law of the iterated logarithm, is that $${\displaystyle \underset{n\to \mathrm{\infty }}{lim sup}\sqrt{\frac{n}{2\ln \ln n}}{\Vert {\hat{F}}_n-F\Vert }_{\mathrm{\infty }}\le \frac{1}{2},\text{a.s.}}$$ and $${\displaystyle \underset{n\to \mathrm{\infty }}{lim inf}\sqrt{2n\ln \ln n}{\Vert {\hat{F}}_n-F\Vert }_{\mathrm{\infty }}=\frac{\pi }{2},\text{a.s.}}$$

Confidence intervals

As per Dvoretzky–Kiefer–Wolfowitz inequality the interval that contains the true CDF, ${\displaystyle F(x)}$, with probability ${\displaystyle 1-\alpha }$ is specified as

$${\displaystyle F_n(x)-\epsilon \le F(x)\le F_n(x)+\epsilon \text{ where }\epsilon =\sqrt{\frac{\ln \frac{2}{\alpha }}{2n}}.}$$

As per the above bounds, we can plot the Empirical CDF, CDF and confidence intervals for different distributions by using any one of the statistical implementations.

Statistical implementation

A non-exhaustive list of software implementations of Empirical Distribution function includes:

• In R software, we compute an empirical cumulative distribution function, with several methods for plotting, printing and computing with such an “ecdf” object.
• In GNU Octave or MATLAB we can use Empirical cumulative distribution function (cdf) plot
• jmp from SAS, the CDF plot creates a plot of the empirical cumulative distribution function.
• Minitab, create an Empirical CDF
• Mathwave, we can fit probability distribution to our data
• Dataplot, we can plot Empirical CDF plot
• Scipy, we can use scipy.stats.ecdf
• Statsmodels, we can use statsmodels.distributions.empirical_distribution.ECDF
• Matplotlib, using the matplotlib.pyplot.ecdf function (new in version 3.8.0)
• Seaborn, using the seaborn.ecdfplot function
• Plotly, using the plotly.express.ecdf function
• Excel, we can plot Empirical CDF plot
• ArviZ, using the az.plot_dist function