Edgeworth series

The Edgeworth series

Edgeworth developed a similar expansion as an improvement to the central limit theorem. The advantage of the Edgeworth series is that the error is controlled, so that it is a true asymptotic expansion.

Let ${\displaystyle \{Z_i\}}$ be a sequence of independent and identically distributed random variables with finite mean ${\displaystyle \mu }$ and variance ${\displaystyle {\sigma }^2}$, and let ${\displaystyle X_n}$ be their standardized sums:

${\displaystyle X_n=\frac{1}{\sqrt{n}}\sum _{i=1}^n\frac{Z_i-\mu }{\sigma }.}$

Let ${\displaystyle F_n}$ denote the cumulative distribution functions of the variables ${\displaystyle X_n}$. Then by the central limit theorem,

${\displaystyle \underset{n\to \mathrm{\infty }}{lim}{F}_n(x)=\mathrm{\Phi }(x)\equiv {\int }_{-\mathrm{\infty }}^x\frac{1}{\sqrt{2\pi }}{e}^{-\frac{1}{2}{q}^2}dq}$

for every ${\displaystyle x}$, as long as the mean and variance are finite.

The standardization of ${\displaystyle \{Z_i\}}$ ensures that the first two cumulants of ${\displaystyle X_n}$ are ${\displaystyle {\kappa }_1^{F_n}=0}$ and ${\displaystyle {\kappa }_2^{F_n}=1.}$ Now assume that, in addition to having mean ${\displaystyle \mu }$ and variance ${\displaystyle {\sigma }^2}$, the i.i.d. random variables ${\displaystyle Z_i}$ have higher cumulants ${\displaystyle {\kappa }_r}$. From the additivity and homogeneity properties of cumulants, the cumulants of ${\displaystyle X_n}$ in terms of the cumulants of ${\displaystyle Z_i}$ are for ${\displaystyle r\ge 2}$,

${\displaystyle {\kappa }_r^{F_n}=\frac{n{\kappa }_r}{{\sigma }^r{n}^{r/2}}=\frac{{\lambda }_r}{n^{r/2-1}}\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}{\lambda }_r=\frac{{\kappa }_r}{{\sigma }^r}.}$

If we expand the formal expression of the characteristic function ${\displaystyle {\hat{f}}_n(t)}$ of ${\displaystyle F_n}$ in terms of the standard normal distribution, that is, if we set

${\displaystyle \phi (x)=\frac{1}{\sqrt{2\pi }}\exp (-\frac{1}{2}{x}^2),}$

then the cumulant differences in the expansion are

${\displaystyle {\kappa }_1^{F_n}-{\gamma }_1=0,}$

${\displaystyle {\kappa }_2^{F_n}-{\gamma }_2=0,}$

${\displaystyle {\kappa }_r^{F_n}-{\gamma }_r=\frac{{\lambda }_r}{n^{r/2-1}};r\ge 3.}$

The Gram–Charlier A series for the density function of ${\displaystyle X_n}$ is now

${\displaystyle f_n(x)=\phi (x)\sum _{r=0}^{\mathrm{\infty }}\frac{1}{r!}{B}_r\left(0,0,\frac{{\lambda }_3}{n^{1/2}},\dots ,\frac{{\lambda }_r}{n^{r/2-1}}\right)H{e}_r(x).}$

The Edgeworth series is developed similarly to the Gram–Charlier A series, only that now terms are collected according to powers of ${\displaystyle n}$. The coefficients of n^−m/2 term can be obtained by collecting the monomials of the Bell polynomials corresponding to the integer partitions of m. Thus, we have the characteristic function as

${\displaystyle {\hat{f}}_n(t)=\left[1+\sum _{j=1}^{\mathrm{\infty }}\frac{P_j(it)}{n^{j/2}}\right]\exp (-t^2/2),}$

where ${\displaystyle P_j(x)}$ is a polynomial of degree ${\displaystyle 3j}$. Again, after inverse Fourier transform, the density function ${\displaystyle f_n}$ follows as

${\displaystyle f_n(x)=\phi (x)+\sum _{j=1}^{\mathrm{\infty }}\frac{P_j(-D)}{n^{j/2}}\phi (x).}$

Likewise, integrating the series, we obtain the distribution function

${\displaystyle F_n(x)=\mathrm{\Phi }(x)+\sum _{j=1}^{\mathrm{\infty }}\frac{1}{n^{j/2}}\frac{P_j(-D)}{D}\phi (x).}$

We can explicitly write the polynomial ${\displaystyle P_m(-D)}$ as

${\displaystyle P_m(-D)=\sum \prod _i\frac{1}{k_i!}{\left(\frac{{\lambda }_{l_i}}{l_i!}\right)}^{k_i}(-D{)}^s,}$

where the summation is over all the integer partitions of m such that ${\displaystyle \sum _{i}i{k}_i=m}$ and ${\displaystyle l_i=i+2}$ and ${\displaystyle s=\sum _i{k}_i{l}_i.}$

For example, if m = 3, then there are three ways to partition this number: 1 + 1 + 1 = 2 + 1 = 3. As such we need to examine three cases:

• 1 + 1 + 1 = 1 · k₁, so we have k₁ = 3, l₁ = 3, and s = 9.
• 1 + 2 = 1 · k₁ + 2 · k₂, so we have k₁ = 1, k₂ = 1, l₁ = 3, l₂ = 4, and s = 7.
• 3 = 3 · k₃, so we have k₃ = 1, l₃ = 5, and s = 5.

Thus, the required polynomial is

${\displaystyle \begin{array}{cc}{P}_3(-D)& =\frac{1}{3!}{\left(\frac{{\lambda }_3}{3!}\right)}^3(-D{)}^9+\frac{1}{1!1!}\left(\frac{{\lambda }_3}{3!}\right)\left(\frac{{\lambda }_4}{4!}\right)(-D{)}^7+\frac{1}{1!}\left(\frac{{\lambda }_5}{5!}\right)(-D{)}^5\\ & =\frac{{\lambda }_3^3}{1296}(-D{)}^9+\frac{{\lambda }_3{\lambda }_4}{144}(-D{)}^7+\frac{{\lambda }_5}{120}(-D{)}^5.\end{array}}$

The first five terms of the expansion are

${\displaystyle \begin{array}{cc}{f}_n(x)& =\phi (x)\\ & -n^{-\frac{1}{2}}\left(\frac{1}{6}{\lambda }_3{\phi }^{(3)}(x)\right)\\ & +n^{-1}\left(\frac{1}{24}{\lambda }_4{\phi }^{(4)}(x)+\frac{1}{72}{\lambda }_3^2{\phi }^{(6)}(x)\right)\\ & -n^{-\frac{3}{2}}\left(\frac{1}{120}{\lambda }_5{\phi }^{(5)}(x)+\frac{1}{144}{\lambda }_3{\lambda }_4{\phi }^{(7)}(x)+\frac{1}{1296}{\lambda }_3^3{\phi }^{(9)}(x)\right)\\ & +n^{-2}\left(\frac{1}{720}{\lambda }_6{\phi }^{(6)}(x)+\left(\frac{1}{1152}{\lambda }_4^2+\frac{1}{720}{\lambda }_3{\lambda }_5\right){\phi }^{(8)}(x)+\frac{1}{1728}{\lambda }_3^2{\lambda }_4{\phi }^{(10)}(x)+\frac{1}{31104}{\lambda }_3^4{\phi }^{(12)}(x)\right)\\ & +O\left(n^{-\frac{5}{2}}\right).\end{array}}$

Here, φ^(j)(x) is the j-th derivative of φ(·) at point x. Remembering that the derivatives of the density of the normal distribution are related to the normal density by ${\displaystyle {\phi }^{(n)}(x)=(-1{)}^n{\mathrm{He}}_n(x)\phi (x)}$, (where ${\displaystyle H{e}_n}$ is the Hermite polynomial of order n), this explains the alternative representations in terms of the density function. Blinnikov and Moessner (1998) have given a simple algorithm to calculate higher-order terms of the expansion.

Note that in case of a lattice distributions (which have discrete values), the Edgeworth expansion must be adjusted to account for the discontinuous jumps between lattice points.

Illustration: density of the sample mean of three χ² distributions

Take ${\displaystyle X_i\sim {\chi }^2(k=2),i=1,2,3(n=3)}$ and the sample mean ${\displaystyle \overline{X}=\frac{1}{3}\sum _{i=1}^3{X}_i}$.

We can use several distributions for ${\displaystyle \overline{X}}$:

• The exact distribution, which follows a gamma distribution: ${\displaystyle \overline{X}\sim \mathrm{G}\mathrm{a}\mathrm{m}\mathrm{m}\mathrm{a}\left(\alpha =n\cdot k/2,\theta =2/n\right)=\mathrm{G}\mathrm{a}\mathrm{m}\mathrm{m}\mathrm{a}\left(\alpha =3,\theta =2/3\right)}$.
• The asymptotic normal distribution: ${\displaystyle \overline{X}\stackrel{n\to \mathrm{\infty }}{\to }N(k,2\cdot k/n)=N(2,4/3)}$.
• Two Edgeworth expansions, of degrees 2 and 3.

Discussion of results

• For finite samples, an Edgeworth expansion is not guaranteed to be a proper probability distribution as the CDF values at some points may go beyond ${\displaystyle [0,1]}$.
• They guarantee (asymptotically) absolute errors, but relative errors can be easily assessed by comparing the leading Edgeworth term in the remainder with the overall leading term.