Normal distribution

Properties

The normal distribution is the only distribution whose cumulants beyond the first two (i.e., other than the mean and variance) are zero. It is also the continuous distribution with the maximum entropy for a specified mean and variance. Geary has shown, assuming that the mean and variance are finite, that the normal distribution is the only distribution where the mean and variance calculated from a set of independent draws are independent of each other.

The normal distribution is a subclass of the elliptical distributions. The normal distribution is symmetric about its mean, and is non-zero over the entire real line. As such it may not be a suitable model for variables that are inherently positive or strongly skewed, such as the weight of a person or the price of a share of stock. Such variables may be better described by other distributions, such as the log-normal distribution or the Pareto distribution.

The value of the normal density is practically zero when the value ⁠${\displaystyle x}$⁠ lies more than a few standard deviations away from the mean (e.g., a spread of three standard deviations covers all but 0.27% of the total distribution). Therefore, it may not be an appropriate model when one expects a significant fraction of outliers—values that lie many standard deviations away from the mean—and least squares and other statistical inference methods that are optimal for normally distributed variables often become highly unreliable when applied to such data. In those cases, a more heavy-tailed distribution should be assumed and appropriate robust statistical inference methods applied.

The Gaussian distribution belongs to the family of stable distributions which are the attractors of sums of independent, identically distributed distributions whether or not the mean or variance is finite. Except for the Gaussian which is a limiting case, all stable distributions have heavy tails and infinite variance. It is one of the few distributions that are stable and that have probability density functions that can be expressed analytically, the others being the Cauchy distribution and the Lévy distribution.

Symmetries and derivatives

The normal distribution with density $f(x)$ (mean ⁠${\displaystyle \mu }$⁠ and variance ${\sigma }^2>0$) has the following properties:

• It is symmetric around the point $x=\mu ,$ which is at the same time the mode, the median and the mean of the distribution.
• It is unimodal: its first derivative is positive for $x<\mu ,$ negative for $x>\mu ,$ and zero only at $x=\mu .$
• The area bounded by the curve and the ⁠${\displaystyle x}$⁠-axis is unity (i.e. equal to one).
• Its first derivative is $f^{\prime }(x)=-\frac{x-\mu }{{\sigma }^2}f(x).$
• Its second derivative is $f^{″}(x)=\frac{(x-\mu {)}^2-{\sigma }^2}{{\sigma }^4}f(x).$
• Its density has two inflection points (where the second derivative of ⁠${\displaystyle f}$⁠ is zero and changes sign), located one standard deviation away from the mean, namely at $x=\mu -\sigma$ and $x=\mu +\sigma .$
• Its density is log-concave.
• Its density is infinitely differentiable, indeed supersmooth of order 2.

Furthermore, the density ⁠${\displaystyle \phi }$⁠ of the standard normal distribution (i.e. $\mu =0$ and $\sigma =1$) also has the following properties:

• Its first derivative is ${\phi }^{\prime }(x)=-x\phi (x).$
• Its second derivative is ${\phi }^{″}(x)=(x^2-1)\phi (x)$
• More generally, its nth derivative is ${\phi }^{(n)}(x)=(-1{)}^n{\mathrm{He}}_n(x)\phi (x),$ where ${\mathrm{He}}_n(x)$ is the nth (probabilist) Hermite polynomial.
• The probability that a normally distributed variable ⁠${\displaystyle X}$⁠ with known ⁠${\displaystyle \mu }$⁠ and ${\sigma }^2$ is in a particular set, can be calculated given that the fraction $Z=(X-\mu )/\sigma$ has a standard normal distribution.

Moments

The plain and absolute moments of a variable ⁠${\displaystyle X}$⁠ are the expected values of $X^p$ and $|X{|}^p$, respectively. If the expected value ⁠${\displaystyle \mu }$⁠ of ⁠${\displaystyle X}$⁠ is zero, these parameters are called central moments; otherwise, these parameters are called non-central moments. Usually we are interested only in moments with integer order ⁠${\displaystyle p}$⁠.

If ⁠${\displaystyle X}$⁠ has a normal distribution, the non-central moments exist and are finite for any ⁠${\displaystyle p}$⁠ whose real part is greater than −1. For any non-negative integer ⁠${\displaystyle p}$⁠, the plain central moments are: $${\displaystyle \mathrm{E}\left[(X-\mu {)}^p\right]=\{\begin{array}{ll}0& \text{if }p\text{ is odd,}\\ {\sigma }^p(p-1)!!& \text{if }p\text{ is even.}\end{array}}$$ Here $n!!$ denotes the double factorial, that is, the product of all numbers from ⁠${\displaystyle n}$⁠ to 1 that have the same parity as $n.$

The central absolute moments coincide with plain moments for all even orders, but are nonzero for odd orders. For any non-negative integer $p,$

$${\displaystyle \begin{array}{cc}\mathrm{E}\left[|X-\mu {|}^p\right]& ={\sigma }^p(p-1)!!\cdot \{\begin{array}{ll}\sqrt{\frac{2}{\pi }}& \text{if }p\text{ is odd}\\ 1& \text{if }p\text{ is even}\end{array}\\ & ={\sigma }^p\cdot \frac{2^{p/2}\mathrm{\Gamma }\left(\frac{p+1}{2}\right)}{\sqrt{\pi }}.\end{array}}$$ The last formula is valid also for any non-integer $p>-1.$ When the mean $\mu \ne 0,$ the plain and absolute moments can be expressed in terms of confluent hypergeometric functions ${}_1{F}_1$ and $U.$ $${\displaystyle \begin{array}{cc}\mathrm{E}\left[X^p\right]& ={\sigma }^p\cdot {\left(-i\sqrt{2}\right)}^{p}U\left(-\frac{p}{2},\frac{1}{2},-\frac{{\mu }^2}{2{\sigma }^2}\right),\\ \mathrm{E}\left[|X{|}^p\right]& ={\sigma }^p\cdot 2^{p/2}\frac{\mathrm{\Gamma }\left(\frac{1+p}{2}\right)}{\sqrt{\pi }}{}_1{F}_1\left(-\frac{p}{2},\frac{1}{2},-\frac{{\mu }^2}{2{\sigma }^2}\right).\end{array}}$$

These expressions remain valid even when ⁠${\displaystyle p>-1}$⁠ is not an integer. See also generalized Hermite polynomials.

The expectation of ⁠${\displaystyle X}$⁠ conditioned on the event that ⁠${\displaystyle X}$⁠ lies in an interval $[a,b]$ is given by $${\displaystyle \mathrm{E}\left[X\mid a<X<b\right]=\mu -{\sigma }^2\frac{f(b)-f(a)}{F(b)-F(a)},}$$ where ⁠${\displaystyle f}$⁠ and ⁠${\displaystyle F}$⁠ respectively are the density and the cumulative distribution function of ⁠${\displaystyle X}$⁠. For $b=\mathrm{\infty }$ this is known as the inverse Mills ratio. Note that above, density ⁠${\displaystyle f}$⁠ of ⁠${\displaystyle X}$⁠ is used instead of standard normal density as in inverse Mills ratio, so here we have ${\sigma }^2$ instead of ⁠${\displaystyle \sigma }$⁠.

Fourier transform and characteristic function

The Fourier transform of a normal density ⁠${\displaystyle f}$⁠ with mean ⁠${\displaystyle \mu }$⁠ and variance ${\sigma }^2$ is

$${\displaystyle \hat{f}(t)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x)e^{-itx}dx=e^{-i\mu t}{e}^{-\frac{1}{2}{\sigma }^2{t}^2},}$$

where ⁠${\displaystyle i}$⁠ is the imaginary unit. If the mean $\mu =0$, the first factor is 1, and the Fourier transform is, apart from a constant factor, a normal density on the frequency domain, with mean 0 and variance ⁠${\displaystyle 1/{\sigma }^2}$⁠. In particular, the standard normal distribution ⁠${\displaystyle \phi }$⁠ is an eigenfunction of the Fourier transform.

In probability theory, the Fourier transform of the probability distribution of a real-valued random variable ⁠${\displaystyle X}$⁠ is closely connected to the characteristic function ${\phi }_X(t)$ of that variable, which is defined as the expected value of $e^{itX}$, as a function of the real variable ⁠${\displaystyle t}$⁠ (the frequency parameter of the Fourier transform). This definition can be analytically extended to a complex-value variable ⁠${\displaystyle t}$⁠. The relation between both is: $${\displaystyle {\phi }_X(t)=\hat{f}(-t).}$$

The real and imaginary parts of ${\displaystyle \hat{f}(t)=\mathrm{E}[e^{-itx}]=e^{-i\mu t}{e}^{-\frac{1}{2}{\sigma }^2{t}^2}}$ give: $${\displaystyle \mathrm{E}[\cos (tx)]=\cos (\mu t)e^{-\frac{1}{2}{\sigma }^2{t}^2}}$$ and $${\displaystyle \mathrm{E}[\sin (tx)]=\sin (\mu t)e^{-\frac{1}{2}{\sigma }^2{t}^2}.}$$

Similarly, $${\displaystyle \mathrm{E}[\cosh (tx)]=\cosh (\mu t)e^{\frac{1}{2}{\sigma }^2{t}^2}}$$ and $${\displaystyle \mathrm{E}[\sinh (tx)]=\sinh (\mu t)e^{\frac{1}{2}{\sigma }^2{t}^2}.}$$

These formulas evaluated at ${\displaystyle t=1}$ give the expected value of these basic trigonometric and hyperbolic functions over a Gaussian random variable ${\displaystyle X\sim N(\mu ,{\sigma }^2)}$, which also could be seen as consequences of the Isserlis's theorem.

Moment- and cumulant-generating functions

The moment generating function of a real random variable ⁠${\displaystyle X}$⁠ is the expected value of $e^{tX}$, as a function of the real parameter ⁠${\displaystyle t}$⁠. For a normal distribution with density ⁠${\displaystyle f}$⁠, mean ⁠${\displaystyle \mu }$⁠ and variance ${\sigma }^2$, the moment generating function exists and is equal to

$${\displaystyle M(t)=\mathrm{E}\left[e^{tX}\right]=\hat{f}(it)=e^{\mu t}{e}^{{\sigma }^2{t}^2/2}.}$$ For any ⁠${\displaystyle k}$⁠, the coefficient of ⁠${\displaystyle t^k/k!}$⁠ in the moment generating function (expressed as an exponential power series in ⁠${\displaystyle t}$⁠) is the normal distribution's expected value ⁠${\displaystyle \mathrm{E}[X^k]}$⁠.

The cumulant generating function is the logarithm of the moment generating function, namely $${\displaystyle g(t)=\ln M(t)=\mu t+\frac{1}{2}{\sigma }^2{t}^2.}$$

The coefficients of this exponential power series define the cumulants, but because this is a quadratic polynomial in ⁠${\displaystyle t}$⁠, only the first two cumulants are nonzero, namely the mean ⁠${\displaystyle \mu }$⁠ and the variance ⁠${\displaystyle {\sigma }^2}$⁠.

Some authors prefer to instead work with the characteristic function E[e^itX] = e^{iμt − σ2t2/2} and ln E[e^itX] = iμt − ⁠1/2⁠σ²t².

Stein operator and class

Within Stein's method the Stein operator and class of a random variable $X\sim \mathcal{N}(\mu ,{\sigma }^2)$ are $\mathcal{A}f(x)={\sigma }^2{f}^{\prime }(x)-(x-\mu )f(x)$ and $\mathcal{F}$ the class of all absolutely continuous functions ⁠${\displaystyle f:\mathbb{R}\to \mathbb{R}}$⁠ such that ⁠${\displaystyle \mathrm{E}[|f^{\prime }(X)|]<\mathrm{\infty }}$⁠.

Zero-variance limit

In the limit when ${\sigma }^2$ approaches zero, the probability density $f$ approaches zero everywhere except at $\mu$, where it approaches $\mathrm{\infty }$, while its integral remains equal to 1. An extension of the normal distribution to the case with zero variance can be defined using the Dirac delta measure ${\delta }_{\mu }$, although the resulting random variables are not absolutely continuous and thus do not have probability density functions. The cumulative distribution function of such a random variable is then the Heaviside step function translated by the mean $\mu$, namely $${\displaystyle F(x)=\{\begin{array}{ll}0& \text{if }x<\mu \\ 1& \text{if }x\ge \mu .\end{array}}$$

Maximum entropy

Of all probability distributions over the reals with a specified finite mean ⁠${\displaystyle \mu }$⁠ and finite variance ⁠${\displaystyle {\sigma }^2}$⁠, the normal distribution $N(\mu ,{\sigma }^2)$ is the one with maximum entropy. To see this, let ⁠${\displaystyle X}$⁠ be a continuous random variable with probability density ⁠${\displaystyle f(x)}$⁠. The entropy of ⁠${\displaystyle X}$⁠ is defined as $${\displaystyle H(X)=-{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x)\ln f(x)dx,}$$ where $f(x)\log f(x)$ is understood to be zero whenever ⁠${\displaystyle f(x)=0}$⁠. This functional can be maximized, subject to the constraints that the distribution is properly normalized and has a specified mean and variance, by using variational calculus. A function with three Lagrange multipliers is defined: $${\displaystyle L=-{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x)\ln f(x)dx-{\lambda }_0\left(1-{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x)dx\right)-{\lambda }_1\left(\mu -{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x)xdx\right)-{\lambda }_2\left({\sigma }^2-{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x)(x-\mu {)}^{2}dx\right).}$$

At maximum entropy, a small variation $\delta f(x)$ about $f(x)$ will produce a variation $\delta L$ about ⁠${\displaystyle L}$⁠ which is equal to 0: $${\displaystyle 0=\delta L={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\delta f(x)\left(-\ln f(x)-1+{\lambda }_0+{\lambda }_{1}x+{\lambda }_2(x-\mu {)}^2\right)dx.}$$

Since this must hold for any small ⁠${\displaystyle \delta f(x)}$⁠, the factor multiplying ⁠${\displaystyle \delta f(x)}$⁠ must be zero, and solving for ⁠${\displaystyle f(x)}$⁠ yields: $${\displaystyle f(x)=\exp \left(-1+{\lambda }_0+{\lambda }_{1}x+{\lambda }_2(x-\mu {)}^2\right).}$$

The Lagrange constraints that ⁠${\displaystyle f(x)}$⁠ is properly normalized and has the specified mean and variance are satisfied if and only if ⁠${\displaystyle {\lambda }_0}$⁠, ⁠${\displaystyle {\lambda }_1}$⁠, and ⁠${\displaystyle {\lambda }_2}$⁠ are chosen so that $${\displaystyle f(x)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}.}$$ The entropy of a normal distribution $X\sim N(\mu ,{\sigma }^2)$ is equal to $${\displaystyle H(X)=\frac{1}{2}(1+\ln 2{\sigma }^2\pi ),}$$ which is independent of the mean ⁠${\displaystyle \mu }$⁠.

Other properties

1. If the characteristic function ${\varphi }_X$ of some random variable ⁠${\displaystyle X}$⁠ is of the form ${\varphi }_X(t)=\exp Q(t)$ in a neighborhood of zero, where $Q(t)$ is a polynomial, then the Marcinkiewicz theorem (named after Józef Marcinkiewicz) asserts that ⁠${\displaystyle Q}$⁠ can be at most a quadratic polynomial, and therefore ⁠${\displaystyle X}$⁠ is a normal random variable. The consequence of this result is that the normal distribution is the only distribution with a finite number (two) of non-zero cumulants.
2. If ⁠${\displaystyle X}$⁠ and ⁠${\displaystyle Y}$⁠ are jointly normal and uncorrelated, then they are independent. The requirement that ⁠${\displaystyle X}$⁠ and ⁠${\displaystyle Y}$⁠ should be jointly normal is essential; without it the property does not hold.^[proof] For non-normal random variables uncorrelatedness does not imply independence.
3. The Kullback–Leibler divergence of one normal distribution $X_1\sim N({\mu }_1,{\sigma }_1^2)$ from another $X_2\sim N({\mu }_2,{\sigma }_2^2)$ is given by: $${\displaystyle D_{\mathrm{K}\mathrm{L}}(X_1\parallel X_2)=\frac{({\mu }_1-{\mu }_2{)}^2}{2{\sigma }_2^2}+\frac{1}{2}\left(\frac{{\sigma }_1^2}{{\sigma }_2^2}-1-\ln \frac{{\sigma }_1^2}{{\sigma }_2^2}\right)}$$ The Hellinger distance between the same distributions is equal to $${\displaystyle H^2(X_1,X_2)=1-\sqrt{\frac{2{\sigma }_1{\sigma }_2}{{\sigma }_1^2+{\sigma }_2^2}}\exp \left(-\frac{1}{4}\frac{({\mu }_1-{\mu }_2{)}^2}{{\sigma }_1^2+{\sigma }_2^2}\right)}$$
4. The Fisher information matrix for a normal distribution w.r.t. ⁠${\displaystyle \mu }$⁠ and ${\sigma }^2$ is diagonal and takes the form $${\displaystyle \mathcal{I}(\mu ,{\sigma }^2)=\left(\begin{array}{cc}\frac{1}{{\sigma }^2}& 0\\ 0& \frac{1}{2{\sigma }^4}\end{array}\right)}$$
5. The conjugate prior of the mean of a normal distribution is another normal distribution. Specifically, if $x_1,\dots ,x_n$ are iid $\sim N(\mu ,{\sigma }^2)$ and the prior is $\mu \sim N({\mu }_0,{\sigma }_0^2)$, then the posterior distribution for the estimator of ⁠${\displaystyle \mu }$⁠ will be $${\displaystyle \mu \mid x_1,\dots ,x_n\sim \mathcal{N}\left(\frac{\frac{{\sigma }^2}{n}{\mu }_0+{\sigma }_0^2\overline{x}}{\frac{{\sigma }^2}{n}+{\sigma }_0^2},{\left(\frac{n}{{\sigma }^2}+\frac{1}{{\sigma }_0^2}\right)}^{-1}\right)}$$
6. The family of normal distributions not only forms an exponential family (EF), but in fact forms a natural exponential family (NEF) with quadratic variance function (NEF-QVF). Many properties of normal distributions generalize to properties of NEF-QVF distributions, NEF distributions, or EF distributions generally. NEF-QVF distributions comprises 6 families, including Poisson, Gamma, binomial, and negative binomial distributions, while many of the common families studied in probability and statistics are NEF or EF.
7. In information geometry, the family of normal distributions forms a statistical manifold with constant curvature ⁠${\displaystyle -1}$⁠. The same family is flat with respect to the (±1)-connections ${\mathrm{\nabla }}^{(e)}$ and ${\mathrm{\nabla }}^{(m)}$.
8. If $X_1,\dots ,X_n$ are distributed according to $N(0,{\sigma }^2)$, then $E[\underset{i}{max}{X}_i]\le \sigma \sqrt{2\ln n}$. Note that there is no assumption of independence.

Related distributions

Central limit theorem

The central limit theorem states that under certain (fairly common) conditions, the sum of many random variables will have an approximately normal distribution. More specifically, where $X_1,\dots ,X_n$ are independent and identically distributed random variables with the same arbitrary distribution, zero mean, and variance ${\sigma }^2$ and ⁠${\displaystyle Z}$⁠ is their mean scaled by $\sqrt{n}$ $${\displaystyle Z=\sqrt{n}(\frac{1}{n}\sum _{i=1}^n{X}_i)}$$ Then, as ⁠${\displaystyle n}$⁠ increases, the probability distribution of ⁠${\displaystyle Z}$⁠ will tend to the normal distribution with zero mean and variance ⁠${\displaystyle {\sigma }^2}$⁠.

The theorem can be extended to variables $(X_i)$ that are not independent and/or not identically distributed if certain constraints are placed on the degree of dependence and the moments of the distributions.

Many test statistics, scores, and estimators encountered in practice contain sums of certain random variables in them, and even more estimators can be represented as sums of random variables through the use of influence functions. The central limit theorem implies that those statistical parameters will have asymptotically normal distributions.

The central limit theorem also implies that certain distributions can be approximated by the normal distribution, for example:

• The binomial distribution $B(n,p)$ is approximately normal with mean $np$ and variance $np(1-p)$ for large ⁠${\displaystyle n}$⁠ and for ⁠${\displaystyle p}$⁠ not too close to 0 or 1.
• The Poisson distribution with parameter ⁠${\displaystyle \lambda }$⁠ is approximately normal with mean ⁠${\displaystyle \lambda }$⁠ and variance ⁠${\displaystyle \lambda }$⁠, for large values of ⁠${\displaystyle \lambda }$⁠.
• The chi-squared distribution ${\chi }^2(k)$ is approximately normal with mean ⁠${\displaystyle k}$⁠ and variance $2k$, for large ⁠${\displaystyle k}$⁠.
• The Student's t-distribution $t(\nu )$ is approximately normal with mean 0 and variance 1 when ⁠${\displaystyle \nu }$⁠ is large.

Whether these approximations are sufficiently accurate depends on the purpose for which they are needed, and the rate of convergence to the normal distribution. It is typically the case that such approximations are less accurate in the tails of the distribution.

A general upper bound for the approximation error in the central limit theorem is given by the Berry–Esseen theorem, improvements of the approximation are given by the Edgeworth expansions.

This theorem can also be used to justify modeling the sum of many uniform noise sources as Gaussian noise. See AWGN.

Operations and functions of normal variables

Operations on a single normal variable

If ⁠${\displaystyle X}$⁠ is distributed normally with mean ⁠${\displaystyle \mu }$⁠ and variance ${\sigma }^2$, then

• $aX+b$, for any real numbers ⁠${\displaystyle a}$⁠ and ⁠${\displaystyle b}$⁠, is also normally distributed, with mean $a\mu +b$ and variance $a^2{\sigma }^2$. That is, the family of normal distributions is closed under linear transformations.
• The exponential of ⁠${\displaystyle X}$⁠ is distributed log-normally: $e^X\sim \ln (N(\mu ,{\sigma }^2))$.
• The standard sigmoid of ⁠${\displaystyle X}$⁠ is logit-normally distributed: $\sigma (X)\sim P(\mathcal{N}(\mu ,{\sigma }^2))$.
• The absolute value of ⁠${\displaystyle X}$⁠ has folded normal distribution: $\left|X\right|\sim N_f(\mu ,{\sigma }^2)$. If $\mu =0$ this is known as the half-normal distribution.
• The absolute value of normalized residuals, $|X-\mu |/\sigma$, has chi distribution with one degree of freedom: $|X-\mu |/\sigma \sim {\chi }_1$.
• The square of $X/\sigma$ has the noncentral chi-squared distribution with one degree of freedom: $X^2/{\sigma }^2\sim {\chi }_1^2({\mu }^2/{\sigma }^2)$. If $\mu =0$, the distribution is called simply chi-squared.
• The log-likelihood of a normal variable ⁠${\displaystyle x}$⁠ is simply the log of its probability density function: $${\displaystyle \ln p(x)=-\frac{1}{2}{\left(\frac{x-\mu }{\sigma }\right)}^2-\ln \left(\sigma \sqrt{2\pi }\right).}$$ Since this is a scaled and shifted square of a standard normal variable, it is distributed as a scaled and shifted chi-squared variable.
• The distribution of the variable ⁠${\displaystyle X}$⁠ restricted to an interval $[a,b]$ is called the truncated normal distribution.
• $(X-\mu {)}^{-2}$ has a Lévy distribution with location 0 and scale ${\sigma }^{-2}$.

Operations on two independent normal variables

• If $X_1$ and $X_2$ are two independent normal random variables, with means ${\mu }_1$, ${\mu }_2$ and variances ${\sigma }_1^2$, ${\sigma }_2^2$, then their sum $X_1+X_2$ will also be normally distributed,^[proof] with mean ${\mu }_1+{\mu }_2$ and variance ${\sigma }_1^2+{\sigma }_2^2$.
• In particular, if ⁠${\displaystyle X}$⁠ and ⁠${\displaystyle Y}$⁠ are independent normal deviates with zero mean and variance ${\sigma }^2$, then $X+Y$ and $X-Y$ are also independent and normally distributed, with zero mean and variance $2{\sigma }^2$. This is a special case of the polarization identity.
• If $X_1$, $X_2$ are two independent normal deviates with mean ⁠${\displaystyle \mu }$⁠ and variance ${\sigma }^2$, and ⁠${\displaystyle a}$⁠, ⁠${\displaystyle b}$⁠ are arbitrary real numbers, then the variable $${\displaystyle X_3=\frac{a{X}_1+b{X}_2-(a+b)\mu }{\sqrt{a^2+b^2}}+\mu }$$ is also normally distributed with mean ⁠${\displaystyle \mu }$⁠ and variance ${\sigma }^2$. It follows that the normal distribution is stable (with exponent $\alpha =2$).
• If $X_k\sim \mathcal{N}(m_k,{\sigma }_k^2)$, $k\in \{0,1\}$ are normal distributions, then their normalized geometric mean $\frac{1}{{\int }_{{\mathbb{R}}^n}{X}_0^{\alpha }(x)X_1^{1-\alpha }(x)\text{d}x}{X}_0^{\alpha }{X}_1^{1-\alpha }$ is a normal distribution $\mathcal{N}(m_{\alpha },{\sigma }_{\alpha }^2)$ with $m_{\alpha }=\frac{\alpha m_0{\sigma }_1^2+(1-\alpha )m_1{\sigma }_0^2}{\alpha {\sigma }_1^2+(1-\alpha ){\sigma }_0^2}$ and ${\sigma }_{\alpha }^2=\frac{{\sigma }_0^2{\sigma }_1^2}{\alpha {\sigma }_1^2+(1-\alpha ){\sigma }_0^2}$.

Operations on two independent standard normal variables

If $X_1$ and $X_2$ are two independent standard normal random variables with mean 0 and variance 1, then

• Their sum and difference is distributed normally with mean zero and variance two: $X_1\pm X_2\sim \mathcal{N}(0,2)$.
• Their product $Z=X_1{X}_2$ follows the product distribution with density function $f_Z(z)={\pi }^{-1}{K}_0(|z|)$ where $K_0$ is the modified Bessel function of the second kind. This distribution is symmetric around zero, unbounded at $z=0$, and has the characteristic function ${\varphi }_Z(t)=(1+t^2{)}^{-1/2}$.
• Their ratio follows the standard Cauchy distribution: $X_1/X_2\sim \mathrm{Cauchy}(0,1)$.
• Their Euclidean norm $\sqrt{X_1^2+X_2^2}$ has the Rayleigh distribution.

Operations on multiple independent normal variables

• Any linear combination of independent normal deviates is a normal deviate.
• If $X_1,X_2,\dots ,X_n$ are independent standard normal random variables, then the sum of their squares has the chi-squared distribution with ⁠${\displaystyle n}$⁠ degrees of freedom $${\displaystyle X_1^2+\cdots +X_n^2\sim {\chi }_n^2.}$$
• If $X_1,X_2,\dots ,X_n$ are independent normally distributed random variables with means ⁠${\displaystyle \mu }$⁠ and variances ${\sigma }^2$, then their sample mean is independent from the sample standard deviation, which can be demonstrated using Basu's theorem or Cochran's theorem. The ratio of these two quantities will have the Student's t-distribution with $n-1$ degrees of freedom: $${\displaystyle t=\frac{\overline{X}-\mu }{S/\sqrt{n}}=\frac{\frac{1}{n}(X_1+\cdots +X_n)-\mu }{\sqrt{\frac{1}{n(n-1)}\left[(X_1-\overline{X}{)}^2+\cdots +(X_n-\overline{X}{)}^2\right]}}\sim t_{n-1}.}$$
• If $X_1,X_2,\dots ,X_n$, $Y_1,Y_2,\dots ,Y_m$ are independent standard normal random variables, then the ratio of their normalized sums of squares will have the F-distribution with (n, m) degrees of freedom: $${\displaystyle F=\frac{\left(X_1^2+X_2^2+\cdots +X_n^2\right)/n}{\left(Y_1^2+Y_2^2+\cdots +Y_m^2\right)/m}\sim F_{n,m}.}$$

Operations on multiple correlated normal variables

• A quadratic form of a normal vector, i.e. a quadratic function $q=\sum x_i^2+\sum x_j+c$ of multiple independent or correlated normal variables, is a generalized chi-square variable.

Operations on the density function

The split normal distribution is most directly defined in terms of joining scaled sections of the density functions of different normal distributions and rescaling the density to integrate to one. The truncated normal distribution results from rescaling a section of a single density function.

Infinite divisibility and Cramér's theorem

For any positive integer n, any normal distribution with mean ⁠${\displaystyle \mu }$⁠ and variance ${\sigma }^2$ is the distribution of the sum of n independent normal deviates, each with mean $\frac{\mu }{n}$ and variance $\frac{{\sigma }^2}{n}$. This property is called infinite divisibility.

Conversely, if $X_1$ and $X_2$ are independent random variables and their sum $X_1+X_2$ has a normal distribution, then both $X_1$ and $X_2$ must be normal deviates.

This result is known as Cramér's decomposition theorem, and is equivalent to saying that the convolution of two distributions is normal if and only if both are normal. Cramér's theorem implies that a linear combination of independent non-Gaussian variables will never have an exactly normal distribution, although it may approach it arbitrarily closely.

The Kac–Bernstein theorem

The Kac–Bernstein theorem states that if $X$ and ⁠${\displaystyle Y}$⁠ are independent and $X+Y$ and $X-Y$ are also independent, then both X and Y must necessarily have normal distributions.

More generally, if $X_1,\dots ,X_n$ are independent random variables, then two distinct linear combinations $\sum a_k{X}_k$ and $\sum b_k{X}_k$will be independent if and only if all $X_k$ are normal and $\sum a_k{b}_k{\sigma }_k^2=0$, where ${\sigma }_k^2$ denotes the variance of $X_k$.

Extensions

The notion of normal distribution, being one of the most important distributions in probability theory, has been extended far beyond the standard framework of the univariate (that is one-dimensional) case (Case 1). All these extensions are also called normal or Gaussian laws, so a certain ambiguity in names exists.

• The multivariate normal distribution describes the Gaussian law in the k-dimensional Euclidean space. A vector X ∈ R^k is multivariate-normally distributed if any linear combination of its components Σ^k
  _j=1a_j X_j has a (univariate) normal distribution. The variance of X is a k × k symmetric positive-definite matrix V. The multivariate normal distribution is a special case of the elliptical distributions. As such, its iso-density loci in the k = 2 case are ellipses and in the case of arbitrary k are ellipsoids.
• Rectified Gaussian distribution a rectified version of normal distribution with all the negative elements reset to 0.
• Complex normal distribution deals with the complex normal vectors. A complex vector X ∈ C^k is said to be normal if both its real and imaginary components jointly possess a 2k-dimensional multivariate normal distribution. The variance-covariance structure of X is described by two matrices: the variance matrix Γ, and the relation matrix C.
• Matrix normal distribution describes the case of normally distributed matrices.
• Gaussian processes are the normally distributed stochastic processes. These can be viewed as elements of some infinite-dimensional Hilbert space H, and thus are the analogues of multivariate normal vectors for the case k = ∞. A random element h ∈ H is said to be normal if for any constant a ∈ H the scalar product (a, h) has a (univariate) normal distribution. The variance structure of such Gaussian random element can be described in terms of the linear covariance operator K: H → H. Several Gaussian processes became popular enough to have their own names:
  • Brownian motion;
  • Brownian bridge; and
  • Ornstein–Uhlenbeck process.
• Gaussian q-distribution is an abstract mathematical construction that represents a q-analogue of the normal distribution.
• the q-Gaussian is an analogue of the Gaussian distribution, in the sense that it maximises the Tsallis entropy, and is one type of Tsallis distribution. This distribution is different from the Gaussian q-distribution above.
• The Kaniadakis κ-Gaussian distribution is a generalization of the Gaussian distribution which arises from the Kaniadakis statistics, being one of the Kaniadakis distributions.

A random variable X has a two-piece normal distribution if it has a distribution $${\displaystyle f_X(x)=\{\begin{array}{ll}N(\mu ,{\sigma }_1^2),& \text{ if }x\le \mu \\ N(\mu ,{\sigma }_2^2),& \text{ if }x\ge \mu \end{array}}$$ where μ is the mean and σ2
1 and σ2
2 are the variances of the distribution to the left and right of the mean respectively.

The mean E(X), variance V(X), and third central moment T(X) of this distribution have been determined $${\displaystyle \begin{array}{cc}\mathrm{E}(X)& =\mu +\sqrt{\frac{2}{\pi }}({\sigma }_2-{\sigma }_1),\\ \mathrm{V}(X)& =\left(1-\frac{2}{\pi }\right)({\sigma }_2-{\sigma }_1{)}^2+{\sigma }_1{\sigma }_2,\\ \mathrm{T}(X)& =\sqrt{\frac{2}{\pi }}({\sigma }_2-{\sigma }_1)\left[\left(\frac{4}{\pi }-1\right)({\sigma }_2-{\sigma }_1{)}^2+{\sigma }_1{\sigma }_2\right].\end{array}}$$

One of the main practical uses of the Gaussian law is to model the empirical distributions of many different random variables encountered in practice. In such case a possible extension would be a richer family of distributions, having more than two parameters and therefore being able to fit the empirical distribution more accurately. The examples of such extensions are:

• Pearson distribution — a four-parameter family of probability distributions that extend the normal law to include different skewness and kurtosis values.
• The generalized normal distribution, also known as the exponential power distribution, allows for distribution tails with thicker or thinner asymptotic behaviors.

Statistical inference

Estimation of parameters

It is often the case that we do not know the parameters of the normal distribution, but instead want to estimate them. That is, having a sample $(x_1,\dots ,x_n)$ from a normal $\mathcal{N}(\mu ,{\sigma }^2)$ population we would like to learn the approximate values of parameters ⁠${\displaystyle \mu }$⁠ and ${\sigma }^2$. The standard approach to this problem is the maximum likelihood method, which requires maximization of the log-likelihood function: $${\displaystyle \begin{array}{cc}\ln \mathcal{L}(\mu ,{\sigma }^2)& =\sum _{i=1}^n\ln f(x_i\mid \mu ,{\sigma }^2)\\ & =-\frac{n}{2}\ln (2\pi )-\frac{n}{2}\ln {\sigma }^2-\frac{1}{2{\sigma }^2}\sum _{i=1}^n(x_i-\mu {)}^2.\end{array}}$$ Taking derivatives with respect to ⁠${\displaystyle \mu }$⁠ and ${\sigma }^2$ and solving the resulting system of first order conditions yields the maximum likelihood estimates: $${\displaystyle \hat{\mu }=\overline{x}\equiv \frac{1}{n}\sum _{i=1}^n{x}_i,{\hat{\sigma }}^2=\frac{1}{n}\sum _{i=1}^n(x_i-\overline{x}{)}^2.}$$

Then $\ln \mathcal{L}(\hat{\mu },{\hat{\sigma }}^2)$ is as follows: $${\displaystyle \ln \mathcal{L}(\hat{\mu },{\hat{\sigma }}^2)=-\frac{n}{2}[\ln \left(2\pi {\hat{\sigma }}^2\right)+1]}$$

Sample mean

Estimator ${\displaystyle \hat{\mu }}$ is called the sample mean, since it is the arithmetic mean of all observations. The statistic ${\displaystyle \overline{x}}$ is complete and sufficient for ⁠${\displaystyle \mu }$⁠, and therefore by the Lehmann–Scheffé theorem, ${\displaystyle \hat{\mu }}$ is the uniformly minimum variance unbiased (UMVU) estimator. In finite samples it is distributed normally: $${\displaystyle \hat{\mu }\sim \mathcal{N}(\mu ,{\sigma }^2/n).}$$ The variance of this estimator is equal to the μμ-element of the inverse Fisher information matrix ${\displaystyle {\mathcal{I}}^{-1}}$. This implies that the estimator is finite-sample efficient. Of practical importance is the standard error of ${\displaystyle \hat{\mu }}$ being proportional to ${\displaystyle 1/\sqrt{n}}$, that is, if one wishes to decrease the standard error by a factor of 10, one must increase the number of points in the sample by a factor of 100. This fact is widely used in determining sample sizes for opinion polls and the number of trials in Monte Carlo simulations.

From the standpoint of the asymptotic theory, ${\displaystyle \hat{\mu }}$ is consistent, that is, it converges in probability to ⁠${\displaystyle \mu }$⁠ as $n\to \mathrm{\infty }$. The estimator is also asymptotically normal, which is a simple corollary of it being normal in finite samples: $${\displaystyle \sqrt{n}(\hat{\mu }-\mu )\stackrel{d}{\to }\mathcal{N}(0,{\sigma }^2).}$$

Sample variance

The estimator ${\displaystyle {\hat{\sigma }}^2}$ is called the sample variance, since it is the variance of the sample ($(x_1,\dots ,x_n)$). In practice, another estimator is often used instead of the ${\displaystyle {\hat{\sigma }}^2}$. This other estimator is denoted $s^2$, and is also called the sample variance, which represents a certain ambiguity in terminology; its square root ⁠${\displaystyle s}$⁠ is called the sample standard deviation. The estimator $s^2$ differs from ${\displaystyle {\hat{\sigma }}^2}$ by having (n − 1) instead of n in the denominator (the so-called Bessel's correction): $${\displaystyle s^2=\frac{n}{n-1}{\hat{\sigma }}^2=\frac{1}{n-1}\sum _{i=1}^n(x_i-\overline{x}{)}^2.}$$ The difference between $s^2$ and ${\displaystyle {\hat{\sigma }}^2}$ becomes negligibly small for large n's. In finite samples however, the motivation behind the use of $s^2$ is that it is an unbiased estimator of the underlying parameter ${\sigma }^2$, whereas ${\displaystyle {\hat{\sigma }}^2}$ is biased. Also, by the Lehmann–Scheffé theorem the estimator $s^2$ is uniformly minimum variance unbiased (UMVU), which makes it the "best" estimator among all unbiased ones. However it can be shown that the biased estimator ${\displaystyle {\hat{\sigma }}^2}$ is better than the $s^2$ in terms of the mean squared error (MSE) criterion. In finite samples both $s^2$ and ${\displaystyle {\hat{\sigma }}^2}$ have scaled chi-squared distribution with (n − 1) degrees of freedom: $${\displaystyle s^2\sim \frac{{\sigma }^2}{n-1}\cdot {\chi }_{n-1}^2,{\hat{\sigma }}^2\sim \frac{{\sigma }^2}{n}\cdot {\chi }_{n-1}^2.}$$ The first of these expressions shows that the variance of $s^2$ is equal to $2{\sigma }^4/(n-1)$, which is slightly greater than the σσ-element of the inverse Fisher information matrix ${\displaystyle {\mathcal{I}}^{-1}}$, which is $2{\sigma }^4/n$. Thus, $s^2$ is not an efficient estimator for ${\sigma }^2$, and moreover, since $s^2$ is UMVU, we can conclude that the finite-sample efficient estimator for ${\sigma }^2$ does not exist.

Applying the asymptotic theory, both estimators $s^2$ and ${\displaystyle {\hat{\sigma }}^2}$ are consistent, that is they converge in probability to ${\sigma }^2$ as the sample size $n\to \mathrm{\infty }$. The two estimators are also both asymptotically normal: $${\displaystyle \sqrt{n}({\hat{\sigma }}^2-{\sigma }^2)\simeq \sqrt{n}(s^2-{\sigma }^2)\stackrel{d}{\to }\mathcal{N}(0,2{\sigma }^4).}$$ In particular, both estimators are asymptotically efficient for ${\sigma }^2$.

Confidence intervals

By Cochran's theorem, for normal distributions the sample mean ${\displaystyle \hat{\mu }}$ and the sample variance s² are independent, which means there can be no gain in considering their joint distribution. There is also a converse theorem: if in a sample the sample mean and sample variance are independent, then the sample must have come from the normal distribution. The independence between ${\displaystyle \hat{\mu }}$ and s can be employed to construct the so-called t-statistic: $${\displaystyle t=\frac{\hat{\mu }-\mu }{s/\sqrt{n}}=\frac{\overline{x}-\mu }{\sqrt{\frac{1}{n(n-1)}\sum (x_i-\overline{x}{)}^2}}\sim t_{n-1}}$$ This quantity t has the Student's t-distribution with (n − 1) degrees of freedom, and it is an ancillary statistic (independent of the value of the parameters). Inverting the distribution of this t-statistics will allow us to construct the confidence interval for μ; similarly, inverting the χ² distribution of the statistic s² will give us the confidence interval for σ²: $${\displaystyle \mu \in \left[\hat{\mu }-t_{n-1,1-\alpha /2}\frac{s}{\sqrt{n}},\hat{\mu }+t_{n-1,1-\alpha /2}\frac{s}{\sqrt{n}}\right]}$$ $${\displaystyle {\sigma }^2\in \left[\frac{n-1}{{\chi }_{n-1,1-\alpha /2}^2}{s}^2,\frac{n-1}{{\chi }_{n-1,\alpha /2}^2}{s}^2\right]}$$ where t_k,p and χ 2
k,p are the pth quantiles of the t- and χ²-distributions respectively. These confidence intervals are of the confidence level 1 − α, meaning that the true values μ and σ² fall outside of these intervals with probability (or significance level) α. In practice people usually take α = 5%, resulting in the 95% confidence intervals. The confidence interval for σ can be found by taking the square root of the interval bounds for σ².

Approximate formulas can be derived from the asymptotic distributions of ${\displaystyle \hat{\mu }}$ and s²: $${\displaystyle \mu \in \left[\hat{\mu }-\frac{|z_{\alpha /2}|}{\sqrt{n}}s,\hat{\mu }+\frac{|z_{\alpha /2}|}{\sqrt{n}}s\right]}$$ $${\displaystyle {\sigma }^2\in \left[s^2-\sqrt{2}\frac{|z_{\alpha /2}|}{\sqrt{n}}{s}^2,s^2+\sqrt{2}\frac{|z_{\alpha /2}|}{\sqrt{n}}{s}^2\right]}$$ The approximate formulas become valid for large values of n, and are more convenient for the manual calculation since the standard normal quantiles z_α/2 do not depend on n. In particular, the most popular value of α = 5%, results in |z_0.025| = 1.96.