Scaled inverse chi-squared distribution

Parameter estimation

The maximum likelihood estimate of ${\displaystyle {\tau }^2}$ is

${\displaystyle {\tau }^2=n/\sum _{i=1}^n\frac{1}{x_i}.}$

The maximum likelihood estimate of ${\displaystyle \frac{\nu }{2}}$ can be found using Newton's method on:

${\displaystyle \ln \left(\frac{\nu }{2}\right)-\psi \left(\frac{\nu }{2}\right)=\frac{1}{n}\sum _{i=1}^n\ln \left(x_i\right)-\ln \left({\tau }^2\right),}$

where ${\displaystyle \psi (x)}$ is the digamma function. An initial estimate can be found by taking the formula for mean and solving it for ${\displaystyle \nu .}$ Let ${\displaystyle \overline{x}=\frac{1}{n}\sum _{i=1}^n{x}_i}$ be the sample mean. Then an initial estimate for ${\displaystyle \nu }$ is given by:

${\displaystyle \frac{\nu }{2}=\frac{\overline{x}}{\overline{x}-{\tau }^2}.}$

Bayesian estimation of the variance of a normal distribution

The scaled inverse chi-squared distribution has a second important application, in the Bayesian estimation of the variance of a Normal distribution.

According to Bayes' theorem, the posterior probability distribution for quantities of interest is proportional to the product of a prior distribution for the quantities and a likelihood function:

${\displaystyle p({\sigma }^2|D,I)\propto p({\sigma }^2|I)p(D|{\sigma }^2)}$

where D represents the data and I represents any initial information about σ² that we may already have.

The simplest scenario arises if the mean μ is already known; or, alternatively, if it is the conditional distribution of σ² that is sought, for a particular assumed value of μ.

Then the likelihood term L(σ²|D) = p(D|σ²) has the familiar form

${\displaystyle \mathcal{L}({\sigma }^2|D,\mu )=\frac{1}{{\left(\sqrt{2\pi }\sigma \right)}^n}\exp \left[-\frac{\sum _i^n(x_i-\mu {)}^2}{2{\sigma }^2}\right]}$

Combining this with the rescaling-invariant prior p(σ²|I) = 1/σ², which can be argued (e.g. following Jeffreys) to be the least informative possible prior for σ² in this problem, gives a combined posterior probability

${\displaystyle p({\sigma }^2|D,I,\mu )\propto \frac{1}{{\sigma }^{n+2}}\exp \left[-\frac{\sum _i^n(x_i-\mu {)}^2}{2{\sigma }^2}\right]}$

This form can be recognised as that of a scaled inverse chi-squared distribution, with parameters ν = n and τ² = s² = (1/n) Σ (x_i-μ)²

Gelman and co-authors remark that the re-appearance of this distribution, previously seen in a sampling context, may seem remarkable; but given the choice of prior "this result is not surprising."

In particular, the choice of a rescaling-invariant prior for σ² has the result that the probability for the ratio of σ² / s² has the same form (independent of the conditioning variable) when conditioned on s² as when conditioned on σ²:

${\displaystyle p(\frac{{\sigma }^2}{s^2}|s^2)=p(\frac{{\sigma }^2}{s^2}|{\sigma }^2)}$

In the sampling-theory case, conditioned on σ², the probability distribution for (1/s²) is a scaled inverse chi-squared distribution; and so the probability distribution for σ² conditioned on s², given a scale-agnostic prior, is also a scaled inverse chi-squared distribution.

Use as an informative prior

If more is known about the possible values of σ², a distribution from the scaled inverse chi-squared family, such as Scale-inv-χ²(n₀, s₀²) can be a convenient form to represent a more informative prior for σ², as if from the result of n₀ previous observations (though n₀ need not necessarily be a whole number):

${\displaystyle p({\sigma }^2|I^{\mathrm{\prime }},\mu )\propto \frac{1}{{\sigma }^{n_0+2}}\exp \left[-\frac{n_0{s}_0^2}{2{\sigma }^2}\right]}$

Such a prior would lead to the posterior distribution

${\displaystyle p({\sigma }^2|D,I^{\mathrm{\prime }},\mu )\propto \frac{1}{{\sigma }^{n+n_0+2}}\exp \left[-\frac{n{s}^2+n_0{s}_0^2}{2{\sigma }^2}\right]}$

which is itself a scaled inverse chi-squared distribution. The scaled inverse chi-squared distributions are thus a convenient conjugate prior family for σ² estimation.

Estimation of variance when mean is unknown

If the mean is not known, the most uninformative prior that can be taken for it is arguably the translation-invariant prior p(μ|I) ∝ const., which gives the following joint posterior distribution for μ and σ²,

${\displaystyle \begin{array}{cc}p(\mu ,{\sigma }^2\mid D,I)& \propto \frac{1}{{\sigma }^{n+2}}\exp \left[-\frac{\sum _i^n(x_i-\mu {)}^2}{2{\sigma }^2}\right]\\ & =\frac{1}{{\sigma }^{n+2}}\exp \left[-\frac{\sum _i^n(x_i-\overline{x}{)}^2}{2{\sigma }^2}\right]\exp \left[-\frac{n(\mu -\overline{x}{)}^2}{2{\sigma }^2}\right]\end{array}}$

The marginal posterior distribution for σ² is obtained from the joint posterior distribution by integrating out over μ,

${\displaystyle \begin{array}{cc}p({\sigma }^2|D,I)\propto & \frac{1}{{\sigma }^{n+2}}\exp \left[-\frac{\sum _i^n(x_i-\overline{x}{)}^2}{2{\sigma }^2}\right]{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\exp \left[-\frac{n(\mu -\overline{x}{)}^2}{2{\sigma }^2}\right]d\mu \\ =& \frac{1}{{\sigma }^{n+2}}\exp \left[-\frac{\sum _i^n(x_i-\overline{x}{)}^2}{2{\sigma }^2}\right]\sqrt{2\pi {\sigma }^2/n}\\ \propto & ({\sigma }^2{)}^{-(n+1)/2}\exp \left[-\frac{(n-1)s^2}{2{\sigma }^2}\right]\end{array}}$

This is again a scaled inverse chi-squared distribution, with parameters ${\displaystyle n-1}$ and ${\displaystyle s^2=\sum (x_i-\overline{x}{)}^2/(n-1)}$.

Related distributions

• If ${\displaystyle X\sim \text{Scale-inv-}{\chi }^2(\nu ,{\tau }^2)}$ then ${\displaystyle kX\sim \text{Scale-inv-}{\chi }^2(\nu ,k{\tau }^2)}$
• If ${\displaystyle X\sim \text{inv-}{\chi }^2(\nu )}$ (Inverse-chi-squared distribution) then ${\displaystyle X\sim \text{Scale-inv-}{\chi }^2(\nu ,1/\nu )}$
• If ${\displaystyle X\sim \text{Scale-inv-}{\chi }^2(\nu ,{\tau }^2)}$ then ${\displaystyle \frac{X}{{\tau }^2\nu }\sim \text{inv-}{\chi }^2(\nu )}$ (Inverse-chi-squared distribution)
• If ${\displaystyle X\sim \text{Scale-inv-}{\chi }^2(\nu ,{\tau }^2)}$ then ${\displaystyle X\sim \text{Inv-Gamma}\left(\frac{\nu }{2},\frac{\nu {\tau }^2}{2}\right)}$ (Inverse-gamma distribution)
• Scaled inverse chi square distribution is a special case of type 5 Pearson distribution