Propagation of uncertainty

Non-linear combinations

When f is a set of non-linear combination of the variables x, an interval propagation could be performed in order to compute intervals which contain all consistent values for the variables. In a probabilistic approach, the function f must usually be linearised by approximation to a first-order Taylor series expansion, though in some cases, exact formulae can be derived that do not depend on the expansion as is the case for the exact variance of products. The Taylor expansion would be: $${\displaystyle f_k\approx f_k^0+\sum _i^n\frac{\mathrm{\partial }{f}_k}{\mathrm{\partial }{x}_i}{x}_i}$$ where ${\displaystyle \mathrm{\partial }{f}_k/\mathrm{\partial }{x}_i}$ denotes the partial derivative of f_k with respect to the i-th variable, evaluated at the mean value of all components of vector x. Or in matrix notation, $${\displaystyle \mathrm{f}\approx {\mathrm{f}}^0+\mathrm{J}\mathrm{x}}$$ where J is the Jacobian matrix. Since f⁰ is a constant it does not contribute to the error on f. Therefore, the propagation of error follows the linear case, above, but replacing the linear coefficients, A_ki and A_kj by the partial derivatives, ${\displaystyle \frac{\mathrm{\partial }{f}_k}{\mathrm{\partial }{x}_i}}$ and ${\displaystyle \frac{\mathrm{\partial }{f}_k}{\mathrm{\partial }{x}_j}}$. In matrix notation, $${\displaystyle {\mathrm{\Sigma }}^{\mathrm{f}}=\mathrm{J}{\mathrm{\Sigma }}^{\mathrm{x}}{\mathrm{J}}^{\mathrm{\top }}.}$$

That is, the Jacobian of the function is used to transform the rows and columns of the variance-covariance matrix of the argument. Note this is equivalent to the matrix expression for the linear case with ${\displaystyle \mathrm{J}=\mathrm{A}}$.

Simplification

Neglecting correlations or assuming independent variables yields a common formula among engineers and experimental scientists to calculate error propagation, the variance formula: $${\displaystyle s_f=\sqrt{{\left(\frac{\mathrm{\partial }f}{\mathrm{\partial }x}\right)}^2{s}_x^2+{\left(\frac{\mathrm{\partial }f}{\mathrm{\partial }y}\right)}^2{s}_y^2+{\left(\frac{\mathrm{\partial }f}{\mathrm{\partial }z}\right)}^2{s}_z^2+\cdots }}$$ where ${\displaystyle s_f}$ represents the standard deviation of the function ${\displaystyle f}$, ${\displaystyle s_x}$ represents the standard deviation of ${\displaystyle x}$, ${\displaystyle s_y}$ represents the standard deviation of ${\displaystyle y}$, and so forth.

This formula is based on the linear characteristics of the gradient of ${\displaystyle f}$ and therefore it is a good estimation for the standard deviation of ${\displaystyle f}$ as long as ${\displaystyle s_x,s_y,s_z,\dots }$ are small enough. Specifically, the linear approximation of ${\displaystyle f}$ has to be close to ${\displaystyle f}$ inside a neighbourhood of radius ${\displaystyle s_x,s_y,s_z,\dots }$.

Example

Any non-linear differentiable function, ${\displaystyle f(a,b)}$, of two variables, ${\displaystyle a}$ and ${\displaystyle b}$, can be expanded as $${\displaystyle f\approx f^0+\frac{\mathrm{\partial }f}{\mathrm{\partial }a}a+\frac{\mathrm{\partial }f}{\mathrm{\partial }b}b.}$$ If we take the variance on both sides and use the formula for the variance of a linear combination of variables $${\displaystyle \mathrm{Var}(aX+bY)=a^2\mathrm{Var}(X)+b^2\mathrm{Var}(Y)+2ab\mathrm{Cov}(X,Y),}$$ then we obtain $${\displaystyle {\sigma }_f^2\approx {\left|\frac{\mathrm{\partial }f}{\mathrm{\partial }a}\right|}^2{\sigma }_a^2+{\left|\frac{\mathrm{\partial }f}{\mathrm{\partial }b}\right|}^2{\sigma }_b^2+2\frac{\mathrm{\partial }f}{\mathrm{\partial }a}\frac{\mathrm{\partial }f}{\mathrm{\partial }b}{\sigma }_{ab},}$$ where ${\displaystyle {\sigma }_f}$ is the standard deviation of the function ${\displaystyle f}$, ${\displaystyle {\sigma }_a}$ is the standard deviation of ${\displaystyle a}$, ${\displaystyle {\sigma }_b}$ is the standard deviation of ${\displaystyle b}$ and ${\displaystyle {\sigma }_{ab}={\sigma }_a{\sigma }_b{\rho }_{ab}}$ is the covariance between ${\displaystyle a}$ and ${\displaystyle b}$.

In the particular case that ${\displaystyle f=ab}$, ${\displaystyle \frac{\mathrm{\partial }f}{\mathrm{\partial }a}=b}$, ${\displaystyle \frac{\mathrm{\partial }f}{\mathrm{\partial }b}=a}$. Then $${\displaystyle {\sigma }_f^2\approx b^2{\sigma }_a^2+a^2{\sigma }_b^2+2ab{\sigma }_{ab}}$$ or $${\displaystyle {\left(\frac{{\sigma }_f}{f}\right)}^2\approx {\left(\frac{{\sigma }_a}{a}\right)}^2+{\left(\frac{{\sigma }_b}{b}\right)}^2+2\left(\frac{{\sigma }_a}{a}\right)\left(\frac{{\sigma }_b}{b}\right){\rho }_{ab}}$$ where ${\displaystyle {\rho }_{ab}}$ is the correlation between ${\displaystyle a}$ and ${\displaystyle b}$.

When the variables ${\displaystyle a}$ and ${\displaystyle b}$ are uncorrelated, ${\displaystyle {\rho }_{ab}=0}$. Then $${\displaystyle {\left(\frac{{\sigma }_f}{f}\right)}^2\approx {\left(\frac{{\sigma }_a}{a}\right)}^2+{\left(\frac{{\sigma }_b}{b}\right)}^2.}$$

Caveats and warnings

Error estimates for non-linear functions are biased on account of using a truncated series expansion. The extent of this bias depends on the nature of the function. For example, the bias on the error calculated for log(1+x) increases as x increases, since the expansion to x is a good approximation only when x is near zero.

A second-order Taylor expansion produces more accurate results, specially for highly non-linear functions and large variance values. For highly non-linear functions, there exist five categories of probabilistic approaches for uncertainty propagation; see Uncertainty quantification for details.

Reciprocal and shifted reciprocal

In the special case of the inverse or reciprocal ${\displaystyle 1/B}$, where ${\displaystyle B=N(0,1)}$ follows a standard normal distribution, the resulting distribution is a reciprocal standard normal distribution, and there is no definable variance.

However, in the slightly more general case of a shifted reciprocal function ${\displaystyle 1/(p-B)}$ for ${\displaystyle B=N(\mu ,\sigma )}$ following a general normal distribution, then mean and variance statistics do exist in a principal value sense, if the difference between the pole ${\displaystyle p}$ and the mean ${\displaystyle \mu }$ is real-valued.

Ratios

Ratios are also problematic; normal approximations exist under certain conditions.

Example formulae

This table shows the variances and standard deviations of simple functions of the real variables ${\displaystyle A,B}$ with standard deviations ${\displaystyle {\sigma }_A,{\sigma }_B,}$ covariance ${\displaystyle {\sigma }_{AB}={\rho }_{AB}{\sigma }_A{\sigma }_B,}$ and correlation ${\displaystyle {\rho }_{AB}.}$ The real-valued coefficients ${\displaystyle a}$ and ${\displaystyle b}$ are assumed exactly known (deterministic), i.e., ${\displaystyle {\sigma }_a={\sigma }_b=0.}$

In the right-hand columns of the table, ${\displaystyle A}$ and ${\displaystyle B}$ are expectation values, and ${\displaystyle f}$ is the value of the function calculated at those values.

For uncorrelated variables (${\displaystyle {\rho }_{AB}=0}$, ${\displaystyle {\sigma }_{AB}=0}$) expressions for more complicated functions can be derived by combining simpler functions. For example, repeated multiplication, assuming no correlation, gives $${\displaystyle f=ABC;{\left(\frac{{\sigma }_f}{f}\right)}^2\approx {\left(\frac{{\sigma }_A}{A}\right)}^2+{\left(\frac{{\sigma }_B}{B}\right)}^2+{\left(\frac{{\sigma }_C}{C}\right)}^2.}$$

For the case ${\displaystyle f=AB}$ we also have Goodman's expression for the exact variance: for the uncorrelated case it is $${\displaystyle \mathrm{V}[XY]=\mathrm{E}[X{]}^2\mathrm{V}[Y]+\mathrm{E}[Y{]}^2\mathrm{V}[X]+\mathrm{V}[X]\mathrm{V}[Y],}$$ and therefore we have $${\displaystyle {\sigma }_f^2=A^2{\sigma }_B^2+B^2{\sigma }_A^2+{\sigma }_A^2{\sigma }_B^2.}$$ The last term represents a small correction to the usual formula as can be seen by dividing both sides by ${\displaystyle f^2=A^2{B}^2}$. $${\displaystyle {\left(\frac{{\sigma }_f}{f}\right)}^2={\left(\frac{{\sigma }_A}{A}\right)}^2+{\left(\frac{{\sigma }_B}{B}\right)}^2+{\left(\frac{{\sigma }_A{\sigma }_B}{AB}\right)}^2.}$$

Effect of correlation on differences

If A and B are uncorrelated, their difference A − B will have more variance than either of them. An increasing positive correlation (${\displaystyle {\rho }_{AB}\to 1}$) will decrease the variance of the difference, converging to zero variance for perfectly correlated variables with the same variance. On the other hand, a negative correlation (${\displaystyle {\rho }_{AB}\to -1}$) will further increase the variance of the difference, compared to the uncorrelated case.

For example, the self-subtraction f = A − A has zero variance ${\displaystyle {\sigma }_f^2=0}$ only if the variate is perfectly autocorrelated (${\displaystyle {\rho }_A=1}$). If A is uncorrelated, ${\displaystyle {\rho }_A=0,}$ then the output variance is twice the input variance, ${\displaystyle {\sigma }_f^2=2{\sigma }_A^2.}$ And if A is perfectly anticorrelated, ${\displaystyle {\rho }_A=-1,}$ then the input variance is quadrupled in the output, ${\displaystyle {\sigma }_f^2=4{\sigma }_A^2}$ (notice ${\displaystyle 1-{\rho }_A=2}$ for f = aA − aA in the table above).

Example calculations

Inverse tangent function

We can calculate the uncertainty propagation for the inverse tangent function as an example of using partial derivatives to propagate error.

Define $${\displaystyle f(x)=\arctan (x),}$$ where ${\displaystyle {\mathrm{\Delta }}_x}$ is the absolute uncertainty on our measurement of x. The derivative of f(x) with respect to x is $${\displaystyle \frac{df}{dx}=\frac{1}{1+x^2}.}$$

Therefore, our propagated uncertainty is $${\displaystyle {\mathrm{\Delta }}_f\approx \frac{{\mathrm{\Delta }}_x}{1+x^2},}$$ where ${\displaystyle {\mathrm{\Delta }}_f}$ is the absolute propagated uncertainty.

Resistance measurement

A practical application is an experiment in which one measures current, I, and voltage, V, on a resistor in order to determine the resistance, R, using Ohm's law, R = V / I.

Given the measured variables with uncertainties, I ± σ_I and V ± σ_V, and neglecting their possible correlation, the uncertainty in the computed quantity, σ_R, is:

$${\displaystyle {\sigma }_R\approx \sqrt{{\sigma }_V^2{\left(\frac{1}{I}\right)}^2+{\sigma }_I^2{\left(\frac{-V}{I^2}\right)}^2}=R\sqrt{{\left(\frac{{\sigma }_V}{V}\right)}^2+{\left(\frac{{\sigma }_I}{I}\right)}^2}.}$$