Omitted-variable bias

More specifically, OVB is the bias that appears in the estimates of parameters in a regression analysis, when the assumed specification is incorrect in that it omits an independent variable that is a determinant of the dependent variable and correlated with one or more of the included independent variables.

In linear regression

Intuition

Suppose the true cause-and-effect relationship is given by:

${\displaystyle y=a+bx+cz+u}$

with parameters a, b, c, dependent variable y, independent variables x and z, and error term u. We wish to know the effect of x itself upon y (that is, we wish to obtain an estimate of b).

Two conditions must hold true for omitted-variable bias to exist in linear regression:

• the omitted variable must be a determinant of the dependent variable (i.e., its true regression coefficient must not be zero); and
• the omitted variable must be correlated with an independent variable specified in the regression (i.e., cov(z,x) must not equal zero).

Suppose we omit z from the regression, and suppose the relation between x and z is given by

${\displaystyle z=d+fx+e}$

with parameters d, f and error term e. Substituting the second equation into the first gives

${\displaystyle y=(a+cd)+(b+cf)x+(u+ce).}$

If a regression of y is conducted upon x only, this last equation is what is estimated, and the regression coefficient on x is actually an estimate of (b + cf ), giving not simply an estimate of the desired direct effect of x upon y (which is b), but rather of its sum with the indirect effect (the effect f of x on z times the effect c of z on y). Thus by omitting the variable z from the regression, we have estimated the total derivative of y with respect to x rather than its partial derivative with respect to x. These differ if both c and f are non-zero.

The direction and extent of the bias are both contained in cf, since the effect sought is b but the regression estimates b+cf. The extent of the bias is the absolute value of cf, and the direction of bias is upward (toward a more positive or less negative value) if cf > 0 (if the direction of correlation between y and z is the same as that between x and z), and it is downward otherwise.

Detailed analysis

As an example, consider a linear model of the form

${\displaystyle y_i=x_i\beta +z_i\delta +u_i,i=1,\dots ,n}$

where

• x_i is a 1 × p row vector of values of p independent variables observed at time i or for the i ^th study participant;
• β is a p × 1 column vector of unobservable parameters (the response coefficients of the dependent variable to each of the p independent variables in x_i) to be estimated;
• z_i is a scalar and is the value of another independent variable that is observed at time i or for the i ^th study participant;
• δ is a scalar and is an unobservable parameter (the response coefficient of the dependent variable to z_i) to be estimated;
• u_i is the unobservable error term occurring at time i or for the i ^th study participant; it is an unobserved realization of a random variable having expected value 0 (conditionally on x_i and z_i);
• y_i is the observation of the dependent variable at time i or for the i ^th study participant.

We collect the observations of all variables subscripted i = 1, ..., n, and stack them one below another, to obtain the matrix X and the vectors Y, Z, and U:

${\displaystyle X=\left[\begin{array}{c}{x}_1\\ ⋮\\ x_n\end{array}\right]\in {\mathbb{R}}^{n\times p},}$

and

${\displaystyle Y=\left[\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right],Z=\left[\begin{array}{c}{z}_1\\ ⋮\\ z_n\end{array}\right],U=\left[\begin{array}{c}{u}_1\\ ⋮\\ u_n\end{array}\right]\in {\mathbb{R}}^{n\times 1}.}$

If the independent variable z is omitted from the regression, then the estimated values of the response parameters of the other independent variables will be given by the usual least squares calculation,

${\displaystyle \hat{\beta }=(X^{\prime }X{)}^{-1}{X}^{\prime }Y}$

(where the "prime" notation means the transpose of a matrix and the -1 superscript is matrix inversion).

Substituting for Y based on the assumed linear model,

${\displaystyle \begin{array}{cc}\hat{\beta }& =(X^{\prime }X{)}^{-1}{X}^{\prime }(X\beta +Z\delta +U)\\ & =(X^{\prime }X{)}^{-1}{X}^{\prime }X\beta +(X^{\prime }X{)}^{-1}{X}^{\prime }Z\delta +(X^{\prime }X{)}^{-1}{X}^{\prime }U\\ & =\beta +(X^{\prime }X{)}^{-1}{X}^{\prime }Z\delta +(X^{\prime }X{)}^{-1}{X}^{\prime }U.\end{array}}$

On taking expectations, the contribution of the final term is zero; this follows from the assumption that U is uncorrelated with the regressors X. On simplifying the remaining terms:

${\displaystyle \begin{array}{cc}E[\hat{\beta }\mid X]& =\beta +(X^{\prime }X{)}^{-1}E[X^{\prime }Z\mid X]\delta \\ & =\beta +\text{bias}.\end{array}}$

The second term after the equal sign is the omitted-variable bias in this case, which is non-zero if the omitted variable z is correlated with any of the included variables in the matrix X (that is, if X′Z does not equal a vector of zeroes). Note that the bias is equal to the weighted portion of z_i which is "explained" by x_i.