Generalized method of moments

Properties

Consistency

Consistency is a statistical property of an estimator stating that, having a sufficient number of observations, the estimator will converge in probability to the true value of parameter:

${\displaystyle \hat{\theta }\stackrel{p}{\to }{\theta }_0\text{as}T\to \mathrm{\infty }.}$

Sufficient conditions for a GMM estimator to be consistent are as follows:

1. ${\displaystyle {\hat{W}}_T\stackrel{p}{\to }W,}$ where W is a positive semi-definite matrix,
2. ${\displaystyle W\mathrm{E}[g(Y_t,\theta )]=0}$ only for ${\displaystyle \theta ={\theta }_0,}$
3. The space of possible parameters ${\displaystyle \mathrm{\Theta }\subset {\mathbb{R}}^k}$ is compact,
4. ${\displaystyle g(Y,\theta )}$ is continuous at each θ with probability one,
5. ${\displaystyle \mathrm{E}[\underset{\theta \in \mathrm{\Theta }}{sup}\Vert g(Y,\theta )\Vert ]<\mathrm{\infty }.}$

The second condition here (so-called Global identification condition) is often particularly hard to verify. There exist simpler necessary but not sufficient conditions, which may be used to detect non-identification problem:

• Order condition. The dimension of moment function m(θ) should be at least as large as the dimension of parameter vector θ.
• Local identification. If g(Y,θ) is continuously differentiable in a neighborhood of ${\displaystyle {\theta }_0}$, then matrix ${\displaystyle W\mathrm{E}[{\mathrm{\nabla }}_{\theta }g(Y_t,{\theta }_0)]}$ must have full column rank.

In practice applied econometricians often simply assume that global identification holds, without actually proving it.

Asymptotic normality

Asymptotic normality is a useful property, as it allows us to construct confidence bands for the estimator, and conduct different tests. Before we can make a statement about the asymptotic distribution of the GMM estimator, we need to define two auxiliary matrices:

${\displaystyle G=\mathrm{E}[{\mathrm{\nabla }}_{\theta }g(Y_t,{\theta }_0)],\mathrm{\Omega }=\mathrm{E}[g(Y_t,{\theta }_0)g(Y_t,{\theta }_0{)}^{\mathsf{T}}]}$

Then under conditions 1–6 listed below, the GMM estimator will be asymptotically normal with limiting distribution:

${\displaystyle \sqrt{T}(\hat{\theta }-{\theta }_0)\stackrel{d}{\to }\mathcal{N}[0,(G^{\mathsf{T}}WG{)}^{-1}{G}^{\mathsf{T}}W\mathrm{\Omega }{W}^{\mathsf{T}}G(G^{\mathsf{T}}{W}^{\mathsf{T}}G{)}^{-1}].}$

Conditions:

1. ${\displaystyle \hat{\theta }}$ is consistent (see previous section),
2. The set of possible parameters ${\displaystyle \mathrm{\Theta }\subset {\mathbb{R}}^k}$ is compact,
3. ${\displaystyle g(Y,\theta )}$ is continuously differentiable in some neighborhood N of ${\displaystyle {\theta }_0}$ with probability one,
4. ${\displaystyle \mathrm{E}[\Vert g(Y_t,\theta ){\Vert }^2]<\mathrm{\infty },}$
5. ${\displaystyle \mathrm{E}[\underset{\theta \in N}{sup}\Vert {\mathrm{\nabla }}_{\theta }g(Y_t,\theta )\Vert ]<\mathrm{\infty },}$
6. the matrix ${\displaystyle G^{\prime }WG}$ is nonsingular.

Relative efficiency

So far we have said nothing about the choice of matrix W, except that it must be positive semi-definite. In fact any such matrix will produce a consistent and asymptotically normal GMM estimator, the only difference will be in the asymptotic variance of that estimator. It can be shown that taking

${\displaystyle W\propto {\mathrm{\Omega }}^{-1}}$

will result in the most efficient estimator in the class of all (generalized) method of moment estimators. Only infinite number of orthogonal conditions obtains the smallest variance, the Cramér–Rao bound.

In this case the formula for the asymptotic distribution of the GMM estimator simplifies to

${\displaystyle \sqrt{T}(\hat{\theta }-{\theta }_0)\stackrel{d}{\to }\mathcal{N}[0,(G^{\mathsf{T}}{\mathrm{\Omega }}^{-1}G{)}^{-1}]}$

The proof that such a choice of weighting matrix is indeed locally optimal is often adopted with slight modifications when establishing efficiency of other estimators. As a rule of thumb, a weighting matrix inches closer to optimality when it turns into an expression closer to the Cramér–Rao bound.

Implementation

One difficulty with implementing the outlined method is that we cannot take W = Ω⁻¹ because, by the definition of matrix Ω, we need to know the value of θ₀ in order to compute this matrix, and θ₀ is precisely the quantity we do not know and are trying to estimate in the first place. In the case of Y_t being iid we can estimate W as

${\displaystyle {\hat{W}}_T(\hat{\theta })=(\frac{1}{T}\sum _{t=1}^{T}g(Y_t,\hat{\theta })g(Y_t,\hat{\theta }{)}^{\mathsf{T}}{)}^{-1}.}$

Several approaches exist to deal with this issue, the first one being the most popular:

• Two-step feasible GMM:
  • Step 1: Take W = I (the identity matrix) or some other positive-definite matrix, and compute preliminary GMM estimate ${\displaystyle {\hat{\theta }}_{(1)}}$. This estimator is consistent for θ₀, although not efficient.
  • Step 2: ${\displaystyle {\hat{W}}_T({\hat{\theta }}_{(1)})}$ converges in probability to Ω⁻¹ and therefore if we compute ${\displaystyle \hat{\theta }}$ with this weighting matrix, the estimator will be asymptotically efficient.
• Iterated GMM. Essentially the same procedure as 2-step GMM, except that the matrix ${\displaystyle {\hat{W}}_T}$ is recalculated several times. That is, the estimate obtained in step 2 is used to calculate the weighting matrix for step 3, and so on until some convergence criterion is met.

  ${\displaystyle {\hat{\theta }}_{(i+1)}=\arg \underset{\theta \in \mathrm{\Theta }}{min}(\frac{1}{T}\sum _{t=1}^{T}g(Y_t,\theta ){)}^{\mathsf{T}}{\hat{W}}_T({\hat{\theta }}_{(i)})(\frac{1}{T}\sum _{t=1}^{T}g(Y_t,\theta ))}$

  Asymptotically no improvement can be achieved through such iterations, although certain Monte-Carlo experiments suggest that finite-sample properties of this estimator are slightly better.
• Continuously updating GMM (CUGMM, or CUE). Estimates ${\displaystyle \hat{\theta }}$ simultaneously with estimating the weighting matrix W:

  ${\displaystyle \hat{\theta }=\arg \underset{\theta \in \mathrm{\Theta }}{min}(\frac{1}{T}\sum _{t=1}^{T}g(Y_t,\theta ){)}^{\mathsf{T}}{\hat{W}}_T(\theta )(\frac{1}{T}\sum _{t=1}^{T}g(Y_t,\theta ))}$

  In Monte-Carlo experiments this method demonstrated a better performance than the traditional two-step GMM: the estimator has smaller median bias (although fatter tails), and the J-test for overidentifying restrictions in many cases was more reliable.

Another important issue in implementation of minimization procedure is that the function is supposed to search through (possibly high-dimensional) parameter space Θ and find the value of θ which minimizes the objective function. No generic recommendation for such procedure exists, it is a subject of its own field, numerical optimization.

Sargan–Hansen J-test

When the number of moment conditions is greater than the dimension of the parameter vector θ, the model is said to be over-identified. Sargan (1958) proposed tests for over-identifying restrictions based on instrumental variables estimators that are distributed in large samples as Chi-square variables with degrees of freedom that depend on the number of over-identifying restrictions. Subsequently, Hansen (1982) applied this test to the mathematically equivalent formulation of GMM estimators. Note, however, that such statistics can be negative in empirical applications where the models are misspecified, and likelihood ratio tests can yield insights since the models are estimated under both null and alternative hypotheses (Bhargava and Sargan, 1983).

Conceptually we can check whether ${\displaystyle \hat{m}(\hat{\theta })}$ is sufficiently close to zero to suggest that the model fits the data well. The GMM method has then replaced the problem of solving the equation ${\displaystyle \hat{m}(\theta )=0}$, which chooses ${\displaystyle \theta }$ to match the restrictions exactly, by a minimization calculation. The minimization can always be conducted even when no ${\displaystyle {\theta }_0}$ exists such that ${\displaystyle m({\theta }_0)=0}$. This is what J-test does. The J-test is also called a test for over-identifying restrictions.

Formally we consider two hypotheses:

• ${\displaystyle H_0:m({\theta }_0)=0}$ (the null hypothesis that the model is “valid”), and
• ${\displaystyle H_1:m(\theta )\ne 0,\mathrm{\forall }\theta \in \mathrm{\Theta }}$ (the alternative hypothesis that model is “invalid”; the data does not come close to meeting the restrictions)

Under hypothesis ${\displaystyle H_0}$, the following so-called J-statistic is asymptotically chi-squared distributed with k–l degrees of freedom. Define J to be:

${\displaystyle J\equiv T\cdot (\frac{1}{T}\sum _{t=1}^{T}g(Y_t,\hat{\theta }){)}^{\mathsf{T}}{\hat{W}}_T(\frac{1}{T}\sum _{t=1}^{T}g(Y_t,\hat{\theta }))\stackrel{d}{\to }{\chi }_{k-\ell }^2}$ under ${\displaystyle H_0,}$

where ${\displaystyle \hat{\theta }}$ is the GMM estimator of the parameter ${\displaystyle {\theta }_0}$, k is the number of moment conditions (dimension of vector g), and l is the number of estimated parameters (dimension of vector θ). Matrix ${\displaystyle {\hat{W}}_T}$ must converge in probability to ${\displaystyle {\mathrm{\Omega }}^{-1}}$, the efficient weighting matrix (note that previously we only required that W be proportional to ${\displaystyle {\mathrm{\Omega }}^{-1}}$ for estimator to be efficient; however in order to conduct the J-test W must be exactly equal to ${\displaystyle {\mathrm{\Omega }}^{-1}}$, not simply proportional).

Under the alternative hypothesis ${\displaystyle H_1}$, the J-statistic is asymptotically unbounded:

${\displaystyle J\stackrel{p}{\to }\mathrm{\infty }}$ under ${\displaystyle H_1}$

To conduct the test we compute the value of J from the data. It is a nonnegative number. We compare it with (for example) the 0.95 quantile of the ${\displaystyle {\chi }_{k-\ell }^2}$ distribution:

• ${\displaystyle H_0}$ is rejected at 95% confidence level if ${\displaystyle J>q_{0.95}^{{\chi }_{k-\ell }^2}}$
• ${\displaystyle H_0}$ cannot be rejected at 95% confidence level if ${\displaystyle J<q_{0.95}^{{\chi }_{k-\ell }^2}}$

Scope

Many other popular estimation techniques can be cast in terms of GMM optimization:

• Ordinary least squares (OLS) is equivalent to GMM with moment conditions:

  ${\displaystyle \mathrm{E}[x_t(y_t-x_t^{\mathsf{T}}\beta )]=0}$

• Weighted least squares (WLS)

  ${\displaystyle \mathrm{E}[x_t(y_t-x_t^{\mathsf{T}}\beta )/{\sigma }^2(x_t)]=0}$

• Instrumental variables regression (IV)

  ${\displaystyle \mathrm{E}[z_t(y_t-x_t^{\mathsf{T}}\beta )]=0}$

• Non-linear least squares (NLLS):

  ${\displaystyle \mathrm{E}[{\mathrm{\nabla }}_{\beta }g(x_t,\beta )\cdot (y_t-g(x_t,\beta ))]=0}$

• Maximum likelihood estimation (MLE):

  ${\displaystyle \mathrm{E}[{\mathrm{\nabla }}_{\theta }\ln f(x_t,\theta )]=0}$

Alternative to the GMM

In method of moments, an alternative to the original (non-generalized) Method of Moments (MoM) is described, and references to some applications and a list of theoretical advantages and disadvantages relative to the traditional method are provided. This Bayesian-Like MoM (BL-MoM) is distinct from all the related methods described above, which are subsumed by the GMM. The literature does not contain a direct comparison between the GMM and the BL-MoM in specific applications.

Implementations

• R Programming wikibook, Method of Moments
• R
• Stata
• EViews
• SAS
• Gretl