Extremum estimator

Consistency

If the parameter space Θ is compact and there is a limiting function Q₀(θ) such that: ${\displaystyle {\hat{Q}}_n(\theta )}$ converges to Q₀(θ) in probability uniformly over Θ, and the function Q₀(θ) is continuous and has a unique maximum at θ = θ₀ then ${\displaystyle \hat{\theta }}$ is consistent for θ₀.

The uniform convergence in probability of ${\displaystyle {\hat{Q}}_n(\theta )}$ means that

${\displaystyle \underset{\theta \in \mathrm{\Theta }}{sup}|{\hat{Q}}_n(\theta )-Q_0(\theta )|\stackrel{p}{\to }0.}$

The requirement for Θ to be compact can be replaced with a weaker assumption that the maximum of Q₀ was well-separated, that is there should not exist any points θ that are distant from θ₀ but such that Q₀(θ) were close to Q₀(θ₀). Formally, it means that for any sequence {θ_i} such that Q₀(θ_i) → Q₀(θ₀), it should be true that θ_i → θ₀.

Asymptotic normality

Assuming that consistency has been established and the derivatives of the sample ${\displaystyle Q_n}$ satisfy some other conditions, the extremum estimator converges to an asymptotically Normal distribution.

Examples

• Maximum likelihood estimation uses the objective function

  ${\displaystyle {\hat{Q}}_n(\theta )=\log \left[\prod _{i=1}^{n}f(x_i|\theta )\right]=\sum _{i=1}^n\log f(x_i|\theta ),}$

  where f(·|θ) is the density function of the distribution from where the observations are drawn. This objective function is called the log-likelihood function.
• Generalized method of moments estimator is defined through the objective function

  ${\displaystyle {\hat{Q}}_n(\theta )=-(\frac{1}{n}\sum _{i=1}^{n}g(x_i,\theta ){)}^{\prime }{\hat{W}}_n(\frac{1}{n}\sum _{i=1}^{n}g(x_i,\theta )),}$

  where g(·|θ) is the moment condition of the model.
• Minimum distance estimator
• Least squares estimator