Influential observation

Assessment

Various methods have been proposed for measuring influence. Assume an estimated regression ${\displaystyle \mathbf{y}=\mathbf{X}\mathbf{b}+\mathbf{e}}$, where ${\displaystyle \mathbf{y}}$ is an n×1 column vector for the response variable, ${\displaystyle \mathbf{X}}$ is the n×k design matrix of explanatory variables (including a constant), ${\displaystyle \mathbf{e}}$ is the n×1 residual vector, and ${\displaystyle \mathbf{b}}$ is a k×1 vector of estimates of some population parameter ${\displaystyle \beta \in {\mathbb{R}}^k}$. Also define ${\displaystyle \mathbf{H}\equiv \mathbf{X}{\left({\mathbf{X}}^{\mathsf{T}}\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\mathsf{T}}}$, the projection matrix of ${\displaystyle \mathbf{X}}$. Then we have the following measures of influence:

1. ${\displaystyle {\text{DFBETA}}_i\equiv \mathbf{b}-{\mathbf{b}}_{(-i)}=\frac{{\left({\mathbf{X}}^{\mathsf{T}}\mathbf{X}\right)}^{-1}{\mathbf{x}}_i^{\mathsf{T}}{e}_i}{1-h_{ii}}}$, where ${\displaystyle {\mathbf{b}}_{(-i)}}$ denotes the coefficients estimated with the i-th row ${\displaystyle {\mathbf{x}}_i}$ of ${\displaystyle \mathbf{X}}$ deleted, ${\displaystyle h_{ii}={\mathbf{x}}_i{\left({\mathbf{X}}^{\mathsf{T}}\mathbf{X}\right)}^{-1}{\mathbf{x}}_i^{\mathsf{T}}}$ denotes the i-th value of matrix's ${\displaystyle \mathbf{H}}$ main diagonal. Thus DFBETA measures the difference in each parameter estimate with and without the influential point. There is a DFBETA for each variable and each observation (if there are N observations and k variables there are N·k DFBETAs). Table shows DFBETAs for the third dataset from Anscombe's quartet (bottom left chart in the figure):

1. DFFITS - difference in fits
2. Cook's D measures the effect of removing a data point on all the parameters combined.