Partial regression plot

Calculation

Partial regression plots are formed by:

1. Computing the residuals of regressing the response variable against the independent variables but omitting X_i
2. Computing the residuals from regressing X_i against the remaining independent variables
3. Plotting the residuals from (1) against the residuals from (2).

Velleman and Welsch express this mathematically as:

${\displaystyle Y_{\bullet [i]}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{u}\mathrm{s}{X}_{i\bullet [i]}}$

where

Y_•[i] = residuals from regressing Y (the response variable) against all the independent variables except Xi

X_i•[i] = residuals from regressing X_i against the remaining independent variables.

Properties

Velleman and Welsch list the following useful properties for this plot:

1. The least squares linear fit to this plot has an intercept of 0 and a slope ${\displaystyle {\beta }_i}$, where ${\displaystyle {\beta }_i}$ corresponds to the regression coefficient for X_i of a regression of Y on all of the covariates.
2. The residuals from the least squares linear fit to this plot are identical to the residuals from the least squares fit of the original model (Y against all the independent variables including Xi).
3. The influences of individual data values on the estimation of a coefficient are easy to see in this plot.
4. It is easy to see many kinds of failures of the model or violations of the underlying assumptions (nonlinearity, heteroscedasticity, unusual patterns). .

Partial regression plots are related to, but distinct from, partial residual plots. Partial regression plots are most commonly used to identify data points with high leverage and influential data points that might not have high leverage. Partial residual plots are most commonly used to identify the nature of the relationship between Y and X_i (given the effect of the other independent variables in the model). Note that since the simple correlation between the two sets of residuals plotted is equal to the partial correlation between the response variable and X_i, partial regression plots will show the correct strength of the linear relationship between the response variable and X_i. This is not true for partial residual plots. On the other hand, for the partial regression plot, the x-axis is not X_i. This limits its usefulness in determining the need for a transformation (which is the primary purpose of the partial residual plot).