Coefficient of multiple correlation

Computation

The square of the coefficient of multiple correlation can be computed using the vector ${\displaystyle \mathbf{c}={(r_{x_{1}y},r_{x_{2}y},\dots ,r_{x_{N}y})}^{\mathrm{\top }}}$ of correlations ${\displaystyle r_{x_{n}y}}$ between the predictor variables ${\displaystyle x_n}$ (independent variables) and the target variable ${\displaystyle y}$ (dependent variable), and the correlation matrix ${\displaystyle R_{xx}}$ of correlations between predictor variables. It is given by

${\displaystyle R^2={\mathbf{c}}^{\mathrm{\top }}{R}_{xx}^{-1}\mathbf{c},}$

where ${\displaystyle {\mathbf{c}}^{\mathrm{\top }}}$ is the transpose of ${\displaystyle \mathbf{c}}$, and ${\displaystyle R_{xx}^{-1}}$ is the inverse of the matrix

${\displaystyle R_{xx}=\left(\begin{array}{cccc}{r}_{x_1{x}_1}& r_{x_1{x}_2}& \dots & r_{x_1{x}_N}\\ r_{x_2{x}_1}& \ddots & & ⋮\\ ⋮& & \ddots & \\ r_{x_N{x}_1}& \dots & & r_{x_N{x}_N}\end{array}\right).}$

If all the predictor variables are uncorrelated, the matrix ${\displaystyle R_{xx}}$ is the identity matrix and ${\displaystyle R^2}$ simply equals ${\displaystyle {\mathbf{c}}^{\mathrm{\top }}\mathbf{c}}$, the sum of the squared correlations with the dependent variable. If the predictor variables are correlated among themselves, the inverse of the correlation matrix ${\displaystyle R_{xx}}$ accounts for this.

The squared coefficient of multiple correlation can also be computed as the fraction of variance of the dependent variable that is explained by the independent variables, which in turn is 1 minus the unexplained fraction. The unexplained fraction can be computed as the sum of squares of residuals—that is, the sum of the squares of the prediction errors—divided by the sum of squares of deviations of the values of the dependent variable from its expected value.

Properties

With more than two variables being related to each other, the value of the coefficient of multiple correlation depends on the choice of dependent variable: a regression of ${\displaystyle y}$ on ${\displaystyle x}$ and ${\displaystyle z}$ will in general have a different ${\displaystyle R}$ than will a regression of ${\displaystyle z}$ on ${\displaystyle x}$ and ${\displaystyle y}$. For example, suppose that in a particular sample the variable ${\displaystyle z}$ is uncorrelated with both ${\displaystyle x}$ and ${\displaystyle y}$, while ${\displaystyle x}$ and ${\displaystyle y}$ are linearly related to each other. Then a regression of ${\displaystyle z}$ on ${\displaystyle y}$ and ${\displaystyle x}$ will yield an ${\displaystyle R}$ of zero, while a regression of ${\displaystyle y}$ on ${\displaystyle x}$ and ${\displaystyle z}$ will yield a strictly positive ${\displaystyle R}$. This follows since the correlation of ${\displaystyle y}$ with its best predictor based on ${\displaystyle x}$ and ${\displaystyle z}$ is in all cases at least as large as the correlation of ${\displaystyle y}$ with its best predictor based on ${\displaystyle x}$ alone, and in this case with ${\displaystyle z}$ providing no explanatory power it will be exactly as large.