Definite quadratic form

Examples

As an example, let ${\displaystyle V={\mathbb{R}}^2}$, and consider the quadratic form

${\displaystyle Q(x)=c_1{x_1}^2+c_2{x_2}^2}$

where ${\displaystyle x=[x_1,x_2]\in V}$ and c₁ and c₂ are constants. If c₁ > 0 and c₂ > 0 , the quadratic form Q is positive-definite, so Q evaluates to a positive number whenever ${\displaystyle [x_1,x_2]\ne [0,0].}$ If one of the constants is positive and the other is 0, then Q is positive semidefinite and always evaluates to either 0 or a positive number. If c₁ > 0 and c₂ < 0 , or vice versa, then Q is indefinite and sometimes evaluates to a positive number and sometimes to a negative number. If c₁ < 0 and c₂ < 0 , the quadratic form is negative-definite and always evaluates to a negative number whenever ${\displaystyle [x_1,x_2]\ne [0,0].}$ And if one of the constants is negative and the other is 0, then Q is negative semidefinite and always evaluates to either 0 or a negative number.

In general a quadratic form in two variables will also involve a cross-product term in x₁·x₂:

${\displaystyle Q(x)=c_1{x_1}^2+c_2{x_2}^2+2c_3{x}_1{x}_2.}$

This quadratic form is positive-definite if ${\displaystyle c_1>0}$ and ${\displaystyle c_1{c}_2-{c_3}^2>0,}$ negative-definite if ${\displaystyle c_1<0}$ and ${\displaystyle c_1{c}_2-{c_3}^2>0,}$ and indefinite if ${\displaystyle c_1{c}_2-{c_3}^2<0.}$ It is positive or negative semidefinite if ${\displaystyle c_1{c}_2-{c_3}^2=0,}$ with the sign of the semidefiniteness coinciding with the sign of ${\displaystyle c_1.}$

This bivariate quadratic form appears in the context of conic sections centered on the origin. If the general quadratic form above is equated to 0, the resulting equation is that of an ellipse if the quadratic form is positive or negative-definite, a hyperbola if it is indefinite, and a parabola if ${\displaystyle c_1{c}_2-{c_3}^2=0.}$

The square of the Euclidean norm in n-dimensional space, the most commonly used measure of distance, is

${\displaystyle {x_1}^2+\cdots +{x_n}^2.}$

In two dimensions this means that the distance between two points is the square root of the sum of the squared distances along the ${\displaystyle x_1}$ axis and the ${\displaystyle x_2}$ axis.

Matrix form

A quadratic form can be written in terms of matrices as

${\displaystyle x^{\mathsf{T}}Ax}$

where x is any n×1 Cartesian vector ${\displaystyle [x_1,\cdots ,x_n{]}^{\mathsf{T}}}$ in which at least one element is not 0; A is an n × n symmetric matrix; and superscript ^T denotes a matrix transpose. If A is diagonal this is equivalent to a non-matrix form containing solely terms involving squared variables; but if A has any non-zero off-diagonal elements, the non-matrix form will also contain some terms involving products of two different variables.

Positive or negative-definiteness or semi-definiteness, or indefiniteness, of this quadratic form is equivalent to the same property of A, which can be checked by considering all eigenvalues of A or by checking the signs of all of its principal minors.

Optimization

Definite quadratic forms lend themselves readily to optimization problems. Suppose the matrix quadratic form is augmented with linear terms, as

${\displaystyle x^{\mathsf{T}}Ax+b^{\mathsf{T}}x,}$

where b is an n×1 vector of constants. The first-order conditions for a maximum or minimum are found by setting the matrix derivative to the zero vector:

${\displaystyle 2Ax+b=\overrightarrow{0},}$

giving

${\displaystyle x=-\frac{1}{2}{A}^{-1}b,}$

assuming A is nonsingular. If the quadratic form, and hence A, is positive-definite, the second-order conditions for a minimum are met at this point. If the quadratic form is negative-definite, the second-order conditions for a maximum are met.

An important example of such an optimization arises in multiple regression, in which a vector of estimated parameters is sought which minimizes the sum of squared deviations from a perfect fit within the dataset.