Invertible matrix

Basic idea

A matrix can be viewed as a rule for transforming vectors. For example, a real ${\displaystyle n\times n}$ matrix ${\displaystyle A}$ defines a linear transformation

${\displaystyle x\mapsto Ax}$

from the set ${\displaystyle {\mathbb{R}}^n}$ of ${\displaystyle n}$-tuples of real numbers to itself. The matrix is invertible when this transformation can be undone by another linear transformation. In that case, there is a matrix ${\displaystyle A^{-1}}$ such that applying ${\displaystyle A}$ and then ${\displaystyle A^{-1}}$, or applying ${\displaystyle A^{-1}}$ and then ${\displaystyle A}$, returns every vector to where it started.

Geometrically, an invertible matrix does not collapse space into a lower-dimensional set. It sends distinct vectors to distinct vectors and reaches every vector in the target space. For a real matrix, this is reflected by its determinant: an invertible matrix has nonzero determinant, while a matrix with determinant zero collapses volume to zero and is not invertible.

Algebraically, invertibility means that the linear system

${\displaystyle Ax=b}$

has a unique solution ${\displaystyle x}$ for every vector ${\displaystyle b}$. Equivalently, the columns of ${\displaystyle A}$ form a basis of the vector space. These and several other equivalent characterizations are summarized by the invertible matrix theorem.

Examples

An invertible matrix

Consider the following 2-by-2 matrix:

${\displaystyle \mathbf{A}=\left(\begin{array}{cc}-1& \frac{3}{2}\\ 1& -1\end{array}\right)}$

The matrix ${\displaystyle \mathbf{A}}$ is invertible, as it has inverse ${\displaystyle \mathbf{B}=\left(\begin{array}{cc}2& 3\\ 2& 2\end{array}\right),}$ which can be confirmed by computing

${\displaystyle \mathbf{A}\mathbf{B}=\left(\begin{array}{cc}-1& \frac{3}{2}\\ 1& -1\end{array}\right)\left(\begin{array}{cc}2& 3\\ 2& 2\end{array}\right)=\left(\begin{array}{cc}(-1)\times 2+\frac{3}{2}\times 2& (-1)\times 3+\frac{3}{2}\times 2\\ 1\times 2+(-1)\times 2& 1\times 3+(-1)\times 2\end{array}\right)=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)={\mathbf{I}}_2}$

To check that it is invertible without finding an inverse, $det\mathbf{A}=-\frac{1}{2}$ can be computed, which is non-zero.

A non-invertible matrix

This is a non-invertible (singular) matrix:

${\displaystyle \mathbf{C}=\left(\begin{array}{cc}2& 4\\ 2& 4\end{array}\right)}$

This matrix is not invertible, because different vectors ${\displaystyle x}$ and ${\displaystyle x^{\prime }}$ have the same value of ${\displaystyle \mathbf{C}x}$, for example, ${\displaystyle x=\left[\begin{array}{c}2\\ -1\end{array}\right]}$ and ${\displaystyle x^{\prime }=\left[\begin{array}{c}0\\ 0\end{array}\right]}$ both give ${\displaystyle \mathbf{C}x=\left[\begin{array}{c}0\\ 0\end{array}\right]}$. So it is impossible to reverse the transformation ${\displaystyle x\mapsto \mathbf{C}x}$, since different inputs produce the same output. The determinant of ${\displaystyle \mathbf{C}}$ is 0, which is a necessary and sufficient condition for a matrix to be non-invertible.

Explicit formula for 2x2 matrices

For an arbitrary 2-by-2 matrix ${\displaystyle \mathbf{A}}$ defined as

${\displaystyle \mathbf{A}=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)}$,

its inverse is

${\displaystyle {\mathbf{A}}^{-1}=\frac{1}{det\mathbf{A}}\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right)}$.

Criteria for invertibility

A matrix ${\displaystyle \mathbf{A}}$ (over a field such as the real numbers) is invertible if and only if the only solution to the equation ${\displaystyle \mathbf{A}x=0}$ is the zero vector. That is, the nullity of ${\displaystyle \mathbf{A}}$ is zero: its nullspace consists only of the zero vector. Equivalently, an ${\displaystyle n\times n}$ matrix is invertible if its rank is ${\displaystyle n}$, by the rank-nullity theorem. Such a matrix is said to be of full rank. Geometrically, this means that the column space of ${\displaystyle \mathbf{A}}$ is all of ${\displaystyle {\mathbb{R}}^n}$. The rank and nullity characterizations are related in the following way: having full rank means that the matrix maps onto all of ${\displaystyle {\mathbb{R}}^n}$ (surjectively), and having nullity zero means that the matrix is one-to-one. That is, it is a bijection, and therefore inveritble.

A more computational criterion for invertibility comes from the standard way of determining the rank and nullity of a matrix. For a square matrix, a matrix is invertible if and only if its row-reduced echelon form is the identity matrix. The reason is that this is the only row-reduced echelon matrix having full rank: all ones down the diagonal and zeros everywhere else.

Another criterion is that a matrix is invertible if and only if its determinant is non-zero.

Methods of matrix inversion

There are many methods to compute the inverse matrix, when it exists. A basic method is to use Gaussian elimination, for example. This proceeds by considering the inverse matrix ${\displaystyle A^{-1}}$ as a product of elementary matrices, ${\displaystyle A^{-1}=E_k{E}_{k-1}\dots E_1}$ where the ${\displaystyle E_i}$ are the elementary matrices corresponding to the elementary row operations needed to put ${\displaystyle A}$ into its row-reduced echelon form. This method has the advantage that it will produce a row-reduced echelon form regardless of invertibility, and so gives a criterion to decide if a matrix is invertible which is often more efficient than computing the determinant: the matrix is invertible if and only if, at the end of the process, the echelon form is the identity matrix rather than some matrix of lower rank.

To convert this to a practical method for determining using this method, one uses an augmented matrix whose left side is the matrix to invert and whose right side is the identity matrix. Then, Gaussian elimination is used to convert the left side into the identity matrix, which causes the right side to become the inverse of the input matrix.

For example, take the following matrix: $${\displaystyle \mathbf{A}=\left(\begin{array}{cc}-1& \frac{3}{2}\\ 1& -1\end{array}\right)}$$

The first step to compute its inverse is to create the augmented matrix $${\displaystyle \left(\begin{array}{cccc}-1& \frac{3}{2}& 1& 0\\ 1& -1& 0& 1\end{array}\right)}$$

Call the first row of this matrix ${\displaystyle R_1}$ and the second row ${\displaystyle R_2}$. Then, add row 1 to row 2 ${\displaystyle (R_1+R_2\to R_2).}$ This yields $${\displaystyle \left(\begin{array}{cccc}-1& \frac{3}{2}& 1& 0\\ 0& \frac{1}{2}& 1& 1\end{array}\right)}$$

Next, subtract row 2, multiplied by 3, from row 1 ${\displaystyle (R_1-3R_2\to R_1),}$ which yields $${\displaystyle \left(\begin{array}{cccc}-1& 0& -2& -3\\ 0& \frac{1}{2}& 1& 1\end{array}\right)}$$

Finally, multiply row 1 by −1 ${\displaystyle (-R_1\to R_1)}$ and row 2 by 2 ${\displaystyle (2R_2\to R_2).}$ This yields the identity matrix on the left side and the inverse matrix on the right:$${\displaystyle \left(\begin{array}{cccc}1& 0& 2& 3\\ 0& 1& 2& 2\end{array}\right)}$$

Thus, $${\displaystyle {\mathbf{A}}^{-1}=\left(\begin{array}{cc}2& 3\\ 2& 2\end{array}\right)}$$ It works because the process of Gaussian elimination can be viewed as a sequence of applying left matrix multiplication using elementary row operations using elementary matrices (${\displaystyle {\mathbf{E}}_n}$), such as ${\displaystyle {\mathbf{E}}_n{\mathbf{E}}_{n-1}\cdots {\mathbf{E}}_2{\mathbf{E}}_1\mathbf{A}=\mathbf{I}}$

Applying right-multiplication using ${\displaystyle {\mathbf{A}}^{-1},}$ we get ${\displaystyle {\mathbf{E}}_n{\mathbf{E}}_{n-1}\cdots {\mathbf{E}}_2{\mathbf{E}}_1\mathbf{I}=\mathbf{I}{\mathbf{A}}^{-1}.}$ And the right side ${\displaystyle \mathbf{I}{\mathbf{A}}^{-1}={\mathbf{A}}^{-1},}$ which is the inverse we want.

Properties

Singularity

Over a field, a square matrix that is not invertible is called singular or degenerate. A square matrix with entries in a field is singular if and only if its determinant is zero.

Invertible matrix theorem

Let A be a square n-by-n matrix over a field K (e.g., the field ⁠${\displaystyle \mathbb{R}}$⁠ of real numbers). The following statements are equivalent, i.e., they are either all true or all false for any given matrix:

• A is invertible, i.e. it has an inverse under matrix multiplication, i.e., there exists a B such that AB = I_n = BA. (In that statement, "invertible" can equivalently be replaced with "left-invertible" or "right-invertible" in which one-sided inverses are considered.)
• The linear transformation mapping x to Ax is invertible, i.e., it has an inverse under function composition. (There, again, "invertible" can equivalently be replaced with either "left-invertible" or "right-invertible".)
• The transpose A^T is an invertible matrix.
• A is row-equivalent to the n-by-n identity matrix I_n.
• A is column-equivalent to the n-by-n identity matrix I_n.
• A has n pivot positions.
• A has full rank: rank A = n.
• A has a trivial kernel: ker(A) = {0}.
• The linear transformation mapping x to Ax is bijective; that is, the equation Ax = b has exactly one solution for each b in Kⁿ. (There, "bijective" can equivalently be replaced with "injective" or "surjective".)
• The columns of A form a basis of Kⁿ. (In this statement, "basis" can equivalently be replaced with either "linearly independent set" or "spanning set")
• The rows of A form a basis of Kⁿ. (Similarly, here, "basis" can equivalently be replaced with either "linearly independent set" or "spanning set")
• The determinant of A is nonzero: det A ≠ 0. In general, a square matrix over a commutative ring is invertible if and only if its determinant is a unit (i.e. multiplicatively invertible element) of that ring.
• The number 0 is not an eigenvalue of A. (More generally, a number ${\displaystyle \lambda }$ is an eigenvalue of A if the matrix ${\displaystyle \mathbf{A}-\lambda \mathbf{I}}$ is singular, where I is the identity matrix.)
• The matrix A can be expressed as a finite product of elementary matrices.

Other properties

Furthermore, the following properties hold for an invertible matrix A:

• ${\displaystyle ({\mathbf{A}}^{-1}{)}^{-1}=\mathbf{A}}$
• ${\displaystyle (k\mathbf{A}{)}^{-1}=k^{-1}{\mathbf{A}}^{-1}}$ for nonzero scalar k
• ${\displaystyle (\mathbf{A}\mathbf{x}{)}^{+}={\mathbf{x}}^{+}{\mathbf{A}}^{-1}}$ if A has orthonormal columns, where ⁺ denotes the Moore–Penrose inverse and x is a vector
• ${\displaystyle ({\mathbf{A}}^{\mathrm{T}}{)}^{-1}=({\mathbf{A}}^{-1}{)}^{\mathrm{T}}}$
• For any invertible n-by-n matrices A and B, ${\displaystyle (\mathbf{A}\mathbf{B}{)}^{-1}={\mathbf{B}}^{-1}{\mathbf{A}}^{-1}.}$ More generally, if ${\displaystyle {\mathbf{A}}_1,\dots ,{\mathbf{A}}_k}$ are invertible n-by-n matrices, then ${\displaystyle ({\mathbf{A}}_1{\mathbf{A}}_2\cdots {\mathbf{A}}_{k-1}{\mathbf{A}}_k{)}^{-1}={\mathbf{A}}_k^{-1}{\mathbf{A}}_{k-1}^{-1}\cdots {\mathbf{A}}_2^{-1}{\mathbf{A}}_1^{-1}.}$
• ${\displaystyle det{\mathbf{A}}^{-1}=(det\mathbf{A}{)}^{-1}.}$
• Left and right inverses are equal. That is, if ${\displaystyle \mathbf{L}\mathbf{A}=\mathbf{I}}$ and ${\displaystyle \mathbf{A}\mathbf{R}=\mathbf{I}}$ then ${\displaystyle \mathbf{L}=\mathbf{L}(\mathbf{A}\mathbf{R})=(\mathbf{L}\mathbf{A})\mathbf{R}=\mathbf{R}}$.

The rows of the inverse matrix V of a matrix U are orthonormal to the columns of U (and vice versa interchanging rows for columns). To see this, suppose that UV = VU = I where the rows of V are denoted as ${\displaystyle v_i^{\mathrm{T}}}$ and the columns of U as ${\displaystyle u_j}$ for ${\displaystyle 1\le i,j\le n.}$ Then clearly, the Euclidean inner product of any two ${\displaystyle v_i^{\mathrm{T}}{u}_j={\delta }_{i,j}.}$ This property can also be useful in constructing the inverse of a square matrix in some instances, where a set of orthogonal vectors (but not necessarily orthonormal vectors) to the columns of U are known. In which case, one can apply the iterative Gram–Schmidt process to this initial set to determine the rows of the inverse V.

A matrix that is its own inverse (i.e., a matrix A such that A = A⁻¹ and consequently A² = I) is called an involutory matrix.

In relation to its adjugate

The adjugate of a matrix A is a matrix ${\displaystyle \mathrm{adj}(A)}$ which exists independently of the invertibility of A. It satisfies the identity $${\displaystyle \mathrm{adj}(\mathbf{A})\mathbf{A}=\mathbf{A}\mathrm{adj}(\mathbf{A})=det(\mathbf{A})I.}$$ In particular, if A is invertible, then

${\displaystyle {\mathbf{A}}^{-1}=\frac{1}{det(\mathbf{A})}\mathrm{adj}(\mathbf{A})}$

In relation to the identity matrix

It follows from the associativity of matrix multiplication that if

${\displaystyle \mathbf{A}\mathbf{B}=\mathbf{I}}$

for finite square matrices A and B, then also

${\displaystyle \mathbf{B}\mathbf{A}=\mathbf{I}}$

This identity does not hold for non-square rectangular matrices, and need not be true for linear operators in infinite dimensions.

Density

Over the field of real numbers, the set of singular n-by-n matrices, considered as a subset of ⁠${\displaystyle {\mathbb{R}}^{n\times n},}$⁠ is a null set, that is, has Lebesgue measure zero. That is true because singular matrices are the roots of the determinant function. It is a continuous function because it is a polynomial in the entries of the matrix. Thus in the language of measure theory, almost all n-by-n matrices are invertible.

Furthermore, the set of n-by-n invertible matrices is open and dense in the topological space of all n-by-n matrices. Equivalently, the set of singular matrices is closed and nowhere dense in the space of n-by-n matrices.

In practice, however, non-invertible matrices may be encountered. In numerical calculations, matrices that are invertible but close to a non-invertible matrix may still be problematic and are said to be ill-conditioned.

Derivative of the matrix inverse

Suppose that the invertible matrix A depends on a parameter t. Then the derivative of the inverse of A with respect to t is given by

${\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}{\mathbf{A}}^{-1}=-{\mathbf{A}}^{-1}\frac{\mathrm{d}\mathbf{A}}{\mathrm{d}t}{\mathbf{A}}^{-1}}$

To derive the above expression for the derivative of the inverse of A, one can differentiate the definition of the matrix inverse ${\displaystyle {\mathbf{A}}^{-1}\mathbf{A}=\mathbf{I}}$ using the product rule, and then solve for the derivative of the inverse of A:

${\displaystyle \mathbf{0}=\frac{\mathrm{d}\mathbf{I}}{\mathrm{d}t}=\frac{\mathrm{d}({\mathbf{A}}^{-1}\mathbf{A})}{\mathrm{d}t}=\frac{\mathrm{d}({\mathbf{A}}^{-1})}{\mathrm{d}t}\mathbf{A}+{\mathbf{A}}^{-1}\frac{\mathrm{d}\mathbf{A}}{\mathrm{d}t}}$

Subtracting ${\displaystyle {\mathbf{A}}^{-1}\frac{\mathrm{d}\mathbf{A}}{\mathrm{d}t}}$ from both ends of this formula, and multiplying on the right by ${\displaystyle {\mathbf{A}}^{-1}}$ finishes the derivation.

If ${\displaystyle \epsilon }$ is a small number then the derivative formula gives:

${\displaystyle {\left(\mathbf{A}+\epsilon \mathbf{X}\right)}^{-1}={\mathbf{A}}^{-1}-\epsilon {\mathbf{A}}^{-1}\mathbf{X}{\mathbf{A}}^{-1}+\mathcal{O}({\epsilon }^2)}$

Given a positive integer ${\displaystyle n}$,

${\displaystyle \begin{array}{cc}\frac{\mathrm{d}}{\mathrm{d}t}{\mathbf{A}}^n& =\sum _{i=1}^n{\mathbf{A}}^{i-1}\frac{\mathrm{d}\mathbf{A}}{\mathrm{d}t}{\mathbf{A}}^{n-i},\\ \frac{\mathrm{d}}{\mathrm{d}t}{\mathbf{A}}^{-n}& =-\sum _{i=1}^n{\mathbf{A}}^{-i}\frac{\mathrm{d}\mathbf{A}}{\mathrm{d}t}{\mathbf{A}}^{-(n+1-i)}\end{array}}$

In particular,

${\displaystyle \begin{array}{cc}(\mathbf{A}+\epsilon \mathbf{X}{)}^n& ={\mathbf{A}}^n+\epsilon \sum _{i=1}^n{\mathbf{A}}^{i-1}\mathbf{X}{\mathbf{A}}^{n-i}+\mathcal{O}\left({\epsilon }^2\right),\\ (\mathbf{A}+\epsilon \mathbf{X}{)}^{-n}& ={\mathbf{A}}^{-n}-\epsilon \sum _{i=1}^n{\mathbf{A}}^{-i}\mathbf{X}{\mathbf{A}}^{-(n+1-i)}+\mathcal{O}\left({\epsilon }^2\right)\end{array}}$

The derivative of a matrix determinant with respect to one of its elements is a matrix cofactor, which exhibits the transpose of the matrix inverse: $${\displaystyle \frac{\mathrm{d}}{\mathrm{d}{A}_{ij}}det(\mathbf{A})={\mathbf{C}}_{ij}={A^{-1}}_{ji}det(\mathbf{A})}$$

Generalizations

Non-square matrices

Non-square matrices, i.e. m-by-n matrices for which m ≠ n, do not have an inverse. However, in some cases such a matrix may have a left inverse or right inverse. If A is m-by-n and the rank of A is equal to n, (n ≤ m), then A has a left inverse, an n-by-m matrix B such that BA = I_n. If A has rank m (m ≤ n), then it has a right inverse, an n-by-m matrix B such that AB = I_m.

Some of the properties of inverse matrices are shared by generalized inverses (such as the Moore–Penrose inverse), which can be defined for any m-by-n matrix.

In Abstract algebra

While the most common case is that of matrices over the real or complex numbers, all of those definitions can be given for matrices over any algebraic structure equipped with addition and multiplication (i.e. rings). However, in the case of a ring being commutative, the condition for a square matrix to be invertible is that its determinant is invertible in the ring, which in general is a stricter requirement than it being nonzero. For a noncommutative ring, the usual determinant is not defined. The conditions for existence of left-inverse or right-inverse are more complicated, since a notion of rank does not exist over rings.

The set of n × n invertible matrices together with the operation of matrix multiplication and entries from ring R form a group, the general linear group of degree n, denoted GL_n(R).

Practical Considerations

For most practical applications, it is not necessary to invert a matrix to solve a system of linear equations, since Gaussian elimination is computationally less expensive. However, for a unique solution to exist, it is necessary for the matrix involved to be invertible.

Matrix decomposition techniques like LU decomposition are much faster than inversion, and various fast algorithms for special classes of linear systems have also been developed.

Applications

Regression/least squares

Although an explicit inverse is not necessary to estimate the vector of unknowns, it is the easiest way to estimate their accuracy and is found in the diagonal of a matrix inverse (the posterior covariance matrix of the vector of unknowns). However, faster algorithms to compute only the diagonal entries of a matrix inverse are known in many cases.

Matrix inverses in real-time simulations

Matrix inversion plays a significant role in computer graphics, particularly in 3D graphics rendering and 3D simulations. Examples include screen-to-world ray casting, world-to-subspace-to-world object transformations, and physical simulations.

Matrix inverses in MIMO wireless communication

Matrix inversion also plays a significant role in the MIMO (Multiple-Input, Multiple-Output) technology in wireless communications. The MIMO system consists of N transmit and M receive antennas. Unique signals, occupying the same frequency band, are sent via N transmit antennas and are received via M receive antennas. The signal arriving at each receive antenna will be a linear combination of the N transmitted signals forming an N × M transmission matrix H. It is crucial for the matrix H to be invertible so that the receiver can figure out the transmitted information.

Diagonalization

Matrix inverses play a prominent role in diagonalization. An arbitrary matrix ${\displaystyle \mathbf{A}}$ is diagonalizable if and only if

${\displaystyle \mathbf{A}=\mathbf{P}\mathbf{D}{\mathbf{P}}^{-1}}$

where ${\displaystyle \mathbf{D}}$ is a diagonal matrix and ${\displaystyle \mathbf{P}}$ is an invertible matrix. This expression allows arbitrary powers of ${\displaystyle \mathbf{A}}$ to be written in the compact form

${\displaystyle {\mathbf{A}}^k=\mathbf{P}{\mathbf{D}}^k{\mathbf{P}}^{-1}}$

where ${\displaystyle k}$ is a positive integer.