Cholesky decomposition

LDL decomposition

A closely related variant of the classical Cholesky decomposition is the LDL decomposition, also known as Bunch-Kaufman factorization

$${\displaystyle \mathbf{A}={\mathbf{L}\mathbf{D}\mathbf{L}}^{\ast },}$$

where L is a lower unit triangular (unitriangular) matrix, and D is a diagonal matrix. That is, the diagonal elements of L are required to be 1 at the cost of introducing an additional diagonal matrix D in the decomposition. The main advantage is that the LDL decomposition can be computed and used with essentially the same algorithms, but avoids extracting square roots.

For this reason, the LDL decomposition is often called the square-root-free Cholesky decomposition. For real matrices, the factorization has the form A = LDL^T and is often referred to as LDLT decomposition (or LDL^T decomposition, or LDL′). It is reminiscent of the eigendecomposition of real symmetric matrices, A = QΛQ^T, but is quite different in practice because Λ and D are not similar matrices.

The LDL decomposition is related to the classical Cholesky decomposition of the form LL* as follows:

$${\displaystyle \mathbf{A}={\mathbf{L}\mathbf{D}\mathbf{L}}^{\ast }=\mathbf{L}{\mathbf{D}}^{1/2}{\left({\mathbf{D}}^{1/2}\right)}^{\ast }{\mathbf{L}}^{\ast }=\mathbf{L}{\mathbf{D}}^{1/2}{\left(\mathbf{L}{\mathbf{D}}^{1/2}\right)}^{\ast }.}$$

Conversely, given the classical Cholesky decomposition $\mathbf{A}=\mathbf{C}{\mathbf{C}}^{\ast }$ of a positive definite matrix, if S is a diagonal matrix that contains the main diagonal of $\mathbf{C}$, then A can be decomposed as $\mathbf{L}\mathbf{D}{\mathbf{L}}^{\ast }$ where $${\displaystyle \mathbf{L}=\mathbf{C}{\mathbf{S}}^{-1}}$$ (this rescales each column to make diagonal elements 1), $${\displaystyle \mathbf{D}=\mathbf{S}{\mathbf{S}}^{\ast }.}$$

If A is positive definite then the diagonal elements of D are all positive. For positive semidefinite A, an $\mathbf{L}\mathbf{D}{\mathbf{L}}^{\ast }$ decomposition exists where the number of non-zero elements on the diagonal D is exactly the rank of A. Some indefinite matrices for which no Cholesky decomposition exists have an LDL decomposition with negative entries in D: it suffices that the first n − 1 leading principal minors of A are non-singular.

Example

Here is the Cholesky decomposition of a symmetric real matrix:

$${\displaystyle \begin{array}{c}\left(\begin{array}{ccc}4& 12& -16\\ 12& 37& -43\\ -16& -43& 98\end{array}\right)=\left(\begin{array}{ccc}2& 0& 0\\ 6& 1& 0\\ -8& 5& 3\end{array}\right)\left(\begin{array}{ccc}2& 6& -8\\ 0& 1& 5\\ 0& 0& 3\end{array}\right).\end{array}}$$

And here is its LDL^T decomposition:

$${\displaystyle \begin{array}{cc}\left(\begin{array}{ccc}4& 12& -16\\ 12& 37& -43\\ -16& -43& 98\end{array}\right)& =\left(\begin{array}{ccc}1& 0& 0\\ 3& 1& 0\\ -4& 5& 1\end{array}\right)\left(\begin{array}{ccc}4& 0& 0\\ 0& 1& 0\\ 0& 0& 9\end{array}\right)\left(\begin{array}{ccc}1& 3& -4\\ 0& 1& 5\\ 0& 0& 1\end{array}\right).\end{array}}$$

Geometric interpretation

The Cholesky decomposition is equivalent to a particular choice of conjugate axes of an ellipsoid. In detail, let the ellipsoid be defined as $y^{T}Ay=1$, then by definition, a set of vectors $v_1,...,v_n$ are conjugate axes of the ellipsoid iff $v_i^{T}A{v}_j={\delta }_{ij}$. Then, the ellipsoid is precisely$${\displaystyle \left\{\sum _i{x}_i{v}_i:x^{T}x=1\right\}=f({\mathbb{S}}^n)}$$where $f$ maps the basis vector $e_i\mapsto v_i$, and ${\mathbb{S}}^n$ is the unit sphere in n dimensions. That is, the ellipsoid is a linear image of the unit sphere.

Define the matrix $V:=[v_1|v_2|\cdots |v_n]$, then $v_i^{T}A{v}_j={\delta }_{ij}$ is equivalent to $V^{T}AV=I$. Different choices of the conjugate axes correspond to different decompositions.

The Cholesky decomposition corresponds to choosing $v_1$ to be parallel to the first axis, $v_2$ to be within the plane spanned by the first two axes, and so on. This makes $V$ an upper-triangular matrix. Then, there is $A=L{L}^T$, where $L=(V^{-1}{)}^T$ is lower-triangular.

Similarly, principal component analysis corresponds to choosing $v_1,...,v_n$ to be perpendicular. Then, let $\lambda =1/\Vert v_i{\Vert }^2$ and $\mathrm{\Sigma }=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\lambda }_1,...,{\lambda }_n)$, and there is $V=U{\mathrm{\Sigma }}^{-1/2}$ where $U$ is an orthogonal matrix. This then yields $A=U\mathrm{\Sigma }{U}^T$.

Applications

Numerical solution of system of linear equations

The Cholesky decomposition is mainly used for the numerical solution of linear equations $\mathbf{A}\mathbf{x}=\mathbf{b}$. If A is symmetric and positive definite, then $\mathbf{A}\mathbf{x}=\mathbf{b}$ can be solved by first computing the Cholesky decomposition $\mathbf{A}={\mathbf{L}\mathbf{L}}^{\ast }$, then solving $\mathbf{L}\mathbf{y}=\mathbf{b}$ for y by forward substitution, and finally solving ${\mathbf{L}}^{\mathbf{\ast }}\mathbf{x}=\mathbf{y}$ for x by back substitution.

An alternative way to eliminate taking square roots in the ${\mathbf{L}\mathbf{L}}^{\ast }$ decomposition is to compute the LDL decomposition $\mathbf{A}={\mathbf{L}\mathbf{D}\mathbf{L}}^{\ast }$, then solving $\mathbf{L}\mathbf{y}=\mathbf{b}$ for y, and finally solving ${\mathbf{D}\mathbf{L}}^{\ast }\mathbf{x}=\mathbf{y}$.

For linear systems that can be put into symmetric form, the Cholesky decomposition (or its LDL variant) is the method of choice, for superior efficiency and numerical stability. Compared to the LU decomposition, it is roughly twice as efficient.

Linear least squares

In linear least squares problem one seeks a solution x of an over-determined system Ax = l, such that quadratic norm of the residual vector Ax-l is minimum. This may be accomplished by solving by Cholesky decomposition normal equations ${\displaystyle \mathbf{N}\mathbf{x}={\mathbf{A}}^{\mathsf{T}}\mathbf{l}}$, where ${\displaystyle \mathbf{N}={\mathbf{A}}^{\mathsf{T}}\mathbf{A}}$ is symmetric positive definite. Symmetric equation matrix may also come from an energy functional, which must be positive from physical considerations; this happens frequently in the numerical solution of partial differential equations.

Such method is economic and works well in many applications, however it fails for near singular N. This is best illustrated in pathological case of square ${\displaystyle \mathbf{A}}$, where determinant of N is square of that of the original system Ax = l. Then it is best to apply SVD or QR decomposition. Givens QR has the advantage that similarly to normal equations there is no need to keep the whole matrix A as it is possible to update Cholesky factor with consecutive rows of A.

Non-linear optimization

Non-linear least squares are a particular case of nonlinear optimization. Let $\mathbf{f}(\mathbf{x})=\mathbf{l}$ be an over-determined system of equations with a non-linear function ${\displaystyle \mathbf{f}}$ returning vector results. The aim is to minimize square norm of residuals $\mathbf{v}=\mathbf{f}(\mathbf{x})-\mathbf{l}$. An approximate Newton's method solution is obtained by expanding ${\displaystyle \mathbf{f}}$ into curtailed Taylor series ${\displaystyle \mathbf{f}\mathbf{(}{\mathbf{x}}_0\mathbf{+}\delta \mathbf{x}\mathbf{)}\approx \mathbf{f}\mathbf{(}{\mathbf{x}}_0\mathbf{)}\mathbf{+}\mathbf{(}\mathrm{\partial }\mathbf{f}\mathbf{/}\mathrm{\partial }\mathbf{x}\mathbf{)}\delta \mathbf{x}}$ yielding linear least squares problem for ${\displaystyle \delta \mathbf{x}}$

${\displaystyle \mathbf{(}\mathrm{\partial }\mathbf{f}\mathbf{/}\mathrm{\partial }\mathbf{x}\mathbf{)}\delta \mathbf{x}\mathbf{=}\mathbf{l}\mathbf{-}\mathbf{f}\mathbf{(}{\mathbf{x}}_0\mathbf{)}\mathbf{=}\mathbf{v}\mathbf{,}\underset{\delta \mathbf{x}}{min}\mathbf{=}\Vert \mathbf{v}{\Vert }^{\mathbf{2}}.}$

Of course because of neglect of higher Taylor terms such solution is only approximate, if it ever exists. Now one could update expansion point to ${\displaystyle {\mathbf{x}}_{\mathrm{n}+1}\mathbf{=}{\mathbf{x}}_{\mathrm{n}}\mathbf{+}\delta \mathbf{x}}$ and repeat the whole procedure, hoping that (i) iterations converge to a solution and (ii) that the solution is the one needed. Unfortunately neither is guaranteed and must be verified.

Non-linear least squares may be also applied to the linear least squares problem by setting ${\displaystyle {\mathbf{x}}_0\mathbf{=}\mathbf{0}}$ and ${\displaystyle \mathbf{f}\mathbf{(}{\mathbf{x}}_0\mathbf{)}\mathbf{=}\mathbf{A}\mathbf{x}}$. This may be useful if Cholesky decomposition yields an inaccurate inverse ${\displaystyle {\mathbf{R}}^{-1}}$ for the triangle matrix where ${\displaystyle {\mathbf{R}}^{\mathrm{T}}\mathbf{R}\mathbf{=}\mathbf{N}}$, because of rounding errors. Such a procedure is called a differential correction of the solution. As long as iterations converge, by virtue of the Banach fixed-point theorem they yield the solution with a precision that is only limited by the precision of the calculated residuals ${\displaystyle \mathbf{v}\mathbf{=}\mathbf{A}\mathbf{x}\mathbf{-}\mathbf{l}}$. The precision is independent rounding errors in ${\displaystyle {\mathbf{R}}^{-1}}$. Poor ${\displaystyle {\mathbf{R}}^{-1}}$ may restrict region of initial ${\displaystyle {\mathbf{x}}_0}$ yielding convergence or altogether preventing it. Usually convergence is slower e.g. linear so that ${\displaystyle \Vert \delta {\mathbf{x}}_{\mathrm{n}+1}\Vert \approx \Vert \mathbf{=}\alpha \delta {\mathbf{x}}_{\mathrm{n}}\Vert }$ where constant ${\displaystyle \alpha <1}$. Such slow convergence may be sped by Aitken ${\displaystyle {\delta }^2}$ method. If calculation of ${\displaystyle {\mathbf{R}}^{-1}}$ is very costly, it is possible to use it from previous iterations as long as convergence is maintained. Such Cholesky procedure may work even for Hilbert matrices, notoriously difficult to invert.

Non-linear multi-variate functions may be minimized over their parameters using variants of Newton's method called quasi-Newton methods. At iteration k, the search steps in a direction $p_k$ defined by solving $B_k{p}_k=-g_k$ for $p_k$, where $p_k$ is the step direction, $g_k$ is the gradient, and $B_k$ is an approximation to the Hessian matrix formed by repeating rank-1 updates at each iteration. Two well-known update formulas are called Davidon–Fletcher–Powell (DFP) and Broyden–Fletcher–Goldfarb–Shanno (BFGS). Loss of the positive-definite condition through round-off error is avoided if rather than updating an approximation to the inverse of the Hessian, one updates the Cholesky decomposition of an approximation of the Hessian matrix itself.

Monte Carlo simulation

The Cholesky decomposition is commonly used in the Monte Carlo method for simulating systems with multiple correlated variables. The covariance matrix is decomposed to give the lower-triangular L. Applying this to a vector of uncorrelated observations in a sample u produces a sample vector Lu with the covariance properties of the system being modeled.

The following simplified example shows the economy one gets from the Cholesky decomposition: suppose the goal is to generate two correlated normal variables $x_1$ and $x_2$ with given correlation coefficient $\rho$. To accomplish that, it is necessary to first generate two uncorrelated Gaussian random variables $z_1$ and $z_2$ (for example, via a Box–Muller transform). Given the required correlation coefficient $\rho$, the correlated normal variables can be obtained via the transformations $x_1=z_1$ and $x_2=\rho z_1+\sqrt{1-{\rho }^2}{z}_2$.

Kalman filters

Unscented Kalman filters commonly use the Cholesky decomposition to choose a set of so-called sigma points. The Kalman filter tracks the average state of a system as a vector x of length N and covariance as an N × N matrix P. The matrix P is always positive semi-definite and can be decomposed into LL^T. The columns of L can be added and subtracted from the mean x to form a set of 2N vectors called sigma points. These sigma points completely capture the mean and covariance of the system state.

Matrix inversion

The explicit inverse of a Hermitian matrix can be computed by Cholesky decomposition, in a manner similar to solving linear systems, using $n^3$ operations ($\frac{1}{2}{n}^3$ multiplications). The entire inversion can even be efficiently performed in-place.

A non-Hermitian matrix B can also be inverted using the following identity, where BB* will always be Hermitian:

$${\displaystyle {\mathbf{B}}^{-1}={\mathbf{B}}^{\ast }({\mathbf{B}\mathbf{B}}^{\ast }{)}^{-1}.}$$

Data Imputation

Cholesky decomposition can also be used to impute data. Variations of the expectation maximization algorithm among other data imputation algorithms, make use of Cholesky decomposition.

Computation

There are various methods for calculating the Cholesky decomposition. The computational complexity of commonly used algorithms is O(n³) in general. The algorithms described below all involve about (1/3)n³ FLOPs (n³/6 multiplications and the same number of additions) for real flavors and (4/3)n³ FLOPs for complex flavors, where n is the size of the matrix A. Hence, they have half the cost of the LU decomposition, which uses 2n³/3 FLOPs (see Trefethen and Bau 1997).

Which of the algorithms below is faster depends on the details of the implementation. Generally, the first algorithm will be slightly slower because it accesses the data in a less regular manner. The Cholesky decomposition was shown to be numerically stable without need for pivoting.

The Cholesky algorithm

The Cholesky algorithm, used to calculate the decomposition matrix L, is a modified version of Gaussian elimination.

The recursive algorithm starts with i := 1 and

A⁽¹⁾ := A.

At step i, the matrix A⁽ⁱ⁾ has the following form: $${\displaystyle {\mathbf{A}}^{(i)}=\left(\begin{array}{ccc}{\mathbf{I}}_{i-1}& 0& 0\\ 0& a_{i,i}& {\mathbf{b}}_i^{\ast }\\ 0& {\mathbf{b}}_i& {\mathbf{B}}^{(i)}\end{array}\right),}$$ where I_i−1 denotes the identity matrix of dimension i − 1.

If the matrix L_i is defined by $${\displaystyle {\mathbf{L}}_i:=\left(\begin{array}{ccc}{\mathbf{I}}_{i-1}& 0& 0\\ 0& \sqrt{a_{i,i}}& 0\\ 0& \frac{1}{\sqrt{a_{i,i}}}{\mathbf{b}}_i& {\mathbf{I}}_{n-i}\end{array}\right),}$$ (note that a_i,i > 0 since A⁽ⁱ⁾ is positive definite), then A⁽ⁱ⁾ can be written as $${\displaystyle {\mathbf{A}}^{(i)}={\mathbf{L}}_i{\mathbf{A}}^{(i+1)}{\mathbf{L}}_i^{\ast }}$$ where $${\displaystyle {\mathbf{A}}^{(i+1)}=\left(\begin{array}{ccc}{\mathbf{I}}_{i-1}& 0& 0\\ 0& 1& 0\\ 0& 0& {\mathbf{B}}^{(i)}-\frac{1}{a_{i,i}}{\mathbf{b}}_i{\mathbf{b}}_i^{\ast }\end{array}\right).}$$ Note that b_i b_i* is an outer product, therefore this algorithm is called the outer-product version in (Golub & Van Loan).

This is repeated for i from 1 to n. After n steps, A⁽ⁿ⁺¹⁾ = I is obtained, and hence, the lower triangular matrix L sought for is calculated as

$${\displaystyle \mathbf{L}:={\mathbf{L}}_1{\mathbf{L}}_2\dots {\mathbf{L}}_n.}$$

The Cholesky–Banachiewicz and Cholesky–Crout algorithms

If the equation $${\displaystyle \begin{array}{cc}\mathbf{A}={\mathbf{L}\mathbf{L}}^T& =\left(\begin{array}{ccc}{L}_{11}& 0& 0\\ L_{21}& L_{22}& 0\\ L_{31}& L_{32}& L_{33}\end{array}\right)\left(\begin{array}{ccc}{L}_{11}& L_{21}& L_{31}\\ 0& L_{22}& L_{32}\\ 0& 0& L_{33}\end{array}\right)\\ & =\left(\begin{array}{ccc}{L}_{11}^2& & (\text{symmetric})\\ L_{21}{L}_{11}& L_{21}^2+L_{22}^2& \\ L_{31}{L}_{11}& L_{31}{L}_{21}+L_{32}{L}_{22}& L_{31}^2+L_{32}^2+L_{33}^2\end{array}\right),\end{array}}$$

is written out, the following is obtained:

$${\displaystyle \begin{array}{c}\mathbf{L}=\left(\begin{array}{ccc}\sqrt{A_{11}}& 0& 0\\ A_{21}/L_{11}& \sqrt{A_{22}-L_{21}^2}& 0\\ A_{31}/L_{11}& \left(A_{32}-L_{31}{L}_{21}\right)/L_{22}& \sqrt{A_{33}-L_{31}^2-L_{32}^2}\end{array}\right)\end{array}}$$

and therefore the following formulas for the entries of L:

$${\displaystyle L_{j,j}=(\pm )\sqrt{A_{j,j}-\sum _{k=1}^{j-1}{L}_{j,k}^2},}$$ $${\displaystyle L_{i,j}=\frac{1}{L_{j,j}}\left(A_{i,j}-\sum _{k=1}^{j-1}{L}_{i,k}{L}_{j,k}\right)\text{for }i>j.}$$

For complex and real matrices, inconsequential arbitrary sign changes of diagonal and associated off-diagonal elements are allowed. The expression under the square root is always positive if A is real and positive-definite.

For complex Hermitian matrix, the following formula applies:

$${\displaystyle L_{j,j}=\sqrt{A_{j,j}-\sum _{k=1}^{j-1}{L}_{j,k}^{\ast }{L}_{j,k}},}$$ $${\displaystyle L_{i,j}=\frac{1}{L_{j,j}}\left(A_{i,j}-\sum _{k=1}^{j-1}{L}_{j,k}^{\ast }{L}_{i,k}\right)\text{for }i>j.}$$

and it can be shown that ${\displaystyle L_{j,j}}$ is always real and positive if A is positive-definite.

So it now is possible to compute the (i, j) entry if the entries to the left and above are known. The computation is usually arranged in either of the following orders:

• The Cholesky–Banachiewicz algorithm starts from the upper left corner of the matrix L and proceeds to calculate the matrix row by row.

<span id="mwAdg"></span><span class="k" id="mwAdk">for</span><span class="w" id="mwAdo"> </span><span class="p" id="mwAds">(</span><span class="n" id="mwAdw">i</span><span class="w" id="mwAd0"> </span><span class="o" id="mwAd4">=</span><span class="w" id="mwAd8"> </span><span class="mi" id="mwAeA">0</span><span class="p" id="mwAeE">;</span><span class="w" id="mwAeI"> </span><span class="n" id="mwAeM">i</span><span class="w" id="mwAeQ"> </span><span class="o" id="mwAeU"><</span><span class="w" id="mwAeY"> </span><span class="n" id="mwAec">dimensionSize</span><span class="p" id="mwAeg">;</span><span class="w" id="mwAek"> </span><span class="n" id="mwAeo">i</span><span class="o" id="mwAes">++</span><span class="p" id="mwAew">)</span><span class="w" id="mwAe0"> </span><span class="p" id="mwAe4">{</span> <span class="w" id="mwAe8"> </span><span class="k" id="mwAfA">for</span><span class="w" id="mwAfE"> </span><span class="p" id="mwAfI">(</span><span class="n" id="mwAfM">j</span><span class="w" id="mwAfQ"> </span><span class="o" id="mwAfU">=</span><span class="w" id="mwAfY"> </span><span class="mi" id="mwAfc">0</span><span class="p" id="mwAfg">;</span><span class="w" id="mwAfk"> </span><span class="n" id="mwAfo">j</span><span class="w" id="mwAfs"> </span><span class="o" id="mwAfw"><=</span><span class="w" id="mwAf0"> </span><span class="n" id="mwAf4">i</span><span class="p" id="mwAf8">;</span><span class="w" id="mwAgA"> </span><span class="n" id="mwAgE">j</span><span class="o" id="mwAgI">++</span><span class="p" id="mwAgM">)</span><span class="w" id="mwAgQ"> </span><span class="p" id="mwAgU">{</span> <span class="w" id="mwAgY"> </span><span class="kt" id="mwAgc">float</span><span class="w" id="mwAgg"> </span><span class="n" id="mwAgk">sum</span><span class="w" id="mwAgo"> </span><span class="o" id="mwAgs">=</span><span class="w" id="mwAgw"> </span><span class="mi" id="mwAg0">0</span><span class="p" id="mwAg4">;</span> <span class="w" id="mwAg8"> </span><span class="k" id="mwAhA">for</span><span class="w" id="mwAhE"> </span><span class="p" id="mwAhI">(</span><span class="n" id="mwAhM">k</span><span class="w" id="mwAhQ"> </span><span class="o" id="mwAhU">=</span><span class="w" id="mwAhY"> </span><span class="mi" id="mwAhc">0</span><span class="p" id="mwAhg">;</span><span class="w" id="mwAhk"> </span><span class="n" id="mwAho">k</span><span class="w" id="mwAhs"> </span><span class="o" id="mwAhw"><</span><span class="w" id="mwAh0"> </span><span class="n" id="mwAh4">j</span><span class="p" id="mwAh8">;</span><span class="w" id="mwAiA"> </span><span class="n" id="mwAiE">k</span><span class="o" id="mwAiI">++</span><span class="p" id="mwAiM">)</span> <span class="w" id="mwAiQ"> </span><span class="n" id="mwAiU">sum</span><span class="w" id="mwAiY"> </span><span class="o" id="mwAic">+=</span><span class="w" id="mwAig"> </span><span class="n" id="mwAik">L</span><span class="p" id="mwAio">[</span><span class="n" id="mwAis">i</span><span class="p" id="mwAiw">][</span><span class="n" id="mwAi0">k</span><span class="p" id="mwAi4">]</span><span class="w" id="mwAi8"> </span><span class="o" id="mwAjA">*</span><span class="w" id="mwAjE"> </span><span class="n" id="mwAjI">L</span><span class="p" id="mwAjM">[</span><span class="n" id="mwAjQ">j</span><span class="p" id="mwAjU">][</span><span class="n" id="mwAjY">k</span><span class="p" id="mwAjc">];</span> <span class="w" id="mwAjg"> </span><span class="k" id="mwAjk">if</span><span class="w" id="mwAjo"> </span><span class="p" id="mwAjs">(</span><span class="n" id="mwAjw">i</span><span class="w" id="mwAj0"> </span><span class="o" id="mwAj4">==</span><span class="w" id="mwAj8"> </span><span class="n" id="mwAkA">j</span><span class="p" id="mwAkE">)</span> <span class="w" id="mwAkI"> </span><span class="n" id="mwAkM">L</span><span class="p" id="mwAkQ">[</span><span class="n" id="mwAkU">i</span><span class="p" id="mwAkY">][</span><span class="n" id="mwAkc">j</span><span class="p" id="mwAkg">]</span><span class="w" id="mwAkk"> </span><span class="o" id="mwAko">=</span><span class="w" id="mwAks"> </span><span class="n" id="mwAkw">sqrt</span><span class="p" id="mwAk0">(</span><span class="n" id="mwAk4">A</span><span class="p" id="mwAk8">[</span><span class="n" id="mwAlA">i</span><span class="p" id="mwAlE">][</span><span class="n" id="mwAlI">i</span><span class="p" id="mwAlM">]</span><span class="w" id="mwAlQ"> </span><span class="o" id="mwAlU">-</span><span class="w" id="mwAlY"> </span><span class="n" id="mwAlc">sum</span><span class="p" id="mwAlg">);</span> <span class="w" id="mwAlk"> </span><span class="k" id="mwAlo">else</span> <span class="w" id="mwAls"> </span><span class="n" id="mwAlw">L</span><span class="p" id="mwAl0">[</span><span class="n" id="mwAl4">i</span><span class="p" id="mwAl8">][</span><span class="n" id="mwAmA">j</span><span class="p" id="mwAmE">]</span><span class="w" id="mwAmI"> </span><span class="o" id="mwAmM">=</span><span class="w" id="mwAmQ"> </span><span class="p" id="mwAmU">(</span><span class="mf" id="mwAmY">1.0</span><span class="w" id="mwAmc"> </span><span class="o" id="mwAmg">/</span><span class="w" id="mwAmk"> </span><span class="n" id="mwAmo">L</span><span class="p" id="mwAms">[</span><span class="n" id="mwAmw">j</span><span class="p" id="mwAm0">][</span><span class="n" id="mwAm4">j</span><span class="p" id="mwAm8">]</span><span class="w" id="mwAnA"> </span><span class="o" id="mwAnE">*</span><span class="w" id="mwAnI"> </span><span class="p" id="mwAnM">(</span><span class="n" id="mwAnQ">A</span><span class="p" id="mwAnU">[</span><span class="n" id="mwAnY">i</span><span class="p" id="mwAnc">][</span><span class="n" id="mwAng">j</span><span class="p" id="mwAnk">]</span><span class="w" id="mwAno"> </span><span class="o" id="mwAns">-</span><span class="w" id="mwAnw"> </span><span class="n" id="mwAn0">sum</span><span class="p" id="mwAn4">));</span> <span class="w" id="mwAn8"> </span><span class="p" id="mwAoA">}</span> <span class="p" id="mwAoE">}</span>

The above algorithm can be succinctly expressed as combining a dot product and matrix multiplication in vectorized programming languages such as Fortran as the following,

<span id="mwAog"></span><span class="k" id="mwAok">do </span><span class="n" id="mwAoo">i</span><span class="w" id="mwAos"> </span><span class="o" id="mwAow">=</span><span class="w" id="mwAo0"> </span><span class="mi" id="mwAo4">1</span><span class="p" id="mwAo8">,</span><span class="w" id="mwApA"> </span><span class="n" id="mwApE">size</span><span class="p" id="mwApI">(</span><span class="n" id="mwApM">A</span><span class="p" id="mwApQ">,</span><span class="mi" id="mwApU">1</span><span class="p" id="mwApY">)</span> <span class="w" id="mwApc"> </span><span class="n" id="mwApg">L</span><span class="p" id="mwApk">(</span><span class="n" id="mwApo">i</span><span class="p" id="mwAps">,</span><span class="n" id="mwApw">i</span><span class="p" id="mwAp0">)</span><span class="w" id="mwAp4"> </span><span class="o" id="mwAp8">=</span><span class="w" id="mwAqA"> </span><span class="nb" id="mwAqE">sqrt</span><span class="p" id="mwAqI">(</span><span class="n" id="mwAqM">A</span><span class="p" id="mwAqQ">(</span><span class="n" id="mwAqU">i</span><span class="p" id="mwAqY">,</span><span class="n" id="mwAqc">i</span><span class="p" id="mwAqg">)</span><span class="w" id="mwAqk"> </span><span class="o" id="mwAqo">-</span><span class="w" id="mwAqs"> </span><span class="nb" id="mwAqw">dot_product</span><span class="p" id="mwAq0">(</span><span class="n" id="mwAq4">L</span><span class="p" id="mwAq8">(</span><span class="n" id="mwArA">i</span><span class="p" id="mwArE">,</span><span class="mi" id="mwArI">1</span><span class="p" id="mwArM">:</span><span class="n" id="mwArQ">i</span><span class="o" id="mwArU">-</span><span class="mi" id="mwArY">1</span><span class="p" id="mwArc">),</span><span class="w" id="mwArg"> </span><span class="n" id="mwArk">L</span><span class="p" id="mwAro">(</span><span class="n" id="mwArs">i</span><span class="p" id="mwArw">,</span><span class="mi" id="mwAr0">1</span><span class="p" id="mwAr4">:</span><span class="n" id="mwAr8">i</span><span class="o" id="mwAsA">-</span><span class="mi" id="mwAsE">1</span><span class="p" id="mwAsI">)))</span> <span class="w" id="mwAsM"> </span><span class="n" id="mwAsQ">L</span><span class="p" id="mwAsU">(</span><span class="n" id="mwAsY">i</span><span class="o" id="mwAsc">+</span><span class="mi" id="mwAsg">1</span><span class="p" id="mwAsk">:,</span><span class="n" id="mwAso">i</span><span class="p" id="mwAss">)</span><span class="w" id="mwAsw"> </span><span class="o" id="mwAs0">=</span><span class="w" id="mwAs4"> </span><span class="p" id="mwAs8">(</span><span class="n" id="mwAtA">A</span><span class="p" id="mwAtE">(</span><span class="n" id="mwAtI">i</span><span class="o" id="mwAtM">+</span><span class="mi" id="mwAtQ">1</span><span class="p" id="mwAtU">:,</span><span class="n" id="mwAtY">i</span><span class="p" id="mwAtc">)</span><span class="w" id="mwAtg"> </span><span class="o" id="mwAtk">-</span><span class="w" id="mwAto"> </span><span class="nb" id="mwAts">matmul</span><span class="p" id="mwAtw">(</span><span class="nb" id="mwAt0">conjg</span><span class="p" id="mwAt4">(</span><span class="n" id="mwAt8">L</span><span class="p" id="mwAuA">(</span><span class="n" id="mwAuE">i</span><span class="p" id="mwAuI">,</span><span class="mi" id="mwAuM">1</span><span class="p" id="mwAuQ">:</span><span class="n" id="mwAuU">i</span><span class="o" id="mwAuY">-</span><span class="mi" id="mwAuc">1</span><span class="p" id="mwAug">)),</span><span class="w" id="mwAuk"> </span><span class="n" id="mwAuo">L</span><span class="p" id="mwAus">(</span><span class="n" id="mwAuw">i</span><span class="o" id="mwAu0">+</span><span class="mi" id="mwAu4">1</span><span class="p" id="mwAu8">:,</span><span class="mi" id="mwAvA">1</span><span class="p" id="mwAvE">:</span><span class="n" id="mwAvI">i</span><span class="o" id="mwAvM">-</span><span class="mi" id="mwAvQ">1</span><span class="p" id="mwAvU">)))</span><span class="w" id="mwAvY"> </span><span class="o" id="mwAvc">/</span><span class="w" id="mwAvg"> </span><span class="n" id="mwAvk">L</span><span class="p" id="mwAvo">(</span><span class="n" id="mwAvs">i</span><span class="p" id="mwAvw">,</span><span class="n" id="mwAv0">i</span><span class="p" id="mwAv4">)</span> <span class="k" id="mwAv8">end do</span>

where conjg refers to complex conjugate of the elements.

• The Cholesky–Crout algorithm starts from the upper left corner of the matrix L and proceeds to calculate the matrix column by column.

  <span id="mwAwg"></span><span class="k" id="mwAwk">for</span><span class="w" id="mwAwo"> </span><span class="p" id="mwAws">(</span><span class="n" id="mwAww">j</span><span class="w" id="mwAw0"> </span><span class="o" id="mwAw4">=</span><span class="w" id="mwAw8"> </span><span class="mi" id="mwAxA">0</span><span class="p" id="mwAxE">;</span><span class="w" id="mwAxI"> </span><span class="n" id="mwAxM">j</span><span class="w" id="mwAxQ"> </span><span class="o" id="mwAxU"><</span><span class="w" id="mwAxY"> </span><span class="n" id="mwAxc">dimensionSize</span><span class="p" id="mwAxg">;</span><span class="w" id="mwAxk"> </span><span class="n" id="mwAxo">j</span><span class="o" id="mwAxs">++</span><span class="p" id="mwAxw">)</span><span class="w" id="mwAx0"> </span><span class="p" id="mwAx4">{</span> <span class="w" id="mwAx8"> </span><span class="kt" id="mwAyA">float</span><span class="w" id="mwAyE"> </span><span class="n" id="mwAyI">sum</span><span class="w" id="mwAyM"> </span><span class="o" id="mwAyQ">=</span><span class="w" id="mwAyU"> </span><span class="mi" id="mwAyY">0</span><span class="p" id="mwAyc">;</span> <span class="w" id="mwAyg"> </span><span class="k" id="mwAyk">for</span><span class="w" id="mwAyo"> </span><span class="p" id="mwAys">(</span><span class="n" id="mwAyw">k</span><span class="w" id="mwAy0"> </span><span class="o" id="mwAy4">=</span><span class="w" id="mwAy8"> </span><span class="mi" id="mwAzA">0</span><span class="p" id="mwAzE">;</span><span class="w" id="mwAzI"> </span><span class="n" id="mwAzM">k</span><span class="w" id="mwAzQ"> </span><span class="o" id="mwAzU"><</span><span class="w" id="mwAzY"> </span><span class="n" id="mwAzc">j</span><span class="p" id="mwAzg">;</span><span class="w" id="mwAzk"> </span><span class="n" id="mwAzo">k</span><span class="o" id="mwAzs">++</span><span class="p" id="mwAzw">)</span><span class="w" id="mwAz0"> </span><span class="p" id="mwAz4">{</span> <span class="w" id="mwAz8"> </span><span class="n" id="mwA0A">sum</span><span class="w" id="mwA0E"> </span><span class="o" id="mwA0I">+=</span><span class="w" id="mwA0M"> </span><span class="n" id="mwA0Q">L</span><span class="p" id="mwA0U">[</span><span class="n" id="mwA0Y">j</span><span class="p" id="mwA0c">][</span><span class="n" id="mwA0g">k</span><span class="p" id="mwA0k">]</span><span class="w" id="mwA0o"> </span><span class="o" id="mwA0s">*</span><span class="w" id="mwA0w"> </span><span class="n" id="mwA00">L</span><span class="p" id="mwA04">[</span><span class="n" id="mwA08">j</span><span class="p" id="mwA1A">][</span><span class="n" id="mwA1E">k</span><span class="p" id="mwA1I">];</span> <span class="w" id="mwA1M"> </span><span class="p" id="mwA1Q">}</span> <span class="w" id="mwA1U"> </span><span class="n" id="mwA1Y">L</span><span class="p" id="mwA1c">[</span><span class="n" id="mwA1g">j</span><span class="p" id="mwA1k">][</span><span class="n" id="mwA1o">j</span><span class="p" id="mwA1s">]</span><span class="w" id="mwA1w"> </span><span class="o" id="mwA10">=</span><span class="w" id="mwA14"> </span><span class="n" id="mwA18">sqrt</span><span class="p" id="mwA2A">(</span><span class="n" id="mwA2E">A</span><span class="p" id="mwA2I">[</span><span class="n" id="mwA2M">j</span><span class="p" id="mwA2Q">][</span><span class="n" id="mwA2U">j</span><span class="p" id="mwA2Y">]</span><span class="w" id="mwA2c"> </span><span class="o" id="mwA2g">-</span><span class="w" id="mwA2k"> </span><span class="n" id="mwA2o">sum</span><span class="p" id="mwA2s">);</span> <span class="w" id="mwA2w"> </span><span class="k" id="mwA20">for</span><span class="w" id="mwA24"> </span><span class="p" id="mwA28">(</span><span class="n" id="mwA3A">i</span><span class="w" id="mwA3E"> </span><span class="o" id="mwA3I">=</span><span class="w" id="mwA3M"> </span><span class="n" id="mwA3Q">j</span><span class="w" id="mwA3U"> </span><span class="o" id="mwA3Y">+</span><span class="w" id="mwA3c"> </span><span class="mi" id="mwA3g">1</span><span class="p" id="mwA3k">;</span><span class="w" id="mwA3o"> </span><span class="n" id="mwA3s">i</span><span class="w" id="mwA3w"> </span><span class="o" id="mwA30"><</span><span class="w" id="mwA34"> </span><span class="n" id="mwA38">dimensionSize</span><span class="p" id="mwA4A">;</span><span class="w" id="mwA4E"> </span><span class="n" id="mwA4I">i</span><span class="o" id="mwA4M">++</span><span class="p" id="mwA4Q">)</span><span class="w" id="mwA4U"> </span><span class="p" id="mwA4Y">{</span> <span class="w" id="mwA4c"> </span><span class="n" id="mwA4g">sum</span><span class="w" id="mwA4k"> </span><span class="o" id="mwA4o">=</span><span class="w" id="mwA4s"> </span><span class="mi" id="mwA4w">0</span><span class="p" id="mwA40">;</span> <span class="w" id="mwA44"> </span><span class="k" id="mwA48">for</span><span class="w" id="mwA5A"> </span><span class="p" id="mwA5E">(</span><span class="n" id="mwA5I">k</span><span class="w" id="mwA5M"> </span><span class="o" id="mwA5Q">=</span><span class="w" id="mwA5U"> </span><span class="mi" id="mwA5Y">0</span><span class="p" id="mwA5c">;</span><span class="w" id="mwA5g"> </span><span class="n" id="mwA5k">k</span><span class="w" id="mwA5o"> </span><span class="o" id="mwA5s"><</span><span class="w" id="mwA5w"> </span><span class="n" id="mwA50">j</span><span class="p" id="mwA54">;</span><span class="w" id="mwA58"> </span><span class="n" id="mwA6A">k</span><span class="o" id="mwA6E">++</span><span class="p" id="mwA6I">)</span><span class="w" id="mwA6M"> </span><span class="p" id="mwA6Q">{</span> <span class="w" id="mwA6U"> </span><span class="n" id="mwA6Y">sum</span><span class="w" id="mwA6c"> </span><span class="o" id="mwA6g">+=</span><span class="w" id="mwA6k"> </span><span class="n" id="mwA6o">L</span><span class="p" id="mwA6s">[</span><span class="n" id="mwA6w">i</span><span class="p" id="mwA60">][</span><span class="n" id="mwA64">k</span><span class="p" id="mwA68">]</span><span class="w" id="mwA7A"> </span><span class="o" id="mwA7E">*</span><span class="w" id="mwA7I"> </span><span class="n" id="mwA7M">L</span><span class="p" id="mwA7Q">[</span><span class="n" id="mwA7U">j</span><span class="p" id="mwA7Y">][</span><span class="n" id="mwA7c">k</span><span class="p" id="mwA7g">];</span> <span class="w" id="mwA7k"> </span><span class="p" id="mwA7o">}</span> <span class="w" id="mwA7s"> </span><span class="n" id="mwA7w">L</span><span class="p" id="mwA70">[</span><span class="n" id="mwA74">i</span><span class="p" id="mwA78">][</span><span class="n" id="mwA8A">j</span><span class="p" id="mwA8E">]</span><span class="w" id="mwA8I"> </span><span class="o" id="mwA8M">=</span><span class="w" id="mwA8Q"> </span><span class="p" id="mwA8U">(</span><span class="mf" id="mwA8Y">1.0</span><span class="w" id="mwA8c"> </span><span class="o" id="mwA8g">/</span><span class="w" id="mwA8k"> </span><span class="n" id="mwA8o">L</span><span class="p" id="mwA8s">[</span><span class="n" id="mwA8w">j</span><span class="p" id="mwA80">][</span><span class="n" id="mwA84">j</span><span class="p" id="mwA88">]</span><span class="w" id="mwA9A"> </span><span class="o" id="mwA9E">*</span><span class="w" id="mwA9I"> </span><span class="p" id="mwA9M">(</span><span class="n" id="mwA9Q">A</span><span class="p" id="mwA9U">[</span><span class="n" id="mwA9Y">i</span><span class="p" id="mwA9c">][</span><span class="n" id="mwA9g">j</span><span class="p" id="mwA9k">]</span><span class="w" id="mwA9o"> </span><span class="o" id="mwA9s">-</span><span class="w" id="mwA9w"> </span><span class="n" id="mwA90">sum</span><span class="p" id="mwA94">));</span> <span class="w" id="mwA98"> </span><span class="p" id="mwA-A">}</span> <span class="p" id="mwA-E">}</span>

The above algorithm can be succinctly expressed as combining a dot product and matrix multiplication in vectorized programming languages such as Fortran as the following,

<span id="mwA-g"></span><span class="k" id="mwA-k">do </span><span class="n" id="mwA-o">i</span><span class="w" id="mwA-s"> </span><span class="o" id="mwA-w">=</span><span class="w" id="mwA-0"> </span><span class="mi" id="mwA-4">1</span><span class="p" id="mwA-8">,</span><span class="w" id="mwA_A"> </span><span class="n" id="mwA_E">size</span><span class="p" id="mwA_I">(</span><span class="n" id="mwA_M">A</span><span class="p" id="mwA_Q">,</span><span class="mi" id="mwA_U">1</span><span class="p" id="mwA_Y">)</span> <span class="w" id="mwA_c"> </span><span class="n" id="mwA_g">L</span><span class="p" id="mwA_k">(</span><span class="n" id="mwA_o">i</span><span class="p" id="mwA_s">,</span><span class="n" id="mwA_w">i</span><span class="p" id="mwA_0">)</span><span class="w" id="mwA_4"> </span><span class="o" id="mwA_8">=</span><span class="w" id="mwBAA"> </span><span class="nb" id="mwBAE">sqrt</span><span class="p" id="mwBAI">(</span><span class="n" id="mwBAM">A</span><span class="p" id="mwBAQ">(</span><span class="n" id="mwBAU">i</span><span class="p" id="mwBAY">,</span><span class="n" id="mwBAc">i</span><span class="p" id="mwBAg">)</span><span class="w" id="mwBAk"> </span><span class="o" id="mwBAo">-</span><span class="w" id="mwBAs"> </span><span class="nb" id="mwBAw">dot_product</span><span class="p" id="mwBA0">(</span><span class="n" id="mwBA4">L</span><span class="p" id="mwBA8">(</span><span class="mi" id="mwBBA">1</span><span class="p" id="mwBBE">:</span><span class="n" id="mwBBI">i</span><span class="o" id="mwBBM">-</span><span class="mi" id="mwBBQ">1</span><span class="p" id="mwBBU">,</span><span class="n" id="mwBBY">i</span><span class="p" id="mwBBc">),</span><span class="w" id="mwBBg"> </span><span class="n" id="mwBBk">L</span><span class="p" id="mwBBo">(</span><span class="mi" id="mwBBs">1</span><span class="p" id="mwBBw">:</span><span class="n" id="mwBB0">i</span><span class="o" id="mwBB4">-</span><span class="mi" id="mwBB8">1</span><span class="p" id="mwBCA">,</span><span class="n" id="mwBCE">i</span><span class="p" id="mwBCI">)))</span> <span class="w" id="mwBCM"> </span><span class="n" id="mwBCQ">L</span><span class="p" id="mwBCU">(</span><span class="n" id="mwBCY">i</span><span class="p" id="mwBCc">,</span><span class="n" id="mwBCg">i</span><span class="o" id="mwBCk">+</span><span class="mi" id="mwBCo">1</span><span class="p" id="mwBCs">:)</span><span class="w" id="mwBCw"> </span><span class="o" id="mwBC0">=</span><span class="w" id="mwBC4"> </span><span class="p" id="mwBC8">(</span><span class="n" id="mwBDA">A</span><span class="p" id="mwBDE">(</span><span class="n" id="mwBDI">i</span><span class="p" id="mwBDM">,</span><span class="n" id="mwBDQ">i</span><span class="o" id="mwBDU">+</span><span class="mi" id="mwBDY">1</span><span class="p" id="mwBDc">:)</span><span class="w" id="mwBDg"> </span><span class="o" id="mwBDk">-</span><span class="w" id="mwBDo"> </span><span class="nb" id="mwBDs">matmul</span><span class="p" id="mwBDw">(</span><span class="nb" id="mwBD0">conjg</span><span class="p" id="mwBD4">(</span><span class="n" id="mwBD8">L</span><span class="p" id="mwBEA">(</span><span class="mi" id="mwBEE">1</span><span class="p" id="mwBEI">:</span><span class="n" id="mwBEM">i</span><span class="o" id="mwBEQ">-</span><span class="mi" id="mwBEU">1</span><span class="p" id="mwBEY">,</span><span class="n" id="mwBEc">i</span><span class="p" id="mwBEg">)),</span><span class="w" id="mwBEk"> </span><span class="n" id="mwBEo">L</span><span class="p" id="mwBEs">(</span><span class="mi" id="mwBEw">1</span><span class="p" id="mwBE0">:</span><span class="n" id="mwBE4">i</span><span class="o" id="mwBE8">-</span><span class="mi" id="mwBFA">1</span><span class="p" id="mwBFE">,</span><span class="n" id="mwBFI">i</span><span class="o" id="mwBFM">+</span><span class="mi" id="mwBFQ">1</span><span class="p" id="mwBFU">:)))</span><span class="w" id="mwBFY"> </span><span class="o" id="mwBFc">/</span><span class="w" id="mwBFg"> </span><span class="n" id="mwBFk">L</span><span class="p" id="mwBFo">(</span><span class="n" id="mwBFs">i</span><span class="p" id="mwBFw">,</span><span class="n" id="mwBF0">i</span><span class="p" id="mwBF4">)</span> <span class="k" id="mwBF8">end do</span>

where conjg refers to complex conjugate of the elements.

Either pattern of access allows the entire computation to be performed in-place if desired.

Stability of the computation

Suppose that there is a desire to solve a well-conditioned system of linear equations. If the LU decomposition is used, then the algorithm is unstable unless some sort of pivoting strategy is used. In the latter case, the error depends on the so-called growth factor of the matrix, which is usually (but not always) small.

Now, suppose that the Cholesky decomposition is applicable. As mentioned above, the algorithm will be twice as fast. Furthermore, no pivoting is necessary, and the error will always be small. Specifically, if Ax = b, and y denotes the computed solution, then y solves the perturbed system (A + E)y = b, where $${\displaystyle \Vert \mathbf{E}{\Vert }_2\le c_n\epsilon \Vert \mathbf{A}{\Vert }_2.}$$ Here ||·||₂ is the matrix 2-norm, c_n is a small constant depending on n, and ε denotes the unit round-off.

One concern with the Cholesky decomposition to be aware of is the use of square roots. If the matrix being factorized is positive definite as required, the numbers under the square roots are always positive in exact arithmetic. Unfortunately, the numbers can become negative because of round-off errors, in which case the algorithm cannot continue. However, this can only happen if the matrix is very ill-conditioned. One way to address this is to add a diagonal correction matrix to the matrix being decomposed in an attempt to promote the positive-definiteness. While this might lessen the accuracy of the decomposition, it can be very favorable for other reasons; for example, when performing Newton's method in optimization, adding a diagonal matrix can improve stability when far from the optimum.

LDL decomposition

An alternative form, eliminating the need to take square roots when A is symmetric, is the symmetric indefinite factorization $${\displaystyle \begin{array}{cc}\mathbf{A}={\mathbf{L}\mathbf{D}\mathbf{L}}^{\mathrm{T}}& =\left(\begin{array}{ccc}1& 0& 0\\ L_{21}& 1& 0\\ L_{31}& L_{32}& 1\end{array}\right)\left(\begin{array}{ccc}{D}_1& 0& 0\\ 0& D_2& 0\\ 0& 0& D_3\end{array}\right)\left(\begin{array}{ccc}1& L_{21}& L_{31}\\ 0& 1& L_{32}\\ 0& 0& 1\end{array}\right)\\ & =\left(\begin{array}{ccc}{D}_1& & (\mathrm{s}\mathrm{y}\mathrm{m}\mathrm{m}\mathrm{e}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{c})\\ L_{21}{D}_1& L_{21}^2{D}_1+D_2& \\ L_{31}{D}_1& L_{31}{L}_{21}{D}_1+L_{32}{D}_2& L_{31}^2{D}_1+L_{32}^2{D}_2+D_3.\end{array}\right).\end{array}}$$

The following recursive relations apply for the entries of D and L: $${\displaystyle D_j=A_{jj}-\sum _{k=1}^{j-1}{L}_{jk}^2{D}_k,}$$ $${\displaystyle L_{ij}=\frac{1}{D_j}\left(A_{ij}-\sum _{k=1}^{j-1}{L}_{ik}{L}_{jk}{D}_k\right)\text{for }i>j.}$$

This works as long as the generated diagonal elements in D stay non-zero. The decomposition is then unique. D and L are real if A is real.

For complex Hermitian matrix A, the following formula applies:

$${\displaystyle D_j=A_{jj}-\sum _{k=1}^{j-1}{L}_{jk}{L}_{jk}^{\ast }{D}_k,}$$ $${\displaystyle L_{ij}=\frac{1}{D_j}\left(A_{ij}-\sum _{k=1}^{j-1}{L}_{ik}{L}_{jk}^{\ast }{D}_k\right)\text{for }i>j.}$$

Again, the pattern of access allows the entire computation to be performed in-place if desired.

Block variant

When used on indefinite matrices, the LDL* factorization is known to be unstable without careful pivoting; specifically, the elements of the factorization can grow arbitrarily. A possible improvement is to perform the factorization on block sub-matrices, commonly 2 × 2:

$${\displaystyle \begin{array}{cc}\mathbf{A}={\mathbf{L}\mathbf{D}\mathbf{L}}^{\mathrm{T}}& =\left(\begin{array}{ccc}\mathbf{I}& 0& 0\\ {\mathbf{L}}_{21}& \mathbf{I}& 0\\ {\mathbf{L}}_{31}& {\mathbf{L}}_{32}& \mathbf{I}\end{array}\right)\left(\begin{array}{ccc}{\mathbf{D}}_1& 0& 0\\ 0& {\mathbf{D}}_2& 0\\ 0& 0& {\mathbf{D}}_3\end{array}\right)\left(\begin{array}{ccc}\mathbf{I}& {\mathbf{L}}_{21}^{\mathrm{T}}& {\mathbf{L}}_{31}^{\mathrm{T}}\\ 0& \mathbf{I}& {\mathbf{L}}_{32}^{\mathrm{T}}\\ 0& 0& \mathbf{I}\end{array}\right)\\ & =\left(\begin{array}{ccc}{\mathbf{D}}_1& & (\mathrm{s}\mathrm{y}\mathrm{m}\mathrm{m}\mathrm{e}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{c})\\ {\mathbf{L}}_{21}{\mathbf{D}}_1& {\mathbf{L}}_{21}{\mathbf{D}}_1{\mathbf{L}}_{21}^{\mathrm{T}}+{\mathbf{D}}_2& \\ {\mathbf{L}}_{31}{\mathbf{D}}_1& {\mathbf{L}}_{31}{\mathbf{D}}_1{\mathbf{L}}_{21}^{\mathrm{T}}+{\mathbf{L}}_{32}{\mathbf{D}}_2& {\mathbf{L}}_{31}{\mathbf{D}}_1{\mathbf{L}}_{31}^{\mathrm{T}}+{\mathbf{L}}_{32}{\mathbf{D}}_2{\mathbf{L}}_{32}^{\mathrm{T}}+{\mathbf{D}}_3\end{array}\right),\end{array}}$$

where every element in the matrices above is a square submatrix. From this, these analogous recursive relations follow:

$${\displaystyle {\mathbf{D}}_j={\mathbf{A}}_{jj}-\sum _{k=1}^{j-1}{\mathbf{L}}_{jk}{\mathbf{D}}_k{\mathbf{L}}_{jk}^{\mathrm{T}},}$$ $${\displaystyle {\mathbf{L}}_{ij}=\left({\mathbf{A}}_{ij}-\sum _{k=1}^{j-1}{\mathbf{L}}_{ik}{\mathbf{D}}_k{\mathbf{L}}_{jk}^{\mathrm{T}}\right){\mathbf{D}}_j^{-1}.}$$

This involves matrix products and explicit inversion, thus limiting the practical block size.

Updating the decomposition

A task that often arises in practice is that one needs to update a Cholesky decomposition. In more details, one has already computed the Cholesky decomposition $\mathbf{A}=\mathbf{L}{\mathbf{L}}^{\ast }$ of some matrix $\mathbf{A}$, then one changes the matrix $\mathbf{A}$ in some way into another matrix, say $\tilde{\mathbf{A}}$, and one wants to compute the Cholesky decomposition of the updated matrix: $\tilde{\mathbf{A}}=\tilde{\mathbf{L}}{\tilde{\mathbf{L}}}^{\ast }$. The question is now whether one can use the Cholesky decomposition of $\mathbf{A}$ that was computed before to compute the Cholesky decomposition of $\tilde{\mathbf{A}}$.

Rank-one update

The specific case, where the updated matrix $\tilde{\mathbf{A}}$ is related to the matrix $\mathbf{A}$ by $\tilde{\mathbf{A}}=\mathbf{A}+c\mathbf{x}{\mathbf{x}}^{\ast }$, is known as a rank-one update. Here the constant ${\displaystyle c}$ is allowed to be negative, but must always be such that the new matrix $\tilde{\mathbf{A}}$ is still positive definite.

Here is a function written in Matlab syntax that realizes a rank-one update:

<span id="mwBMA"></span><span class="k" id="mwBME">function</span><span class="w" id="mwBMI"> </span>L<span class="p" id="mwBMM">=</span><span class="nf" id="mwBMQ">updateChol</span><span class="p" id="mwBMU">(</span>L,x,c<span class="p" id="mwBMY">)</span> <span class="c" id="mwBMc">% given the L*L' Cholesky decomposition of a matrix, compute the updated</span> <span class="c" id="mwBMg">% factor L so that we have the Cholesky decomposition of L*L'+c*x*x';</span> <span class="n" id="mwBMk">n</span><span class="p" id="mwBMo">=</span><span class="nb" id="mwBMs">length</span><span class="p" id="mwBMw">(</span><span class="n" id="mwBM0">x</span><span class="p" id="mwBM4">);</span> <span class="k" id="mwBM8">for</span><span class="w" id="mwBNA"> </span><span class="n" id="mwBNE">k</span><span class="p" id="mwBNI">=</span><span class="mi" id="mwBNM">1</span><span class="p" id="mwBNQ">:</span><span class="n" id="mwBNU">n</span><span class="o" id="mwBNY">-</span><span class="mi" id="mwBNc">1</span> <span class="w" id="mwBNg"> </span><span class="n" id="mwBNk">l</span><span class="p" id="mwBNo">=</span><span class="n" id="mwBNs">L</span><span class="p" id="mwBNw">(:,</span><span class="n" id="mwBN0">k</span><span class="p" id="mwBN4">);</span><span class="w" id="mwBN8"> </span><span class="c" id="mwBOA">% old value of k-th column</span> <span class="w" id="mwBOE"> </span><span class="n" id="mwBOI">lk</span><span class="p" id="mwBOM">=</span><span class="n" id="mwBOQ">l</span><span class="p" id="mwBOU">(</span><span class="n" id="mwBOY">k</span><span class="p" id="mwBOc">);</span> <span class="w" id="mwBOg"> </span><span class="n" id="mwBOk">xk</span><span class="p" id="mwBOo">=</span><span class="n" id="mwBOs">x</span><span class="p" id="mwBOw">(</span><span class="n" id="mwBO0">k</span><span class="p" id="mwBO4">);</span> <span class="w" id="mwBO8"> </span><span class="n" id="mwBPA">dk</span><span class="p" id="mwBPE">=</span><span class="nb" id="mwBPI">sqrt</span><span class="p" id="mwBPM">(</span><span class="n" id="mwBPQ">lk</span><span class="o" id="mwBPU">^</span><span class="mi" id="mwBPY">2</span><span class="o" id="mwBPc">+</span><span class="n" id="mwBPg">c</span><span class="o" id="mwBPk">*</span><span class="n" id="mwBPo">xk</span><span class="o" id="mwBPs">^</span><span class="mi" id="mwBPw">2</span><span class="p" id="mwBP0">);</span><span class="w" id="mwBP4"> </span><span class="c" id="mwBP8">% new diagonal value</span> <span class="w" id="mwBQA"> </span><span class="n" id="mwBQE">L</span><span class="p" id="mwBQI">(:,</span><span class="n" id="mwBQM">k</span><span class="p" id="mwBQQ">)=(</span><span class="n" id="mwBQU">lk</span><span class="o" id="mwBQY">/</span><span class="n" id="mwBQc">dk</span><span class="p" id="mwBQg">)</span><span class="o" id="mwBQk">*</span><span class="n" id="mwBQo">l</span><span class="o" id="mwBQs">+</span><span class="p" id="mwBQw">(</span><span class="n" id="mwBQ0">c</span><span class="o" id="mwBQ4">*</span><span class="n" id="mwBQ8">xk</span><span class="o" id="mwBRA">/</span><span class="n" id="mwBRE">dk</span><span class="p" id="mwBRI">)</span><span class="o" id="mwBRM">*</span><span class="n" id="mwBRQ">x</span><span class="p" id="mwBRU">;</span><span class="w" id="mwBRY"> </span><span class="c" id="mwBRc">% new column value</span> <span class="w" id="mwBRg"> </span><span class="n" id="mwBRk">x</span><span class="p" id="mwBRo">=</span><span class="n" id="mwBRs">x</span><span class="o" id="mwBRw">-</span><span class="n" id="mwBR0">l</span><span class="o" id="mwBR4">*</span><span class="p" id="mwBR8">(</span><span class="n" id="mwBSA">xk</span><span class="o" id="mwBSE">/</span><span class="n" id="mwBSI">lk</span><span class="p" id="mwBSM">);</span> <span class="w" id="mwBSQ"> </span><span class="n" id="mwBSU">c</span><span class="p" id="mwBSY">=</span><span class="n" id="mwBSc">c</span><span class="o" id="mwBSg">*</span><span class="p" id="mwBSk">(</span><span class="n" id="mwBSo">lk</span><span class="o" id="mwBSs">/</span><span class="n" id="mwBSw">dk</span><span class="p" id="mwBS0">)</span><span class="o" id="mwBS4">^</span><span class="mi" id="mwBS8">2</span><span class="p" id="mwBTA">;</span> <span class="k" id="mwBTE">end</span> <span class="n" id="mwBTI">L</span><span class="p" id="mwBTM">(</span><span class="n" id="mwBTQ">n</span><span class="p" id="mwBTU">,</span><span class="n" id="mwBTY">n</span><span class="p" id="mwBTc">)=</span><span class="nb" id="mwBTg">sqrt</span><span class="p" id="mwBTk">(</span><span class="n" id="mwBTo">L</span><span class="p" id="mwBTs">(</span><span class="n" id="mwBTw">n</span><span class="p" id="mwBT0">,</span><span class="n" id="mwBT4">n</span><span class="p" id="mwBT8">)</span><span class="o" id="mwBUA">^</span><span class="mi" id="mwBUE">2</span><span class="o" id="mwBUI">+</span><span class="n" id="mwBUM">c</span><span class="o" id="mwBUQ">*</span><span class="n" id="mwBUU">x</span><span class="p" id="mwBUY">(</span><span class="n" id="mwBUc">n</span><span class="p" id="mwBUg">)</span><span class="o" id="mwBUk">^</span><span class="mi" id="mwBUo">2</span><span class="p" id="mwBUs">);</span> <span class="k" id="mwBUw">end</span>

A rank-n update is one where for a matrix $\mathbf{M}$ one updates the decomposition such that $\tilde{\mathbf{A}}=\mathbf{A}+\mathbf{M}{\mathbf{M}}^{\ast }$. This can be achieved by successively performing rank-one updates for each of the columns of $\mathbf{M}$.

Adding and removing rows and columns

If a symmetric and positive definite matrix $\mathbf{A}$ is represented in block form as

$${\displaystyle \mathbf{A}=\left(\begin{array}{cc}{\mathbf{A}}_{11}& {\mathbf{A}}_{13}\\ {\mathbf{A}}_{13}^{\mathrm{T}}& {\mathbf{A}}_{33}\end{array}\right)}$$

and its upper Cholesky factor $${\displaystyle \mathbf{L}=\left(\begin{array}{cc}{\mathbf{L}}_{11}& {\mathbf{L}}_{13}\\ 0& {\mathbf{L}}_{33}\end{array}\right),}$$

then for a new matrix $\tilde{\mathbf{A}}$, which is the same as $\mathbf{A}$ but with the insertion of new rows and columns, $${\displaystyle \begin{array}{cc}\tilde{\mathbf{A}}& =\left(\begin{array}{ccc}{\mathbf{A}}_{11}& {\mathbf{A}}_{12}& {\mathbf{A}}_{13}\\ {\mathbf{A}}_{12}^{\mathrm{T}}& {\mathbf{A}}_{22}& {\mathbf{A}}_{23}\\ {\mathbf{A}}_{13}^{\mathrm{T}}& {\mathbf{A}}_{23}^{\mathrm{T}}& {\mathbf{A}}_{33}\end{array}\right)\end{array}}$$

Now there is an interest in finding the Cholesky factorization of $\tilde{\mathbf{A}}$, which can be called $\tilde{\mathbf{S}}$, without directly computing the entire decomposition. $${\displaystyle \begin{array}{cc}\tilde{\mathbf{S}}& =\left(\begin{array}{ccc}{\mathbf{S}}_{11}& {\mathbf{S}}_{12}& {\mathbf{S}}_{13}\\ 0& {\mathbf{S}}_{22}& {\mathbf{S}}_{23}\\ 0& 0& {\mathbf{S}}_{33}\end{array}\right).\end{array}}$$

Writing $\mathbf{A}\setminus \mathbf{b}$ for the solution of $\mathbf{A}\mathbf{x}=\mathbf{b}$, which can be found easily for triangular matrices, and $\text{chol}(\mathbf{M})$ for the Cholesky decomposition of $\mathbf{M}$, the following relations can be found: $${\displaystyle \begin{array}{cc}{\mathbf{S}}_{11}& ={\mathbf{L}}_{11},\\ {\mathbf{S}}_{12}& ={\mathbf{L}}_{11}^{\mathrm{T}}\setminus {\mathbf{A}}_{12},\\ {\mathbf{S}}_{13}& ={\mathbf{L}}_{13},\\ {\mathbf{S}}_{22}& =\mathrm{c}\mathrm{h}\mathrm{o}\mathrm{l}\left({\mathbf{A}}_{22}-{\mathbf{S}}_{12}^{\mathrm{T}}{\mathbf{S}}_{12}\right),\\ {\mathbf{S}}_{23}& ={\mathbf{S}}_{22}^{\mathrm{T}}\setminus \left({\mathbf{A}}_{23}-{\mathbf{S}}_{12}^{\mathrm{T}}{\mathbf{S}}_{13}\right),\\ {\mathbf{S}}_{33}& =\mathrm{c}\mathrm{h}\mathrm{o}\mathrm{l}\left({\mathbf{L}}_{33}^{\mathrm{T}}{\mathbf{L}}_{33}-{\mathbf{S}}_{23}^{\mathrm{T}}{\mathbf{S}}_{23}\right).\end{array}}$$

These formulas may be used to determine the Cholesky factor after the insertion of rows or columns in any position, if the row and column dimensions are appropriately set (including to zero). The inverse problem,

$${\displaystyle \begin{array}{cc}\tilde{\mathbf{A}}& =\left(\begin{array}{ccc}{\mathbf{A}}_{11}& {\mathbf{A}}_{12}& {\mathbf{A}}_{13}\\ {\mathbf{A}}_{12}^{\mathrm{T}}& {\mathbf{A}}_{22}& {\mathbf{A}}_{23}\\ {\mathbf{A}}_{13}^{\mathrm{T}}& {\mathbf{A}}_{23}^{\mathrm{T}}& {\mathbf{A}}_{33}\end{array}\right)\end{array}}$$ with known Cholesky decomposition $${\displaystyle \begin{array}{cc}\tilde{\mathbf{S}}& =\left(\begin{array}{ccc}{\mathbf{S}}_{11}& {\mathbf{S}}_{12}& {\mathbf{S}}_{13}\\ 0& {\mathbf{S}}_{22}& {\mathbf{S}}_{23}\\ 0& 0& {\mathbf{S}}_{33}\end{array}\right)\end{array}}$$

and the desire to determine the Cholesky factor $${\displaystyle \begin{array}{cc}\mathbf{L}& =\left(\begin{array}{cc}{\mathbf{L}}_{11}& {\mathbf{L}}_{13}\\ 0& {\mathbf{L}}_{33}\end{array}\right)\end{array}}$$

of the matrix $\mathbf{A}$ with rows and columns removed, $${\displaystyle \begin{array}{cc}\mathbf{A}& =\left(\begin{array}{cc}{\mathbf{A}}_{11}& {\mathbf{A}}_{13}\\ {\mathbf{A}}_{13}^{\mathrm{T}}& {\mathbf{A}}_{33}\end{array}\right),\end{array}}$$

yields the following rules: $${\displaystyle \begin{array}{cc}{\mathbf{L}}_{11}& ={\mathbf{S}}_{11},\\ {\mathbf{L}}_{13}& ={\mathbf{S}}_{13},\\ {\mathbf{L}}_{33}& =\mathrm{c}\mathrm{h}\mathrm{o}\mathrm{l}\left({\mathbf{S}}_{33}^{\mathrm{T}}{\mathbf{S}}_{33}+{\mathbf{S}}_{23}^{\mathrm{T}}{\mathbf{S}}_{23}\right).\end{array}}$$

Notice that the equations above that involve finding the Cholesky decomposition of a new matrix are all of the form $\tilde{\mathbf{A}}=\mathbf{A}+c\mathbf{x}{\mathbf{x}}^{\ast }$ for some constant ${\displaystyle c=\pm 1}$, which allows them to be efficiently calculated using procedure detailed in the previous section.

Proof for positive semi-definite matrices

Proof by limiting argument

The above algorithms show that every positive definite matrix $\mathbf{A}$ has a Cholesky decomposition. This result can be extended to the positive semi-definite case by a limiting argument. The argument is not fully constructive, i.e., it gives no explicit numerical algorithms for computing Cholesky factors.

If $\mathbf{A}$ is an $n\times n$ positive semi-definite matrix, then the sequence ${\left({\mathbf{A}}_k\right)}_k:={\left(\mathbf{A}+\frac{1}{k}{\mathbf{I}}_n\right)}_k$ consists of positive definite matrices. (This is an immediate consequence of, for example, the spectral mapping theorem for the polynomial functional calculus.) Also, $${\displaystyle {\mathbf{A}}_k\to \mathbf{A}\text{for}k\to \mathrm{\infty }}$$ in operator norm. From the positive definite case, each ${\mathbf{A}}_k$ has Cholesky decomposition ${\mathbf{A}}_k={\mathbf{L}}_k{\mathbf{L}}_k^{\ast }$. By property of the operator norm,

$${\displaystyle \Vert {\mathbf{L}}_k{\Vert }^2\le \Vert {\mathbf{L}}_k{\mathbf{L}}_k^{\ast }\Vert =\Vert {\mathbf{A}}_k\Vert .}$$

The $\le$ holds because $M_n(\mathbb{C})$ equipped with the operator norm is a C* algebra. So ${\left({\mathbf{L}}_k\right)}_k$ is a bounded set in the Banach space of operators, therefore relatively compact (because the underlying vector space is finite-dimensional). Consequently, it has a convergent subsequence, also denoted by ${\left({\mathbf{L}}_k\right)}_k$, with limit $\mathbf{L}$. It can be easily checked that this $\mathbf{L}$ has the desired properties, i.e. $\mathbf{A}=\mathbf{L}{\mathbf{L}}^{\ast }$, and $\mathbf{L}$ is lower triangular with non-negative diagonal entries: for all $x$ and $y$,

$${\displaystyle ⟨\mathbf{A}x,y⟩=⟨lim{\mathbf{A}}_{k}x,y⟩=⟨lim{\mathbf{L}}_k{\mathbf{L}}_k^{\ast }x,y⟩=⟨\mathbf{L}{\mathbf{L}}^{\ast }x,y⟩.}$$

Therefore, $\mathbf{A}=\mathbf{L}{\mathbf{L}}^{\ast }$. Because the underlying vector space is finite-dimensional, all topologies on the space of operators are equivalent. So ${\left({\mathbf{L}}_k\right)}_k$ tends to $\mathbf{L}$ in norm means ${\left({\mathbf{L}}_k\right)}_k$ tends to $\mathbf{L}$ entrywise. This in turn implies that, since each ${\mathbf{L}}_k$ is lower triangular with non-negative diagonal entries, $\mathbf{L}$ is also.

Proof by QR decomposition

Let $\mathbf{A}$ be a positive semi-definite Hermitian matrix. Then it can be written as a product of its square root matrix, $\mathbf{A}=\mathbf{B}{\mathbf{B}}^{\ast }$. Now QR decomposition can be applied to ${\mathbf{B}}^{\ast }$, resulting in ${\mathbf{B}}^{\ast }=\mathbf{Q}\mathbf{R}$ , where $\mathbf{Q}$ is unitary and $\mathbf{R}$ is upper triangular. Inserting the decomposition into the original equality yields $A=\mathbf{B}{\mathbf{B}}^{\ast }=(\mathbf{Q}\mathbf{R}{)}^{\ast }\mathbf{Q}\mathbf{R}={\mathbf{R}}^{\ast }{\mathbf{Q}}^{\ast }\mathbf{Q}\mathbf{R}={\mathbf{R}}^{\ast }\mathbf{R}$. Setting $\mathbf{L}={\mathbf{R}}^{\ast }$ completes the proof.

Generalization

The Cholesky factorization can be generalized to (not necessarily finite) matrices with operator entries. Let $\{{\mathcal{H}}_n\}$ be a sequence of Hilbert spaces. Consider the operator matrix

$${\displaystyle \mathbf{A}=\left[\begin{array}{cccc}{\mathbf{A}}_{11}& {\mathbf{A}}_{12}& {\mathbf{A}}_{13}& \\ {\mathbf{A}}_{12}^{\ast }& {\mathbf{A}}_{22}& {\mathbf{A}}_{23}& \\ {\mathbf{A}}_{13}^{\ast }& {\mathbf{A}}_{23}^{\ast }& {\mathbf{A}}_{33}& \\ & & & \ddots \end{array}\right]}$$

acting on the direct sum

$${\displaystyle \mathcal{H}=\underset{n}{⨁}{\mathcal{H}}_n,}$$

where each

$${\displaystyle {\mathbf{A}}_{ij}:{\mathcal{H}}_j\to {\mathcal{H}}_i}$$

is a bounded operator. If A is positive (semidefinite) in the sense that for all finite k and for any

$${\displaystyle h\in \underset{n=1}{\overset{k}{⨁}}{\mathcal{H}}_k,}$$

there is $⟨h,\mathbf{A}h⟩\ge 0$, then there exists a lower triangular operator matrix L such that A = LL*. One can also take the diagonal entries of L to be positive.

Implementations in programming libraries

• C programming language: the GNU Scientific Library provides several implementations of Cholesky decomposition.
• Maxima computer algebra system: function cholesky computes Cholesky decomposition.
• GNU Octave numerical computations system provides several functions to calculate, update, and apply a Cholesky decomposition.
• The LAPACK library provides a high performance implementation of the Cholesky decomposition that can be accessed from Fortran, C and most languages. The Cholesky decomposition is available through the *POTRF family of subroutines, and the LDL decomposition through the *HETRF family of subroutines.
• In Python, the function cholesky from the numpy.linalg module performs Cholesky decomposition. The scipy.linalg module contains the ldl function for the LDL decomposition.
• In Matlab, the chol function gives the Cholesky decomposition. Note that chol uses the upper triangular factor of the input matrix by default, i.e. it computes $A=R^{\ast }R$ where $R$ is upper triangular. A flag can be passed to use the lower triangular factor instead.
• In R, the chol function returns the upper triangular Cholesky factor. You can obtain the lower triangular Cholesky factor by taking the transpose of the output.
• In Julia, the cholesky function from the LinearAlgebra standard library gives the Cholesky decomposition.
• In Mathematica, the function "CholeskyDecomposition" can be applied to a matrix.
• In C++, multiple linear algebra libraries support this decomposition:
  • The Armadillo (C++ library) supplies the command chol to perform Cholesky decomposition.
  • The Eigen library supplies Cholesky factorizations for both sparse and dense matrices.
  • In the ROOT package, the TDecompChol class is available.

• In Analytica, the function Decompose gives the Cholesky decomposition.
• The Apache Commons Math library has an implementation which can be used in Java, Scala and any other JVM language.