Operator norm

Examples

Every real ${\displaystyle m}$-by-${\displaystyle n}$ matrix corresponds to a linear map from ${\displaystyle {\mathbb{R}}^n}$ to ${\displaystyle {\mathbb{R}}^m.}$ Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all ${\displaystyle m}$-by-${\displaystyle n}$ matrices of real numbers; these induced norms form a subset of matrix norms.

If we specifically choose the Euclidean norm on both ${\displaystyle {\mathbb{R}}^n}$ and ${\displaystyle {\mathbb{R}}^m,}$ then the matrix norm given to a matrix ${\displaystyle A}$ is the square root of the largest eigenvalue of the matrix ${\displaystyle A^{\ast }A}$ (where ${\displaystyle A^{\ast }}$ denotes the conjugate transpose of ${\displaystyle A}$). This is equivalent to assigning the largest singular value of ${\displaystyle A.}$

Passing to a typical infinite-dimensional example, consider the sequence space ${\displaystyle {\ell }^2,}$ which is an L^p space, defined by $${\displaystyle {\ell }^2=\left\{(a_n{)}_{n\ge 1}:a_n\in \mathbb{C},\sum _n|a_n{|}^2<\mathrm{\infty }\right\}.}$$

This can be viewed as an infinite-dimensional analogue of the Euclidean space ${\displaystyle {\mathbb{C}}^n.}$ Now consider a bounded sequence ${\displaystyle s_{\bullet }={\left(s_n\right)}_{n=1}^{\mathrm{\infty }}.}$ The sequence ${\displaystyle s_{\bullet }}$ is an element of the space ${\displaystyle {\ell }^{\mathrm{\infty }},}$ with a norm given by $${\displaystyle {\Vert s_{\bullet }\Vert }_{\mathrm{\infty }}=\underset{n}{sup}\left|s_n\right|.}$$

Define an operator ${\displaystyle T_s}$ by pointwise multiplication: $${\displaystyle {\left(a_n\right)}_{n=1}^{\mathrm{\infty }}\stackrel{T_s}{\mapsto }{\left(s_n\cdot a_n\right)}_{n=1}^{\mathrm{\infty }}.}$$

The operator ${\displaystyle T_s}$ is bounded with operator norm $${\displaystyle {\Vert T_s\Vert }_{\text{op}}={\Vert s_{\bullet }\Vert }_{\mathrm{\infty }}.}$$

This discussion extends directly to the case where ${\displaystyle {\ell }^2}$ is replaced by a general ${\displaystyle L^p}$ space with ${\displaystyle p>1}$ and ${\displaystyle {\ell }^{\mathrm{\infty }}}$ replaced by ${\displaystyle L^{\mathrm{\infty }}.}$

Equivalent definitions

Let ${\displaystyle A:V\to W}$ be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition ${\displaystyle V\ne \{0\}}$ then they are all equivalent:

${\displaystyle \begin{array}{cccccccccc}\Vert A{\Vert }_{\text{op}}& =inf& & \{c\ge 0& & :\Vert Av\Vert \le c\Vert v\Vert & & \text{ for all }& & v\in V\}\\ & =sup& & \{\Vert Av\Vert & & :\Vert v\Vert \le 1& & \text{ and }& & v\in V\}\\ & =sup& & \{\Vert Av\Vert & & :\Vert v\Vert <1& & \text{ and }& & v\in V\}\\ & =sup& & \{\Vert Av\Vert & & :\Vert v\Vert \in \{0,1\}& & \text{ and }& & v\in V\}\\ & =sup& & \{\Vert Av\Vert & & :\Vert v\Vert =1& & \text{ and }& & v\in V\}\text{ this equality holds if and only if }V\ne \{0\}\\ & =sup& & \{\frac{\Vert Av\Vert }{\Vert v\Vert }& & :v\ne 0& & \text{ and }& & v\in V\}\text{ this equality holds if and only if }V\ne \{0\}.\end{array}}$

If ${\displaystyle V=\{0\}}$ then the sets in the last two rows will be empty, and consequently their supremums over the set ${\displaystyle [-\mathrm{\infty },\mathrm{\infty }]}$ will equal ${\displaystyle -\mathrm{\infty }}$ instead of the correct value of ${\displaystyle 0.}$ If the supremum is taken over the set ${\displaystyle [0,\mathrm{\infty }]}$ instead, then the supremum of the empty set is ${\displaystyle 0}$ and the formulas hold for any ${\displaystyle V.}$

Importantly, a linear operator ${\displaystyle A:V\to W}$ is not, in general, guaranteed to achieve its norm ${\displaystyle \Vert A{\Vert }_{\text{op}}=sup\{\Vert Av\Vert :\Vert v\Vert \le 1,v\in V\}}$ on the closed unit ball ${\displaystyle \{v\in V:\Vert v\Vert \le 1\},}$ meaning that there might not exist any vector ${\displaystyle u\in V}$ of norm ${\displaystyle \Vert u\Vert \le 1}$ such that ${\displaystyle \Vert A{\Vert }_{\text{op}}=\Vert Au\Vert }$ (if such a vector does exist and if ${\displaystyle A\ne 0,}$ then ${\displaystyle u}$ would necessarily have unit norm ${\displaystyle \Vert u\Vert =1}$). R.C. James proved James's theorem in 1964, which states that a Banach space ${\displaystyle V}$ is reflexive if and only if every bounded linear functional ${\displaystyle f\in V^{\ast }}$ achieves its norm on the closed unit ball. It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball.

If ${\displaystyle A:V\to W}$ is bounded then $${\displaystyle \Vert A{\Vert }_{\text{op}}=sup\left\{\left|w^{\ast }(Av)\right|:\Vert v\Vert \le 1,\Vert w^{\ast }\Vert \le 1\text{ where }v\in V,w^{\ast }\in W^{\ast }\right\}}$$ and $${\displaystyle \Vert A{\Vert }_{\text{op}}={\Vert {}^{t}A\Vert }_{\text{op}}}$$ where ${\displaystyle {}^{t}A:W^{\ast }\to V^{\ast }}$ is the transpose of ${\displaystyle A:V\to W,}$ which is the linear operator defined by ${\displaystyle w^{\ast }\mapsto w^{\ast }\circ A.}$

Properties

The operator norm is indeed a norm on the space of all bounded operators between ${\displaystyle V}$ and ${\displaystyle W}$. This means $${\displaystyle \Vert A{\Vert }_{\text{op}}\ge 0\text{ and }\Vert A{\Vert }_{\text{op}}=0\text{ if and only if }A=0,}$$ $${\displaystyle \Vert aA{\Vert }_{\text{op}}=|a|\Vert A{\Vert }_{\text{op}}\text{ for every scalar }a,}$$ $${\displaystyle \Vert A+B{\Vert }_{\text{op}}\le \Vert A{\Vert }_{\text{op}}+\Vert B{\Vert }_{\text{op}}.}$$

The following inequality is an immediate consequence of the definition: $${\displaystyle \Vert Av\Vert \le \Vert A{\Vert }_{\text{op}}\Vert v\Vert \text{ for every }v\in V.}$$

The operator norm is also compatible with the composition, or multiplication, of operators: if ${\displaystyle V}$, ${\displaystyle W}$ and ${\displaystyle X}$ are three normed spaces over the same base field, and ${\displaystyle A:V\to W}$ and ${\displaystyle B:W\to X}$ are two bounded operators, then it is a sub-multiplicative norm, that is: $${\displaystyle \Vert BA{\Vert }_{\text{op}}\le \Vert B{\Vert }_{\text{op}}\Vert A{\Vert }_{\text{op}}.}$$

For bounded operators on ${\displaystyle V}$, this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

Table of common operator norms

By choosing different norms for the codomain, used in computing ${\displaystyle \Vert Av\Vert }$, and the domain, used in computing ${\displaystyle \Vert v\Vert }$, we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in ${\displaystyle N^2}$ operations (for an ${\displaystyle N\times N}$ matrix), with the exception of the ${\displaystyle {\ell }_2-{\ell }_2}$ norm (which requires ${\displaystyle N^3}$ operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations).

The norm of the adjoint or transpose can be computed as follows. We have that for any ${\displaystyle p,q,}$ then ${\displaystyle \Vert A{\Vert }_{p\to q}=\Vert A^{\ast }{\Vert }_{q^{\prime }\to p^{\prime }}}$ where ${\displaystyle p^{\prime },q^{\prime }}$ are Hölder conjugate to ${\displaystyle p,q,}$ that is, ${\displaystyle 1/p+1/p^{\prime }=1}$ and ${\displaystyle 1/q+1/q^{\prime }=1.}$

Operators on a Hilbert space

Suppose ${\displaystyle H}$ is a real or complex Hilbert space. If ${\displaystyle A:H\to H}$ is a bounded linear operator, then we have $${\displaystyle \Vert A{\Vert }_{\text{op}}={\Vert A^{\ast }\Vert }_{\text{op}}}$$ and $${\displaystyle {\Vert A^{\ast }A\Vert }_{\text{op}}=\Vert A{\Vert }_{\text{op}}^2,}$$ where ${\displaystyle A^{\ast }}$ denotes the adjoint operator of ${\displaystyle A}$ (which in Euclidean spaces with the standard inner product corresponds to the conjugate transpose of the matrix ${\displaystyle A}$).

In general, the spectral radius of ${\displaystyle A}$ is bounded above by the operator norm of ${\displaystyle A}$: $${\displaystyle \rho (A)\le \Vert A{\Vert }_{\text{op}}.}$$

To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator ${\displaystyle A}$ has spectrum ${\displaystyle \{0\}.}$ So ${\displaystyle \rho (A)=0}$ while ${\displaystyle \Vert A{\Vert }_{\text{op}}>0.}$

However, when a matrix ${\displaystyle N}$ is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that $${\displaystyle \rho (N)=\Vert N{\Vert }_{\text{op}}.}$$

This formula can sometimes be used to compute the operator norm of a given bounded operator ${\displaystyle A}$: define the Hermitian operator ${\displaystyle B=A^{\ast }A,}$ determine its spectral radius, and take the square root to obtain the operator norm of ${\displaystyle A.}$

The space of bounded operators on ${\displaystyle H,}$ with the topology induced by operator norm, is not separable. For example, consider the Lp space ${\displaystyle L^2[0,1],}$ which is a Hilbert space. For ${\displaystyle 0<t\le 1,}$ let ${\displaystyle {\mathrm{\Omega }}_t}$ be the characteristic function of ${\displaystyle [0,t],}$ and ${\displaystyle P_t}$ be the multiplication operator given by ${\displaystyle {\mathrm{\Omega }}_t,}$ that is, $${\displaystyle P_t(f)=f\cdot {\mathrm{\Omega }}_t.}$$

Then each ${\displaystyle P_t}$ is a bounded operator with operator norm 1 and $${\displaystyle {\Vert P_t-P_s\Vert }_{\text{op}}=1\text{ for all }t\ne s.}$$

But ${\displaystyle \{P_t:0<t\le 1\}}$ is an uncountable set. This implies the space of bounded operators on ${\displaystyle L^2([0,1])}$ is not separable, in operator norm. One can compare this with the fact that the sequence space ${\displaystyle {\ell }^{\mathrm{\infty }}}$ is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.