Gateaux derivative

Linearity and continuity

At each point ${\displaystyle u\in U,}$ the Gateaux differential defines a function $${\displaystyle dF(u;\cdot ):X\to Y.}$$

This function is homogeneous in the sense that for all scalars ${\displaystyle \alpha ,}$ $${\displaystyle dF(u;\alpha \psi )=\alpha dF(u;\psi ).}$$

However, this function need not be additive, so that the Gateaux differential may fail to be linear, unlike the Fréchet derivative. Even if linear, it may fail to depend continuously on ${\displaystyle \psi }$ if ${\displaystyle X}$ and ${\displaystyle Y}$ are infinite dimensional (i.e. in the case that ${\displaystyle dF(u;\cdot )}$ is an unbounded linear operator). Furthermore, for Gateaux differentials that are linear and continuous in ${\displaystyle \psi ,}$ there are several inequivalent ways to formulate their continuous differentiability.

For example, consider the real-valued function ${\displaystyle F}$ of two real variables defined by $${\displaystyle F(x,y)=\{\begin{array}{ll}{\displaystyle \frac{x^3}{x^2+y^2}}& \text{if }(x,y)\ne (0,0),\\ 0& \text{if }(x,y)=(0,0).\end{array}}$$ This is Gateaux differentiable at ${\displaystyle (0,0)}$ with its differential there being $${\displaystyle dF(0,0;a,b)=\{\begin{array}{ll}{\displaystyle \frac{F(\tau a,\tau b)-0}{\tau }}& (a,b)\ne (0,0),\\ 0& (a,b)=(0,0)\end{array}=\{\begin{array}{ll}{\displaystyle \frac{a^3}{a^2+b^2}}& (a,b)\ne (0,0),\\ 0& (a,b)=(0,0).\end{array}}$$ However this is continuous but not linear in the arguments ${\displaystyle (a,b).}$ In infinite dimensions, any discontinuous linear functional on ${\displaystyle X}$ is Gateaux differentiable, but its Gateaux differential at ${\displaystyle 0}$ is linear but not continuous.

Relation with the Fréchet derivative

If ${\displaystyle F}$ is Fréchet differentiable, then it is also Gateaux differentiable, and its Fréchet and Gateaux derivatives agree. The converse is clearly not true, since the Gateaux derivative may fail to be linear or continuous. In fact, it is even possible for the Gateaux derivative to be linear and continuous but for the Fréchet derivative to fail to exist.

Nevertheless, for functions ${\displaystyle F}$ from a complex Banach space ${\displaystyle X}$ to another complex Banach space ${\displaystyle Y,}$ the Gateaux derivative (where the limit is taken over complex ${\displaystyle \tau }$ tending to zero as in the definition of complex differentiability) is automatically linear, a theorem of Zorn (1945). Furthermore, if ${\displaystyle F}$ is (complex) Gateaux differentiable at each ${\displaystyle u\in U}$ with derivative $${\displaystyle DF(u):\psi \mapsto dF(u;\psi )}$$ then ${\displaystyle F}$ is Fréchet differentiable on ${\displaystyle U}$ with Fréchet derivative ${\displaystyle DF}$ (Zorn 1946). This is analogous to the result from basic complex analysis that a function is analytic if it is complex differentiable in an open set, and is a fundamental result in the study of infinite dimensional holomorphy.

Continuous differentiability

Continuous Gateaux differentiability may be defined in two inequivalent ways. Suppose that ${\displaystyle F:U\to Y}$ is Gateaux differentiable at each point of the open set ${\displaystyle U.}$ One notion of continuous differentiability in ${\displaystyle U}$ requires that the mapping on the product space $${\displaystyle dF:U\times X\to Y}$$ be continuous. Linearity need not be assumed: if ${\displaystyle X}$ and ${\displaystyle Y}$ are Fréchet spaces, then ${\displaystyle dF(u;\cdot )}$ is automatically bounded and linear for all ${\displaystyle u}$ (Hamilton 1982).

A stronger notion of continuous differentiability requires that $${\displaystyle u\mapsto DF(u)}$$ be a continuous mapping $${\displaystyle U\to L(X,Y)}$$ from ${\displaystyle U}$ to the space of continuous linear functions from ${\displaystyle X}$ to ${\displaystyle Y.}$ Note that this already presupposes the linearity of ${\displaystyle DF(u).}$

As a matter of technical convenience, this latter notion of continuous differentiability is typical (but not universal) when the spaces ${\displaystyle X}$ and ${\displaystyle Y}$ are Banach, since ${\displaystyle L(X,Y)}$ is also Banach and standard results from functional analysis can then be employed. The former is the more common definition in areas of nonlinear analysis where the function spaces involved are not necessarily Banach spaces. For instance, differentiation in Fréchet spaces has applications such as the Nash–Moser inverse function theorem in which the function spaces of interest often consist of smooth functions on a manifold.

Higher derivatives

Whereas higher order Fréchet derivatives are naturally defined as multilinear functions by iteration, using the isomorphisms ${\displaystyle L^n(X,Y)=L(X,L^{n-1}(X,Y)),}$ higher order Gateaux derivative cannot be defined in this way. Instead the ${\displaystyle n}$th order Gateaux derivative of a function ${\displaystyle F:U\subseteq X\to Y}$ in the direction ${\displaystyle h}$ is defined by

Rather than a multilinear function, this is instead a homogeneous function of degree ${\displaystyle n}$ in ${\displaystyle h.}$

There is another candidate for the definition of the higher order derivative, the function

that arises naturally in the calculus of variations as the second variation of ${\displaystyle F,}$ at least in the special case where ${\displaystyle F}$ is scalar-valued. However, this may fail to have any reasonable properties at all, aside from being separately homogeneous in ${\displaystyle h}$ and ${\displaystyle k.}$ It is desirable to have sufficient conditions in place to ensure that ${\displaystyle D^{2}F(u)\{h,k\}}$ is a symmetric bilinear function of ${\displaystyle h}$ and ${\displaystyle k,}$ and that it agrees with the polarization of ${\displaystyle d^{n}F.}$

For instance, the following sufficient condition holds (Hamilton 1982). Suppose that ${\displaystyle F}$ is ${\displaystyle C^1}$ in the sense that the mapping $${\displaystyle DF:U\times X\to Y}$$ is continuous in the product topology, and moreover that the second derivative defined by (3) is also continuous in the sense that $${\displaystyle D^{2}F:U\times X\times X\to Y}$$ is continuous. Then ${\displaystyle D^{2}F(u)\{h,k\}}$ is bilinear and symmetric in ${\displaystyle h}$ and ${\displaystyle k.}$ By virtue of the bilinearity, the polarization identity holds $${\displaystyle D^{2}F(u)\{h,k\}=\frac{1}{2}{d}^{2}F(u;h+k)-d^{2}F(u;h)-d^{2}F(u;k)}$$ relating the second order derivative ${\displaystyle D^{2}F(u)}$ with the differential ${\displaystyle d^{2}F(u;-).}$ Similar conclusions hold for higher order derivatives.

Properties

A version of the fundamental theorem of calculus holds for the Gateaux derivative of ${\displaystyle F,}$ provided ${\displaystyle F}$ is assumed to be sufficiently continuously differentiable. Specifically:

• Suppose that ${\displaystyle F:X\to Y}$ is ${\displaystyle C^1}$ in the sense that the Gateaux derivative is a continuous function ${\displaystyle dF:U\times X\to Y.}$ Then for any ${\displaystyle u\in U}$ and ${\displaystyle h\in X,}$$${\displaystyle F(u+h)-F(u)={\int }_0^{1}dF(u+th;h)dt}$$ where the integral is the Gelfand–Pettis integral (the weak integral) (Vainberg (1964)).

Many of the other familiar properties of the derivative follow from this, such as multilinearity and commutativity of the higher-order derivatives. Further properties, also consequences of the fundamental theorem, include:

• (The chain rule)
  $${\displaystyle d(G\circ F)(u;x)=dG(F(u);dF(u;x))}$$ for all ${\displaystyle u\in U}$ and ${\displaystyle x\in X.}$ (Importantly, as with simple partial derivatives, the Gateaux derivative does not satisfy the chain rule if the derivative is permitted to be discontinuous.)
• (Taylor's theorem with remainder)
  Suppose that the line segment between ${\displaystyle u\in U}$ and ${\displaystyle u+h}$ lies entirely within ${\displaystyle U.}$ If ${\displaystyle F}$ is ${\displaystyle C^k}$ then $${\displaystyle F(u+h)=F(u)+dF(u;h)+\frac{1}{2!}{d}^{2}F(u;h)+\cdots +\frac{1}{(k-1)!}{d}^{k-1}F(u;h)+R_k}$$ where the remainder term is given by $${\displaystyle R_k(u;h)=\frac{1}{(k-1)!}{\int }_0^1(1-t{)}^{k-1}{d}^{k}F(u+th;h)dt}$$

Example

Let ${\displaystyle X}$ be the Hilbert space of square-integrable functions on a Lebesgue measurable set ${\displaystyle \mathrm{\Omega }}$ in the Euclidean space ${\displaystyle {\mathbb{R}}^n.}$ The functional $${\displaystyle E:X\to \mathbb{R}}$$ $${\displaystyle E(u)={\int }_{\mathrm{\Omega }}F(u(x))dx}$$ where ${\displaystyle F}$ is a real-valued function of a real variable and ${\displaystyle u}$ is defined on ${\displaystyle \mathrm{\Omega }}$ with real values, has Gateaux derivative $${\displaystyle dE(u;\psi )=⟨F^{\prime }(u),\psi ⟩:={\int }_{\mathrm{\Omega }}{F}^{\prime }(u(x))\psi (x)dx.}$$

Indeed, the above is the limit ${\displaystyle \tau \to 0}$ of $${\displaystyle \begin{array}{cc}\frac{E(u+\tau \psi )-E(u)}{\tau }& =\frac{1}{\tau }\left({\int }_{\mathrm{\Omega }}F(u+\tau \psi )dx-{\int }_{\mathrm{\Omega }}F(u)dx\right)\\ & =\frac{1}{\tau }\left({\int }_{\mathrm{\Omega }}{\int }_0^1\frac{d}{ds}F(u+s\tau \psi )dsdx\right)\\ & ={\int }_{\mathrm{\Omega }}{\int }_0^1{F}^{\prime }(u+s\tau \psi )\psi dsdx.\end{array}}$$