Differential of a function

Definition

The differential is defined in modern treatments of differential calculus as follows. The differential of a function ${\displaystyle f(x)}$ of a single real variable ${\displaystyle x}$ is the function ${\displaystyle df}$ of two independent real variables ${\displaystyle x}$ and ${\displaystyle \mathrm{\Delta }x}$ given by

$${\displaystyle df(x,\mathrm{\Delta }x)\stackrel{\mathrm{d}\mathrm{e}\mathrm{f}}{=}{f}^{\prime }(x)\mathrm{\Delta }x.}$$

One or both of the arguments may be suppressed, i.e., one may see ${\displaystyle df(x)}$ or simply ${\displaystyle df}$. If ${\displaystyle y=f(x)}$, the differential may also be written as ${\displaystyle dy}$. Since ${\displaystyle dx(x,\mathrm{\Delta }x)=\mathrm{\Delta }x}$, it is conventional to write ${\displaystyle dx=\mathrm{\Delta }x}$ so that the following equality holds:

$${\displaystyle df(x)=f^{\prime }(x)dx}$$

This notion of differential is broadly applicable when a linear approximation to a function is sought, in which the value of the increment ${\displaystyle \mathrm{\Delta }x}$ is small enough. More precisely, if ${\displaystyle f}$ is a differentiable function at ${\displaystyle x}$, then the difference in ${\displaystyle y}$-values

$${\displaystyle \mathrm{\Delta }y\stackrel{\mathrm{d}\mathrm{e}\mathrm{f}}{=}f(x+\mathrm{\Delta }x)-f(x)}$$

satisfies

$${\displaystyle \mathrm{\Delta }y=f^{\prime }(x)\mathrm{\Delta }x+\epsilon =df(x)+\epsilon }$$

where the error ${\displaystyle \epsilon }$ in the approximation satisfies ${\displaystyle \epsilon /\mathrm{\Delta }x\to 0}$ as ${\displaystyle \mathrm{\Delta }x\to 0}$. In other words, one has the approximate identity

$${\displaystyle \mathrm{\Delta }y\approx dy}$$

in which the error can be made as small as desired relative to ${\displaystyle \mathrm{\Delta }x}$ by constraining ${\displaystyle \mathrm{\Delta }x}$ to be sufficiently small; that is to say, $${\displaystyle \frac{\mathrm{\Delta }y-dy}{\mathrm{\Delta }x}\to 0}$$ as ${\displaystyle \mathrm{\Delta }x\to 0}$. For this reason, the differential of a function is known as the principal (linear) part in the increment of a function: the differential is a linear function of the increment ${\displaystyle \mathrm{\Delta }x}$, and although the error ${\displaystyle \epsilon }$ may be nonlinear, it tends to zero rapidly as ${\displaystyle \mathrm{\Delta }x}$ tends to zero.

Differentials in several variables

Following Goursat (1904, I, §15), for functions of more than one independent variable,

$${\displaystyle y=f(x_1,\dots ,x_n),}$$

the partial differential of y with respect to any one of the variables x_i is the principal part of the change in y resulting from a change dx_i in that one variable. The partial differential is therefore

$${\displaystyle \frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_i}d{x}_i}$$

involving the partial derivative of y with respect to x_i. The sum of the partial differentials with respect to all of the independent variables is the total differential

$${\displaystyle dy=\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_1}d{x}_1+\cdots +\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_n}d{x}_n,}$$

which is the principal part of the change in y resulting from changes in the independent variables x_i.

More precisely, in the context of multivariable calculus, following Courant (1937b), if f is a differentiable function, then by the definition of differentiability, the increment

$${\displaystyle \begin{array}{cc}\mathrm{\Delta }y& \stackrel{\mathrm{d}\mathrm{e}\mathrm{f}}{=}f(x_1+\mathrm{\Delta }{x}_1,\dots ,x_n+\mathrm{\Delta }{x}_n)-f(x_1,\dots ,x_n)\\ & =\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_1}\mathrm{\Delta }{x}_1+\cdots +\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_n}\mathrm{\Delta }{x}_n+{\epsilon }_1\mathrm{\Delta }{x}_1+\cdots +{\epsilon }_n\mathrm{\Delta }{x}_n\end{array}}$$

where the error terms ε _i tend to zero as the increments Δx_i jointly tend to zero. The total differential is then rigorously defined as

$${\displaystyle dy=\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_1}\mathrm{\Delta }{x}_1+\cdots +\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_n}\mathrm{\Delta }{x}_n.}$$

Since, with this definition, $${\displaystyle d{x}_i(\mathrm{\Delta }{x}_1,\dots ,\mathrm{\Delta }{x}_n)=\mathrm{\Delta }{x}_i,}$$ one has $${\displaystyle dy=\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_1}d{x}_1+\cdots +\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_n}d{x}_n.}$$

As in the case of one variable, the approximate identity holds

$${\displaystyle dy\approx \mathrm{\Delta }y}$$

in which the total error can be made as small as desired relative to $\sqrt{\mathrm{\Delta }{x}_1^2+\cdots +\mathrm{\Delta }{x}_n^2}$ by confining attention to sufficiently small increments.

Application of the total differential to error estimation

In measurement, the total differential is used in estimating the error ${\displaystyle \mathrm{\Delta }f}$ of a function ${\displaystyle f}$ based on the errors ${\displaystyle \mathrm{\Delta }x,\mathrm{\Delta }y,\dots }$ of the parameters ${\displaystyle x,y,\dots }$. Assuming that the interval is short enough for the change to be approximately linear:

$${\displaystyle \mathrm{\Delta }f(x)=f^{\prime }(x)\mathrm{\Delta }x}$$

and that the parameters ${\displaystyle x,y,\dots }$ are functionally independent, so that the total differential applies, the change in ${\displaystyle f}$ induced by simultaneous changes in all parameters is

$${\displaystyle \mathrm{\Delta }f\approx f_x\mathrm{\Delta }x+f_y\mathrm{\Delta }y+\cdots .}$$

The derivative ${\displaystyle f_x}$ with respect to the parameter ${\displaystyle x}$ gives the sensitivity of the function ${\displaystyle f}$ to a change in ${\displaystyle x}$, in particular to the error ${\displaystyle \mathrm{\Delta }x}$. How the individual errors ${\displaystyle \mathrm{\Delta }x,\mathrm{\Delta }y,\dots }$ combine into a bound on ${\displaystyle \mathrm{\Delta }f}$ depends on what is known about them.

If the errors are known only as bounds on magnitude (i.e., ${\displaystyle |\mathrm{\Delta }x|\le \delta x}$, etc.) and may have arbitrary signs, the corresponding worst-case bound on ${\displaystyle \mathrm{\Delta }f}$ is obtained by taking absolute values, since signed contributions could otherwise cancel:

$${\displaystyle |\mathrm{\Delta }f|\le |f_x|\delta x+|f_y|\delta y+\cdots .}$$

If instead the errors are modelled as statistically independent random variables with standard deviations ${\displaystyle {\sigma }_x,{\sigma }_y,\dots }$, then by propagation of variance the standard deviation of ${\displaystyle \mathrm{\Delta }f}$ adds in quadrature:

$${\displaystyle {\sigma }_f^2=f_x^2{\sigma }_x^2+f_y^2{\sigma }_y^2+\cdots .}$$

The first formula is a deterministic upper bound, the second a typical (one-standard-deviation) estimate; the two coincide only in the trivial single-variable case.

From the worst-case principle the familiar error rules of summation, multiplication, etc. are derived, e.g.:

Let ${\displaystyle f(a,b)=ab}$. Then the finite error can be approximated as

$${\displaystyle \mathrm{\Delta }f=f_a\mathrm{\Delta }a+f_b\mathrm{\Delta }b.}$$ Evaluating the derivatives: $${\displaystyle \mathrm{\Delta }f=b\mathrm{\Delta }a+a\mathrm{\Delta }b.}$$ Dividing by f, which is a × b

$${\displaystyle \frac{\mathrm{\Delta }f}{f}=\frac{\mathrm{\Delta }a}{a}+\frac{\mathrm{\Delta }b}{b}.}$$

That is to say, in multiplication, the total relative error is the sum of the relative errors of the parameters.

To illustrate how this depends on the function considered, consider the case where the function is ${\displaystyle f(a,b)=a\ln b}$ instead. Then, it can be computed that the error estimate is $${\displaystyle \frac{\mathrm{\Delta }f}{f}=\frac{\mathrm{\Delta }a}{a}+\frac{\mathrm{\Delta }b}{b\ln b}}$$ with an extra ln b factor not found in the case of a simple product. This additional factor tends to make the error smaller, as the denominator b ln b is larger than a bare b.

Higher-order differentials

Higher-order differentials of a function y = f(x) of a single variable x can be defined via: $${\displaystyle d^{2}y=d(dy)=d(f^{\prime }(x)dx)=(d{f}^{\prime }(x))dx=f^{″}(x)(dx{)}^2,}$$ and, in general, $${\displaystyle d^{n}y=f^{(n)}(x)(dx{)}^n.}$$ Informally, this motivates Leibniz's notation for higher-order derivatives $${\displaystyle f^{(n)}(x)=\frac{d^{n}f}{d{x}^n}.}$$ When the independent variable x itself is permitted to depend on other variables, then the expression becomes more complicated, as it must include also higher order differentials in x itself. Thus, for instance, $${\displaystyle \begin{array}{cc}{d}^{2}y& =f^{″}(x)(dx{)}^2+f^{\prime }(x)d^{2}x\\ d^{3}y& =f^{‴}(x)(dx{)}^3+3f^{″}(x)dx{d}^{2}x+f^{\prime }(x)d^{3}x\end{array}}$$ and so forth.

Similar considerations apply to defining higher order differentials of functions of several variables. For example, if f is a function of two variables x and y, then $${\displaystyle d^{n}f=\sum _{k=0}^n(\genfrac{}{}{0ex}{}{n}{k})\frac{{\mathrm{\partial }}^{n}f}{\mathrm{\partial }{x}^k\mathrm{\partial }{y}^{n-k}}(dx{)}^k(dy{)}^{n-k},}$$ where $(\genfrac{}{}{0ex}{}{n}{k})$ is a binomial coefficient. In more variables, an analogous expression holds, but with an appropriate multinomial expansion rather than binomial expansion.

Higher order differentials in several variables also become more complicated when the independent variables are themselves allowed to depend on other variables. For instance, for a function f of x and y which are allowed to depend on auxiliary variables, one has $${\displaystyle d^{2}f=\left(\frac{{\mathrm{\partial }}^{2}f}{\mathrm{\partial }{x}^2}(dx{)}^2+2\frac{{\mathrm{\partial }}^{2}f}{\mathrm{\partial }x\mathrm{\partial }y}dxdy+\frac{{\mathrm{\partial }}^{2}f}{\mathrm{\partial }{y}^2}(dy{)}^2\right)+\frac{\mathrm{\partial }f}{\mathrm{\partial }x}{d}^{2}x+\frac{\mathrm{\partial }f}{\mathrm{\partial }y}{d}^{2}y.}$$

Because of this notational awkwardness, the use of higher order differentials was roundly criticized by Hadamard (1935), who concluded:

Enfin, que signifie ou que représente l'égalité

$${\displaystyle d^{2}z=rd{x}^2+2sdxdy+td{y}^2?}$$

A mon avis, rien du tout.

That is: Finally, what is meant, or represented, by the equality [...]? In my opinion, nothing at all. In spite of this skepticism, higher order differentials did emerge as an important tool in analysis.

In these contexts, the n-th order differential of the function f applied to an increment Δx is defined by $${\displaystyle d^{n}f(x,\mathrm{\Delta }x)={\frac{d^n}{d{t}^n}f(x+t\mathrm{\Delta }x)|}_{t=0}}$$ or an equivalent expression, such as $${\displaystyle \underset{t\to 0}{lim}\frac{{\mathrm{\Delta }}_{t\mathrm{\Delta }x}^{n}f}{t^n}}$$ where ${\displaystyle {\mathrm{\Delta }}_{t\mathrm{\Delta }x}^{n}f}$ is an nth forward difference with increment tΔx.

This definition makes sense as well if f is a function of several variables (for simplicity taken here as a vector argument). Then the n-th differential defined in this way is a homogeneous function of degree n in the vector increment Δx. Furthermore, the Taylor series of f at the point x is given by $${\displaystyle f(x+\mathrm{\Delta }x)\sim f(x)+df(x,\mathrm{\Delta }x)+\frac{1}{2}{d}^{2}f(x,\mathrm{\Delta }x)+\cdots +\frac{1}{n!}{d}^{n}f(x,\mathrm{\Delta }x)+\cdots }$$ The higher order Gateaux derivative generalizes these considerations to infinite dimensional spaces.

Properties

A number of properties of the differential follow in a straightforward manner from the corresponding properties of the derivative, partial derivative, and total derivative. These include:

• Linearity: For constants a and b and differentiable functions f and g, $${\displaystyle d(af+bg)=adf+bdg.}$$
• Product rule: For two differentiable functions f and g, $${\displaystyle d(fg)=fdg+gdf.}$$

An operation d with these two properties is known in abstract algebra as a derivation. They imply the power rule $${\displaystyle d(f^n)=n{f}^{n-1}df}$$ In addition, various forms of the chain rule hold, in increasing level of generality:

• If y = f(u) is a differentiable function of the variable u and u = g(x) is a differentiable function of x, then $${\displaystyle dy=f^{\prime }(u)du=f^{\prime }(g(x))g^{\prime }(x)dx.}$$
• If y = f(x₁, ..., x_n) and all of the variables x₁, ..., x_n depend on another variable t, then by the chain rule for partial derivatives, one has $${\displaystyle \begin{array}{cc}dy=\frac{dy}{dt}dt& =\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_1}d{x}_1+\cdots +\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_n}d{x}_n\\ & =\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_1}\frac{d{x}_1}{dt}dt+\cdots +\frac{\mathrm{\partial }y}{\mathrm{\partial }{x}_n}\frac{d{x}_n}{dt}dt.\end{array}}$$ Heuristically, the chain rule for several variables can itself be understood by dividing through both sides of this equation by the infinitely small quantity dt.
• More general analogous expressions hold, in which the intermediate variables x_i depend on more than one variable.

General formulation

A consistent notion of differential can be developed for a function f : Rⁿ → R^m between two Euclidean spaces. Let x,Δx ∈ Rⁿ be a pair of Euclidean vectors. The increment in the function f is $${\displaystyle \mathrm{\Delta }f=f(\mathbf{x}+\mathrm{\Delta }\mathbf{x})-f(\mathbf{x}).}$$ If there exists an m × n matrix A such that $${\displaystyle \mathrm{\Delta }f=A\mathrm{\Delta }\mathbf{x}+\Vert \mathrm{\Delta }\mathbf{x}\Vert \mathit{\epsilon }}$$ in which the vector ε → 0 as Δx → 0, then f is by definition differentiable at the point x. The matrix A is sometimes known as the Jacobian matrix, and the linear transformation that associates to the increment Δx ∈ Rⁿ the vector AΔx ∈ R^m is, in this general setting, known as the differential df(x) of f at the point x. This is precisely the Fréchet derivative, and the same construction can be made to work for a function between any Banach spaces.

Another fruitful point of view is to define the differential directly as a kind of directional derivative: $${\displaystyle df(\mathbf{x},\mathbf{h})=\underset{t\to 0}{lim}\frac{f(\mathbf{x}+t\mathbf{h})-f(\mathbf{x})}{t}={\frac{d}{dt}f(\mathbf{x}+t\mathbf{h})|}_{t=0},}$$ which is the approach already taken for defining higher order differentials (and is most nearly the definition set forth by Cauchy). If t represents time and x position, then h represents a velocity instead of a displacement as we have heretofore regarded it. This yields yet another refinement of the notion of differential: that it should be a linear function of a kinematic velocity. The set of all velocities through a given point of space is known as the tangent space, and so df gives a linear function on the tangent space: a differential form. With this interpretation, the differential of f is known as the exterior derivative, and has broad application in differential geometry because the notion of velocities and the tangent space makes sense on any differentiable manifold. If, in addition, the output value of f also represents a position (in a Euclidean space), then a dimensional analysis confirms that the output value of df must be a velocity. If one treats the differential in this manner, then it is known as the pushforward since it "pushes" velocities from a source space into velocities in a target space.

Other approaches

Although the notion of having an infinitesimal increment dx is not well-defined in modern mathematical analysis, a variety of techniques exist for defining the infinitesimal differential so that the differential of a function can be handled in a manner that does not clash with the Leibniz notation. These include:

• Defining the differential as a kind of differential form, specifically the exterior derivative of a function. The infinitesimal increments are then identified with vectors in the tangent space at a point. This approach is popular in differential geometry and related fields, because it readily generalizes to mappings between differentiable manifolds.
• Differentials as nilpotent elements of commutative rings. This approach is popular in algebraic geometry.
• Differentials in smooth models of set theory. This approach is known as synthetic differential geometry or smooth infinitesimal analysis and is closely related to the algebraic geometric approach, except that ideas from topos theory are used to hide the mechanisms by which nilpotent infinitesimals are introduced.
• Differentials as infinitesimals in hyperreal number systems, which are extensions of the real numbers which contain invertible infinitesimals and infinitely large numbers. This is the approach of nonstandard analysis pioneered by Abraham Robinson.

Examples and applications

Differentials may be effectively used in numerical analysis to study the propagation of experimental errors in a calculation, and thus the overall numerical stability of a problem (Courant 1937a). Suppose that the variable x represents the outcome of an experiment and y is the result of a numerical computation applied to x. The question is to what extent errors in the measurement of x influence the outcome of the computation of y. If the x is known to within Δx of its true value, then Taylor's theorem gives the following estimate on the error Δy in the computation of y: $${\displaystyle \mathrm{\Delta }y=f^{\prime }(x)\mathrm{\Delta }x+\frac{(\mathrm{\Delta }x{)}^2}{2}{f}^{″}(\xi )}$$ where ξ = x + θΔx for some 0 < θ < 1. If Δx is small, then the second order term is negligible, so that Δy is, for practical purposes, well-approximated by dy = f'(x) Δx.

The differential is often useful to rewrite a differential equation $${\displaystyle \frac{dy}{dx}=g(x)}$$ in the form $${\displaystyle dy=g(x)dx,}$$ in particular when one wants to separate the variables.