Derivative (multivariable calculus)

Differentials

In some advanced calculus texts, the derivative is also called the differential. However, this term has several different, but closely connected meanings, in mathematics and the sciences.

When a differentiable function ${\displaystyle f:{\mathbb{R}}^n\to \mathbb{R}}$ is scalar valued, the derivative of ${\displaystyle f}$ at ${\displaystyle a}$ may be written as the Jacobian matrix, which in this instance is a row matrix (a matrix consisting of elements in a single row, i.e., a row vector): $${\displaystyle D{f}_a=\left[\begin{array}{ccc}\frac{\mathrm{\partial }f}{\mathrm{\partial }{x}_1}(a)& \cdots & \frac{\mathrm{\partial }f}{\mathrm{\partial }{x}_n}(a)\end{array}\right].}$$ The linear approximation property of the derivative implies that if $${\displaystyle \mathrm{\Delta }x={\left[\begin{array}{ccc}\mathrm{\Delta }{x}_1& \cdots & \mathrm{\Delta }{x}_n\end{array}\right]}^{\mathsf{T}}}$$ is a small vector (where the ${\displaystyle \mathsf{T}}$ denotes transpose, so that this vector is a column vector), then $${\displaystyle f(a+\mathrm{\Delta }x)-f(a)\approx D{f}_a\mathrm{\Delta }x=\sum _{i=1}^n\frac{\mathrm{\partial }f}{\mathrm{\partial }{x}_i}(a)\mathrm{\Delta }{x}_i.}$$ Heuristically, this suggests that if ${\displaystyle d{x}_1,\dots ,d{x}_n}$ are infinitesimal increments in the coordinate directions, then $${\displaystyle d{f}_a=\sum _{i=1}^n\frac{\mathrm{\partial }f}{\mathrm{\partial }{x}_i}(a)d{x}_i}$$ and this is the differential of $f$ at $a$. In fact, the notion of the infinitesimal, which is merely symbolic here, can be equipped with extensive mathematical structure. Techniques, such as the theory of differential forms, effectively give analytical and algebraic descriptions of objects like infinitesimal increments, ${\displaystyle d{x}_i}$. For instance, ${\displaystyle d{x}_i}$ may be inscribed as a linear functional on the vector space ${\displaystyle {\mathbb{R}}^n}$. Evaluating ${\displaystyle d{x}_i}$ at a vector ${\displaystyle h}$ in ${\displaystyle {\mathbb{R}}^n}$ measures how much ${\displaystyle h}$ points in the ${\displaystyle i}$-th coordinate direction. The differential ${\displaystyle d{f}_a}$ is a linear combination of linear functionals and hence is itself a linear functional. The evaluation ${\displaystyle d{f}_a(h)}$ is the directional derivative of ${\displaystyle f}$ along ${\displaystyle h}$. This point of view makes the derivative an instance of the exterior derivative.

Being a linear form, the differential is naturally a covector. In a Euclidean space, a covector is naturally dual to a vector (its transpose). This vector is called the gradient of ${\displaystyle f}$, and points in the direction in which ${\displaystyle f}$ increases most rapidly.

Suppose now that ${\displaystyle f}$ is a vector-valued function, that is, ${\displaystyle f:{\mathbb{R}}^n\to {\mathbb{R}}^m}$. In this case, the components ${\displaystyle f_i}$ of ${\displaystyle f}$ are real-valued functions, so they have associated differential forms ${\displaystyle d{f}_i}$. The differential ${\displaystyle df}$ amalgamates these forms into a single object and is therefore an instance of a vector-valued differential form.

If ${\displaystyle f:M\to N}$ is a mapping between differentiable manifolds, differentiability can be formulated by differentiability in any coordinate chart. Invariantly, the differential of ${\displaystyle f}$ at a point ${\displaystyle p\in M}$ is a linear map ${\displaystyle d{f}_p:T_{p}M\to T_{f(p)}N}$ from the tangent space of ${\displaystyle M}$ at ${\displaystyle p}$ to that of ${\displaystyle N}$ at ${\displaystyle f(p)}$. This is also known as the pushforward. This is closely related to the derivative as a linear approximation, since evaluating ${\displaystyle d{f}_p}$ on a tangent vector gives the directional derivative of ${\displaystyle f}$ in that direction. Nevertheless, the differential is not quite the same thing as the derivative of a function between vector spaces: ${\displaystyle f}$ is a mapping of manifolds and not of the vector spaces ${\displaystyle T_{p}M}$ and ${\displaystyle T_{f(p)}N}$ where the linear approximation lives.

In applications such as thermodynamics, the language of differentials often has an additional conceptual role. Expressions such as ${\displaystyle dU=TdS-PdV}$ describe differentials of state functions and lead to questions about exactness, natural variables, and relations among partial derivatives. This use is related to the derivative of a multivariable function, but it is not just another name for the linear map ${\displaystyle D{f}_a}$ associated with an ordinary function.

Total derivative

The term total derivative is also used in more than one way. In some mathematical texts it denotes the full derivative ${\displaystyle D{f}_a}$, as opposed to any one partial derivative. In that sense, the total derivative is the linear map that accounts for variation in all coordinate directions simultaneously.

In many applied contexts, however, "total derivative" refers instead to the derivative of a composite dependence. For example, if $${\displaystyle z=f(x,y),x=x(t),y=y(t),}$$ then the total derivative of ${\displaystyle z}$ with respect to ${\displaystyle t}$ is $${\displaystyle \frac{dz}{dt}=\frac{\mathrm{\partial }f}{\mathrm{\partial }x}\frac{dx}{dt}+\frac{\mathrm{\partial }f}{\mathrm{\partial }y}\frac{dy}{dt}.}$$ This is the ordinary derivative of the composite function ${\displaystyle f(x(t),y(t))}$, computed by the chain rule. Equivalently, it is obtained by applying the derivative of ${\displaystyle f}$ to the velocity vector of the path ${\displaystyle t\mapsto (x(t),y(t))}$: $${\displaystyle \frac{d}{dt}f(x(t),y(t))=D{f}_{(x(t),y(t))}(x^{\prime }(t),y^{\prime }(t)).}$$

This chain-rule sense of "total derivative" is common in physics, engineering, economics, and other applied fields. In mechanics, for instance, the total time derivative of a function ${\displaystyle F(q,p,t)}$ along a trajectory includes both explicit dependence on ${\displaystyle t}$ and implicit dependence through the time-dependent variables ${\displaystyle q(t)}$ and ${\displaystyle p(t)}$. In fluid mechanics, related terminology appears in the material derivative, which differentiates a quantity along the motion of a fluid parcel.

Another example is in classical mechanics, where the total derivative of a function that depends on phase space parameters and time is its partial derivative in time plus its Poisson bracket with the Hamiltonian ${\displaystyle H}$: $${\displaystyle \frac{df}{dt}=\frac{\mathrm{\partial }f}{\mathrm{\partial }t}+\{f,H\}}$$ Like the material derivative, total derivative in mechanics has the property that it is the derivative of the composite when pulled back to any Hamiltonian trajectory, but is still treated as a function of all phase space coordinates and time.

In comparative statics, total derivatives often describe how endogenous variables change with respect to exogenous variables in an implicitly defined system of equations. The endogeneous variables are generally not explicit functions of the exogeneous variables, other than through the implicit function theorem, and the total derivative is handled implicitly.

Thus, although "total derivative" can mean the derivative ${\displaystyle D{f}_a}$ in the sense above, the term also commonly refers to a derivative along a specified dependence or process. This ambiguity is one reason to distinguish between the derivative as a linear map and the various differential or total-derivative notations used in applications.

Chain rule

A form of the chain rule generalizes from one-variable calculus. It says that, for two functions ${\displaystyle f}$ and ${\displaystyle g}$, the derivative of the composite function ${\displaystyle f\circ g}$ at ${\displaystyle a}$ satisfies $${\displaystyle D(f\circ g{)}_a=D{f}_{g(a)}\circ D{g}_a}$$ where the composite on the right-hand side is the composition of linear maps. If the derivatives of ${\displaystyle f}$ and ${\displaystyle g}$ are identified with their Jacobian matrices, then the composite on the right-hand side is simply matrix multiplication.

Example: Differentiation with direct dependencies

Suppose that f is a function of two variables, x and y. If these two variables are independent, so that the domain of f is ${\displaystyle {\mathbb{R}}^2}$, then the behavior of f may be understood in terms of its partial derivatives in the x and y directions. However, in some situations, x and y may be dependent. For example, it might happen that f is constrained to a curve ${\displaystyle y=y(x)}$. In this case, we are actually interested in the behavior of the composite function ${\displaystyle f(x,y(x))}$. The partial derivative of f with respect to x does not give the true rate of change of f with respect to changing x because changing x necessarily changes y, while the partial derivative assumes y is fixed. However, the chain rule takes such dependencies into account. Write ${\displaystyle \gamma (x)=(x,y(x))}$. Then, the chain rule says $${\displaystyle D(f\circ \gamma {)}_{x_0}=D{f}_{(x_0,y(x_0))}\circ D{\gamma }_{x_0}.}$$ By expressing the derivative using Jacobian matrices such as$${\displaystyle \begin{array}{lcl}D{f}_{(x_0,y(x_0))}& =& {\left[\begin{array}{cc}\frac{\mathrm{\partial }f}{\mathrm{\partial }x}& \frac{\mathrm{\partial }f}{\mathrm{\partial }y}\end{array}\right]}_{(x_0,y(x_0))}\\ D{\gamma }_{x_0}& =& {\left[\begin{array}{c}\frac{\mathrm{\partial }x}{\mathrm{\partial }x}\\ \frac{\mathrm{\partial }y}{\mathrm{\partial }x}\end{array}\right]}_{x_0}.\end{array}}$$This becomes: $${\displaystyle \frac{df(x,y(x))}{dx}(x_0)=\frac{\mathrm{\partial }f}{\mathrm{\partial }x}(x_0,y(x_0))\frac{dx}{dx}(x_0)+\frac{\mathrm{\partial }f}{\mathrm{\partial }y}(x_0,y(x_0))\frac{dy}{dx}(x_0).}$$ Suppressing the evaluation at ${\displaystyle x_0}$ for legibility, we may also write this as $${\displaystyle \frac{df(x,y(x))}{dx}=\frac{\mathrm{\partial }f}{\mathrm{\partial }x}\frac{dx}{dx}+\frac{\mathrm{\partial }f}{\mathrm{\partial }y}\frac{dy}{dx}.}$$ This gives a straightforward formula for the derivative of ${\displaystyle f(x,y(x))}$ in terms of the partial derivatives of ${\displaystyle f}$ and the derivative of ${\displaystyle y(x)}$.

For example, suppose $${\displaystyle f(x,y)=xy.}$$ The rate of change of f with respect to x is usually the partial derivative of f with respect to x; in this case, $${\displaystyle \frac{\mathrm{\partial }f}{\mathrm{\partial }x}=y.}$$ However, if y depends on x, the partial derivative does not give the true rate of change of f as x changes because the partial derivative assumes that y is fixed. Suppose we are constrained to the line $${\displaystyle y=x.}$$ Then $${\displaystyle f(x,y)=f(x,x)=x^2,}$$ and the total derivative of f with respect to x is $${\displaystyle \frac{df}{dx}=2x,}$$ which we see is not equal to the partial derivative ${\displaystyle \mathrm{\partial }f/\mathrm{\partial }x}$. Instead of immediately substituting for y in terms of x, however, we can also use the chain rule as above: $${\displaystyle \frac{df}{dx}=\frac{\mathrm{\partial }f}{\mathrm{\partial }x}+\frac{\mathrm{\partial }f}{\mathrm{\partial }y}\frac{dy}{dx}=y+x\cdot 1=x+y=2x.}$$

Example: Differentiation with indirect dependencies

While one can often perform substitutions to eliminate indirect dependencies, the chain rule provides for a more efficient and general technique. Suppose ${\displaystyle L(t,x_1,\dots ,x_n)}$ is a function of time ${\displaystyle t}$ and ${\displaystyle n}$ variables ${\displaystyle x_i}$ which themselves depend on time. Then, the time derivative of ${\displaystyle L}$ is $${\displaystyle \frac{dL}{dt}=\frac{d}{dt}L(t,x_1(t),\dots ,x_n(t)).}$$

The chain rule expresses this derivative in terms of the partial derivatives of ${\displaystyle L}$ and the time derivatives of the functions ${\displaystyle x_i}$: $${\displaystyle \frac{dL}{dt}=\frac{\mathrm{\partial }L}{\mathrm{\partial }t}+\sum _{i=1}^n\frac{\mathrm{\partial }L}{\mathrm{\partial }{x}_i}\frac{d{x}_i}{dt}=(\frac{\mathrm{\partial }}{\mathrm{\partial }t}+\sum _{i=1}^n\frac{d{x}_i}{dt}\frac{\mathrm{\partial }}{\mathrm{\partial }{x}_i})(L).}$$

This expression is often used in physics for a gauge transformation of the Lagrangian, as two Lagrangians that differ only by the total time derivative of a function of time and the ${\displaystyle n}$ generalized coordinates lead to the same equations of motion. An interesting example concerns the resolution of causality concerning the Wheeler–Feynman time-symmetric theory. The operator in brackets (in the final expression above) is also called the total derivative operator (with respect to ${\displaystyle t}$).

For example, the total derivative of ${\displaystyle f(x(t),y(t))}$ is

$${\displaystyle \frac{df}{dt}=\frac{\mathrm{\partial }f}{\mathrm{\partial }x}\frac{dx}{dt}+\frac{\mathrm{\partial }f}{\mathrm{\partial }y}\frac{dy}{dt}.}$$

Here there is no ${\displaystyle \mathrm{\partial }f/\mathrm{\partial }t}$ term since ${\displaystyle f}$ itself does not depend on the independent variable ${\displaystyle t}$ directly.

Total differential equation

A total differential equation is a differential equation expressed in terms of total derivatives. Since the exterior derivative is coordinate-free, in a sense that can be given a technical meaning, such equations are intrinsic and geometric.

Application to equation systems

In economics, it is common for the total derivative to arise in the context of a system of equations. For example, a simple supply-demand system might specify the quantity q of a product demanded as a function D of its price p and consumers' income I, the latter being an exogenous variable, and might specify the quantity supplied by producers as a function S of its price and two exogenous resource cost variables r and w. The resulting system of equations $${\displaystyle \begin{array}{cc}q& =D(p,I)\\ q& =S(p,r,w)\end{array}}$$ determines the market equilibrium values of the variables p and q. The total derivative ${\displaystyle dp/dr}$ of p with respect to r, for example, gives the sign and magnitude of the reaction of the market price to the exogenous variable r. In the indicated system, there are a total of six possible total derivatives, also known in this context as comparative static derivatives: dp/dr, dp/dw, dp/dI, dq/dr, dq/dw, and dq/dI. The total derivatives are found by totally differentiating the system of equations, dividing through by, say dr, treating dq/dr and dp/dr as the unknowns, setting dI = dw = 0, and solving the two totally differentiated equations simultaneously, typically by using Cramer's rule.