Linear combination

Examples and counterexamples

Euclidean vectors

Let the field K be the set R of real numbers, and let the vector space V be the Euclidean space R³. Consider the vectors e₁ = (1,0,0), e₂ = (0,1,0) and e₃ = (0,0,1). Then any vector in R³ is a linear combination of e₁, e₂, and e₃.

To see that this is so, take an arbitrary vector (a₁,a₂,a₃) in R³, and write: $${\displaystyle \begin{array}{cc}(a_1,a_2,a_3)& =(a_1,0,0)+(0,a_2,0)+(0,0,a_3)\\ & =a_1(1,0,0)+a_2(0,1,0)+a_3(0,0,1)\\ & =a_1{\mathbf{e}}_1+a_2{\mathbf{e}}_2+a_3{\mathbf{e}}_3.\end{array}}$$

Functions

Let K be the set C of all complex numbers, and let V be the set C_C(R) of all continuous functions from the real line R to the complex plane C. Consider the vectors (functions) f and g defined by f(t) := e^it and g(t) := e^−it. (Here, e is the base of the natural logarithm, about 2.71828..., and i is the imaginary unit, a square root of −1.) Some linear combinations of f and g are:

• ${\displaystyle \cos t=\frac{1}{2}{e}^{it}+\frac{1}{2}{e}^{-it}}$
• ${\displaystyle 2\sin t=(-i)e^{it}+(i)e^{-it}.}$

On the other hand, the constant function 3 is not a linear combination of f and g. To see this, suppose that 3 could be written as a linear combination of e^it and e^−it. This means that there would exist complex scalars a and b such that ae^it + be^−it = 3 for all real numbers t. Setting t = 0 and t = π gives the equations a + b = 3 and a + b = −3, and clearly this cannot happen.

Polynomials

Let K be R, C, or any field, and let V be the set P of all polynomials with coefficients taken from the field K. Consider the vectors (polynomials) p₁ := 1, p₂ := x + 1, and p₃ := x² + x + 1.

To find out whether x² − 1 is a linear combination of p₁, p₂, and p₃, we consider an arbitrary linear combination of these vectors and try to see when it equals the desired vector x² − 1. Picking arbitrary coefficients a₁, a₂, and a₃, we want $${\displaystyle a_1(1)+a_2(x+1)+a_3(x^2+x+1)=x^2-1.}$$ Multiplying the polynomials out, this means $${\displaystyle (a_1)+(a_{2}x+a_2)+(a_3{x}^2+a_{3}x+a_3)=x^2-1}$$ and collecting like powers of x, we get $${\displaystyle a_3{x}^2+(a_2+a_3)x+(a_1+a_2+a_3)=1x^2+0x+(-1).}$$ Two polynomials are equal if and only if their corresponding coefficients are equal, so we can conclude $${\displaystyle a_3=1,a_2+a_3=0,a_1+a_2+a_3=-1.}$$ This system of linear equations can easily be solved. First, the first equation simply says that a₃ is 1. Knowing that, we can solve the second equation for a₂, which comes out to −1. Finally, the last equation tells us that a₁ is also −1. Therefore, the only possible way to get a linear combination is with these coefficients. Indeed, $${\displaystyle x^2-1=-1-(x+1)+(x^2+x+1)=-p_1-p_2+p_3}$$ so x² − 1 is a linear combination of p₁, p₂, and p₃.

On the other hand, if we try to make x³ − 1 a linear combination of p₁, p₂, and p₃, then following the same process as before, we get the equation

$${\displaystyle \begin{array}{cc}& 0x^3+a_3{x}^2+(a_2+a_3)x+(a_1+a_2+a_3)\\ =& 1x^3+0x^2+0x+(-1).\end{array}}$$

However, when we set corresponding coefficients equal in this case, the equation for x³ is $${\displaystyle 0=1}$$ which is always false. Therefore, there is no way for this to work, and x³ − 1 is not a linear combination of p₁, p₂, and p₃.

The linear span

Take an arbitrary field K, an arbitrary vector space V, and let v₁,...,v_n be vectors (in V). It is interesting to consider the set of all linear combinations of these vectors. This set is called the linear span (or just span) of the vectors, say S = {v₁, ..., v_n}. We write the span of S as span(S) or sp(S): $${\displaystyle \mathrm{span}({\mathbf{v}}_1,\dots ,{\mathbf{v}}_n):=\{a_1{\mathbf{v}}_1+\cdots +a_n{\mathbf{v}}_n:a_1,\dots ,a_n\in K\}.}$$

Linear independence

Suppose that, for some sets of vectors v₁,...,v_n, a single vector can be written in two different ways as a linear combination of them: $${\displaystyle \mathbf{v}=\sum _i{a}_i{\mathbf{v}}_i=\sum _i{b}_i{\mathbf{v}}_i\text{ where }{a}_i\ne b_i.}$$ This is equivalent, by subtracting these (${\displaystyle c_i:=a_i-b_i}$), to saying a non-trivial combination is zero: $${\displaystyle \mathbf{0}=\sum _i{c}_i{\mathbf{v}}_i.}$$

If that is possible, then v₁,...,v_n are called linearly dependent; otherwise, they are linearly independent. Similarly, we can speak of linear dependence or independence of an arbitrary set S of vectors.

If S is linearly independent and the span of S equals V, then S is a basis for V.

Affine, conical, and convex combinations

By restricting the coefficients used in linear combinations, one can define the related concepts of affine combination, conical combination, and convex combination, and the associated notions of sets closed under these operations.

Because these are more restricted operations, more subsets will be closed under them, so affine subsets, convex cones, and convex sets are generalizations of vector subspaces: a vector subspace is also an affine subspace, a convex cone, and a convex set, but a convex set need not be a vector subspace, affine, or a convex cone.

These concepts often arise when one can take certain linear combinations of objects, but not any: for example, probability distributions are closed under convex combination (they form a convex set), but not conical or affine combinations (or linear), and positive measures are closed under conical combination but not affine or linear – hence one defines signed measures as the linear closure.

Linear and affine combinations can be defined over any field (or ring), but conical and convex combination require a notion of "positive", and hence can only be defined over an ordered field (or ordered ring), generally the real numbers.

If one allows only scalar multiplication, not addition, one obtains a (not necessarily convex) cone; one often restricts the definition to only allowing multiplication by positive scalars.

All of these concepts are usually defined as subsets of an ambient vector space (except for affine spaces, which are also considered as "vector spaces forgetting the origin"), rather than being axiomatized independently.

Operad theory

More abstractly, in the language of operad theory, one can consider vector spaces to be algebras over the operad ${\displaystyle {\mathbf{R}}^{\mathrm{\infty }}}$ (the infinite direct sum, so only finitely many terms are non-zero; this corresponds to only taking finite sums), which parametrizes linear combinations: the vector ${\displaystyle (2,3,-5,0,\dots )}$ for instance corresponds to the linear combination ${\displaystyle 2{\mathbf{v}}_1+3{\mathbf{v}}_2-5{\mathbf{v}}_3+0{\mathbf{v}}_4+\cdots }$. Similarly, one can consider affine combinations, conical combinations, and convex combinations to correspond to the sub-operads where the terms sum to 1, the terms are all non-negative, or both, respectively. Graphically, these are the infinite affine hyperplane, the infinite hyper-octant, and the infinite simplex. This formalizes what is meant by ${\displaystyle {\mathbf{R}}^n}$ being or the standard simplex being model spaces, and such observations as that every bounded convex polytope is the image of a simplex. Here suboperads correspond to more restricted operations and thus more general theories.

From this point of view, we can think of linear combinations as the most general sort of operation on a vector space – saying that a vector space is an algebra over the operad of linear combinations is precisely the statement that all possible algebraic operations in a vector space are linear combinations.

The basic operations of addition and scalar multiplication, together with the existence of an additive identity and additive inverses, cannot be combined in any more complicated way than the generic linear combination: the basic operations are a generating set for the operad of all linear combinations.

Ultimately, this fact lies at the heart of the usefulness of linear combinations in the study of vector spaces.

Generalizations

If V is a topological vector space, then there may be a way to make sense of certain infinite linear combinations, using the topology of V. For example, we might be able to speak of a₁v₁ + a₂v₂ + a₃v₃ + ⋯, going on forever. Such infinite linear combinations do not always make sense; we call them convergent when they do. Allowing more linear combinations in this case can also lead to a different concept of span, linear independence, and basis. The articles on the various flavors of topological vector spaces go into more detail about these.

If K is a commutative ring instead of a field, then everything that has been said above about linear combinations generalizes to this case without change. The only difference is that we call spaces like this V modules instead of vector spaces. If K is a noncommutative ring, then the concept still generalizes, with one caveat: since modules over noncommutative rings come in left and right versions, our linear combinations may also come in either of these versions, whatever is appropriate for the given module. This is simply a matter of doing scalar multiplication on the correct side.

A more complicated twist comes when V is a bimodule over two rings, K_L and K_R. In that case, the most general linear combination looks like $${\displaystyle a_1{\mathbf{v}}_1{b}_1+\cdots +a_n{\mathbf{v}}_n{b}_n}$$ where a₁,...,a_n belong to K_L, b₁,...,b_n belong to K_R, and v₁,…,v_n belong to V.