Entropy of mixing

Solutions and temperature dependence of miscibility

Ideal and regular solutions

The above equation for the entropy of mixing of ideal gases is valid also for certain liquid (or solid) solutions—those formed by completely random mixing so that the components move independently in the total volume. Such random mixing of solutions occurs if the interaction energies between unlike molecules are similar to the average interaction energies between like molecules. The value of the entropy corresponds exactly to random mixing for ideal solutions and for regular solutions, and approximately so for many real solutions.

For binary mixtures the entropy of random mixing can be considered as a function of the mole fraction of one component.

${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}=-nR(x_1\ln x_1+x_2\ln x_2)=-nR[x\ln x+(1-x)\ln (1-x)]}$

For all possible mixtures, ${\displaystyle 0<x<1}$, so that ${\displaystyle \ln }$ ${\displaystyle x}$ and ${\displaystyle \ln (1-x)}$ are both negative and the entropy of mixing ${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}}$ is positive and favors mixing of the pure components.

The curvature of ${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}}$ as a function of ${\displaystyle x}$ is given by the second derivative

${\displaystyle {\left(\frac{{\mathrm{\partial }}^2\mathrm{\Delta }{S}_{\text{mix}}}{\mathrm{\partial }{x}^2}\right)}_{T,P}=-nR\left(\frac{1}{x}+\frac{1}{1-x}\right)}$

This curvature is negative for all possible mixtures ${\displaystyle (0<x<1)}$, so that mixing two solutions to form a solution of intermediate composition also increases the entropy of the system. Random mixing therefore always favors miscibility and opposes phase separation.

For ideal solutions, the enthalpy of mixing is zero so that the components are miscible in all proportions. For regular solutions a positive enthalpy of mixing may cause incomplete miscibility (phase separation for some compositions) at temperatures below the upper critical solution temperature (UCST). This is the minimum temperature at which the ${\displaystyle -T\mathrm{\Delta }{S}_{mix}}$ term in the Gibbs energy of mixing is sufficient to produce miscibility in all proportions.

Systems with a lower critical solution temperature

Nonrandom mixing with a lower entropy of mixing can occur when the attractive interactions between unlike molecules are significantly stronger (or weaker) than the mean interactions between like molecules. For some systems this can lead to a lower critical solution temperature (LCST) or lower limiting temperature for phase separation.

For example, triethylamine and water are miscible in all proportions below 19 °C, but above this critical temperature, solutions of certain compositions separate into two phases at equilibrium with each other. This means that ${\displaystyle \mathrm{\Delta }{G}_{\text{mix}}}$ is negative for mixing of the two phases below 19 °C and positive above this temperature. Therefore, ${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}=-{\left(\frac{\mathrm{\partial }\mathrm{\Delta }{G}_{\text{mix}}}{\mathrm{\partial }T}\right)}_P}$ is negative for mixing of these two equilibrium phases. This is due to the formation of attractive hydrogen bonds between the two components that prevent random mixing. Triethylamine molecules cannot form hydrogen bonds with each other but only with water molecules, so in solution they remain associated to water molecules with loss of entropy. The mixing that occurs below 19 °C is due not to entropy but to the enthalpy of formation of the hydrogen bonds.

Lower critical solution temperatures also occur in many polymer-solvent mixtures. For polar systems such as polyacrylic acid in 1,4-dioxane, this is often due to the formation of hydrogen bonds between polymer and solvent. For nonpolar systems such as polystyrene in cyclohexane, phase separation has been observed in sealed tubes (at high pressure) at temperatures approaching the liquid-vapor critical point of the solvent. At such temperatures the solvent expands much more rapidly than the polymer, whose segments are covalently linked. Mixing therefore requires contraction of the solvent for compatibility of the polymer, resulting in a loss of entropy.

Statistical thermodynamical explanation of the entropy of mixing of ideal gases

Since thermodynamic entropy can be related to statistical mechanics or to information theory, it is possible to calculate the entropy of mixing using these two approaches. Here we consider the simple case of mixing ideal gases.

Proof from statistical mechanics

Assume that the molecules of two different substances are approximately the same size, and regard space as subdivided into a square lattice whose cells are the size of the molecules. (In fact, any lattice would do, including close packing.) This is a crystal-like conceptual model to identify the molecular centers of mass. If the two phases are liquids, there is no spatial uncertainty in each one individually. (This is, of course, an approximation. Liquids have a "free volume". This is why they are (usually) less dense than solids.) Everywhere we look in component 1, there is a molecule present, and likewise for component 2. After the two different substances are intermingled (assuming they are miscible), the liquid is still dense with molecules, but now there is uncertainty about what kind of molecule is in which location. Of course, any idea of identifying molecules in given locations is a thought experiment, not something one could do, but the calculation of the uncertainty is well-defined.

We can use Boltzmann's equation for the entropy change as applied to the mixing process

${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}=k_{\text{B}}\ln \mathrm{\Omega }}$

where ${\displaystyle k_{\text{B}}}$ is the Boltzmann constant. We then calculate the number of ways ${\displaystyle \mathrm{\Omega }}$ of arranging ${\displaystyle N_1}$ molecules of component 1 and ${\displaystyle N_2}$ molecules of component 2 on a lattice, where

${\displaystyle N=N_1+N_2}$

is the total number of molecules, and therefore the number of lattice sites. Calculating the number of permutations of ${\displaystyle N}$ objects, correcting for the fact that ${\displaystyle N_1}$ of them are identical to one another, and likewise for ${\displaystyle N_2}$,

${\displaystyle \mathrm{\Omega }=N!/N_1!N_2!}$

After applying Stirling's approximation for the factorial of a large integer m:

${\displaystyle \ln m!=\sum _k\ln k\approx {\int }_1^{m}dk\ln k=m\ln m-m+1\approx m\ln m-m}$,

the result is ${\displaystyle \mathrm{\Delta }{S}_{mix}=-k_{\text{B}}[N_1\ln (N_1/N)+N_2\ln (N_2/N)]=-k_{\text{B}}N[x_1\ln x_1+x_2\ln x_2]}$

where we have introduced the mole fractions, which are also the probabilities of finding any particular component in a given lattice site.

${\displaystyle x_1=N_1/N=p_1\text{and}{x}_2=N_2/N=p_2}$

Since the Boltzmann constant ${\displaystyle k_{\text{B}}=R/N_{\text{A}}}$, where ${\displaystyle N_{\text{A}}}$ is the Avogadro constant, and the number of molecules ${\displaystyle N=n{N}_{\text{A}}}$, we recover the thermodynamic expression for the mixing of two ideal gases, ${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}=-nR[x_1\ln x_1+x_2\ln x_2]}$

This expression can be generalized to a mixture of ${\displaystyle r}$ components, ${\displaystyle N_i}$, with ${\displaystyle i=1,2,3,\dots ,r}$

${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}=-k_{\text{B}}\sum _{i=1}^r{N}_i\ln (N_i/N)=-N{k}_{\text{B}}\sum _{i=1}^r{x}_i\ln x_i=-nR\sum _{i=1}^r{x}_i\ln x_i}$

The Flory–Huggins solution theory is an example of a more detailed model along these lines.

Relationship to information theory

The entropy of mixing is also proportional to the Shannon entropy or compositional uncertainty of information theory, which is defined without requiring Stirling's approximation. Claude Shannon introduced this expression for use in information theory, but similar formulas can be found as far back as the work of Ludwig Boltzmann and J. Willard Gibbs. The Shannon uncertainty is not the same as the Heisenberg uncertainty principle in quantum mechanics which is based on variance. The Shannon entropy is defined as:

${\displaystyle H=-\sum _{i=1}^r{p}_i\ln (p_i)}$

where p_i is the probability that an information source will produce the ith symbol from an r-symbol alphabet and is independent of previous symbols. (thus i runs from 1 to r ). H is then a measure of the expected amount of information (log p_i) missing before the symbol is known or measured, or, alternatively, the expected amount of information supplied when the symbol becomes known. The set of messages of length N symbols from the source will then have an entropy of NH.

The thermodynamic entropy is only due to positional uncertainty, so we may take the "alphabet" to be any of the r different chemical species in the gas, and, at equilibrium, the probability that a given particle is of type i is simply the mole fraction x_i for that particle. Since we are dealing with ideal gases, the identity of nearby particles is irrelevant. Multiplying by the number of particles N yields the change in entropy of the entire system from the unmixed case in which all of the p_i were either 1 or 0. We again obtain the entropy of mixing on multiplying by the Boltzmann constant ${\displaystyle k_{\text{B}}}$.

${\displaystyle \mathrm{\Delta }{S}_{\text{mix}}=-N{k}_{\text{B}}\sum _{i=1}^r{x}_i\ln x_i}$

So thermodynamic entropy with r chemical species with a total of N particles has a parallel to an information source that has r distinct symbols with messages that are N symbols long.

Application to gases

In gases there is a lot more spatial uncertainty because most of their volume is merely empty space. We can regard the mixing process as allowing the contents of the two originally separate contents to expand into the combined volume of the two conjoined containers. The two lattices that allow us to conceptually localize molecular centers of mass also join. The total number of empty cells is the sum of the numbers of empty cells in the two components prior to mixing. Consequently, that part of the spatial uncertainty concerning whether any molecule is present in a lattice cell is the sum of the initial values, and does not increase upon "mixing".

Almost everywhere we look, we find empty lattice cells. Nevertheless, we do find molecules in a few occupied cells. When there is real mixing, for each of those few occupied cells, there is a contingent uncertainty about which kind of molecule it is. When there is no real mixing because the two substances are identical, there is no uncertainty about which kind of molecule it is. Using conditional probabilities, it turns out that the analytical problem for the small subset of occupied cells is exactly the same as for mixed liquids, and the increase in the entropy, or spatial uncertainty, has exactly the same form as obtained previously. Obviously the subset of occupied cells is not the same at different times. But only when there is real mixing and an occupied cell is found do we ask which kind of molecule is there.

See also: Gibbs paradox, in which it would seem that "mixing" two samples of the same gas would produce entropy.

Application to solutions

If the solute is a crystalline solid, the argument is much the same. A pure crystal has no spatial uncertainty at all, except for crystallographic defects, and a (perfect) crystal allows us to localize the molecules using the crystal symmetry group. The fact that volumes do not add when dissolving a solid in a liquid is not important for condensed phases. If the solute is not crystalline, we can still use a spatial lattice, as good an approximation for an amorphous solid as it is for a liquid.

The Flory–Huggins solution theory provides the entropy of mixing for polymer solutions, in which the macromolecules are huge compared to the solvent molecules. In this case, the assumption is made that each monomer subunit in the polymer chain occupies a lattice site.

Note that solids in contact with each other also slowly interdiffuse, and solid mixtures of two or more components may be made at will (alloys, semiconductors, etc.). Again, the same equations for the entropy of mixing apply, but only for homogeneous, uniform phases.

Mixing under other constraints

Mixing with and without change of available volume

In the established customary usage, expressed in the lead section of this article, the entropy of mixing comes from two mechanisms, the intermingling and possible interactions of the distinct molecular species, and the change in the volume available for each molecular species, or the change in concentration of each molecular species. For ideal gases, the entropy of mixing at prescribed common temperature and pressure has nothing to do with mixing in the sense of intermingling and interactions of molecular species, but is only to do with expansion into the common volume.

According to Fowler and Guggenheim (1939/1965), the conflating of the just-mentioned two mechanisms for the entropy of mixing is well established in customary terminology, but can be confusing unless it is borne in mind that the independent variables are the common initial and final temperature and total pressure; if the respective partial pressures or the total volume are chosen as independent variables instead of the total pressure, the description is different.

Arieh Ben-Naim reiterates that the entropy increase is solely due to expansion, and discourages the widespread textbook statement that "mixing of different ideal gases is an inherently irreversible process". He demonstrates cases where an entropy increase occurs with instead demixing, and further argues that the concept of entropy of mixing has contributed to the widespread misconception of entropy as disorder.

Mixing with each gas kept at constant partial volume, with changing total volume

In contrast to the established customary usage, "mixing" might be conducted reversibly at constant volume for each of two fixed masses of gases of equal volume, being mixed by gradually merging their initially separate volumes by use of two ideal semipermeable membranes, each permeable only to one of the respective gases, so that the respective volumes available to each gas remain constant during the merge. Either one of the common temperature or the common pressure is chosen to be independently controlled by the experimenter, the other being allowed to vary so as to maintain constant volume for each mass of gas. In this kind of "mixing", the final common volume is equal to each of the respective separate initial volumes, and each gas finally occupies the same volume as it did initially.

This constant volume kind of "mixing", in the special case of perfect gases, is referred to in what is sometimes called Gibbs' theorem. It states that the entropy of such "mixing" of perfect gases is zero.

Mixing at constant total volume and changing partial volumes, with mechanically controlled varying pressure, and constant temperature

An experimental demonstration may be considered. The two distinct gases, in a cylinder of constant total volume, are at first separated by two contiguous pistons made respectively of two suitably specific ideal semipermeable membranes. Ideally slowly and fictively reversibly, at constant temperature, the gases are allowed to mix in the volume between the separating membranes, forcing them apart, thereby supplying work to an external system. The energy for the work comes from the heat reservoir that keeps the temperature constant. Then, by externally forcing ideally slowly the separating membranes together, back to contiguity, work is done on the mixed gases, fictively reversibly separating them again, so that heat is returned to the heat reservoir at constant temperature. Because the mixing and separation are ideally slow and fictively reversible, the work supplied by the gases as they mix is equal to the work done in separating them again. Passing from fictive reversibility to physical reality, some amount of additional work, that remains external to the gases and the heat reservoir, must be provided from an external source for this cycle, as required by the second law of thermodynamics, because this cycle has only one heat reservoir at constant temperature, and the external provision of work cannot be completely efficient.

Gibbs' paradox: "mixing" of identical species versus mixing of closely similar but non-identical species

For entropy of mixing to exist, the putatively mixing molecular species must be chemically or physically detectably distinct. Thus arises the so-called Gibbs paradox, as follows. If molecular species are identical, there is no entropy change on mixing them, because, defined in thermodynamic terms, there is no mass transfer, and thus no thermodynamically recognized process of mixing. Yet the slightest detectable difference in constitutive properties between the two species yields a thermodynamically recognized process of transfer with mixing, and a possibly considerable entropy change, namely the entropy of mixing.

The "paradox" arises because any detectable constitutive distinction, no matter how slight, can lead to a considerably large change in amount of entropy as a result of mixing. Though a continuous change in the properties of the materials that are mixed might make the degree of constitutive difference tend continuously to zero, the entropy change would nonetheless vanish discontinuously when the difference reached zero.

From a general physical viewpoint, this discontinuity is paradoxical. But from a specifically thermodynamic viewpoint, it is not paradoxical, because in that discipline the degree of constitutive difference is not questioned; it is either there or not there. Gibbs himself did not see it as paradoxical. Distinguishability of two materials is a constitutive, not a thermodynamic, difference, for the laws of thermodynamics are the same for every material, while their constitutive characteristics are diverse.

Though one might imagine a continuous decrease of the constitutive difference between any two chemical substances, physically it cannot be continuously decreased till it actually vanishes. It is hard to think of a smaller difference than that between ortho- and para-hydrogen. Yet they differ by a finite amount. The hypothesis, that the distinction might tend continuously to zero, is unphysical. This is neither examined nor explained by thermodynamics. Differences of constitution are explained by quantum mechanics, which postulates discontinuity of physical processes.

For a detectable distinction, some means should be physically available. One theoretical means would be through an ideal semi-permeable membrane. It should allow passage, backwards and forwards, of one species, while passage of the other is prevented entirely. The entirety of prevention should include perfect efficacy over a practically infinite time, in view of the nature of thermodynamic equilibrium. Even the slightest departure from ideality, as assessed over a finite time, would extend to utter non-ideality, as assessed over a practically infinite time. Such quantum phenomena as tunneling ensure that nature does not allow such membrane ideality as would support the theoretically demanded continuous decrease, to zero, of detectable distinction. The decrease to zero detectable distinction must be discontinuous.

It was pointed out by Jaynes that if the two different gases have been found to be experimentally indistinguishable, then any thermodynamic calculation assuming no entropy change upon mixing will be experimentally accurate. If a new experiment is performed for which thermodynamic predictions are incorrect due to the diffence between the gases, then that experiment provides a method to detect the difference between the two gases, requiring the entropy of mixing to be recalculated.

For ideal gases, the entropy of mixing does not depend on the degree of difference between the distinct molecular species, but only on the fact that they are distinct; for non-ideal gases, the entropy of mixing can depend on the degree of difference of the distinct molecular species. The suggested or putative "mixing" of identical molecular species is not in thermodynamic terms a mixing at all, because thermodynamics refers to states specified by state variables, and does not permit an imaginary labelling of particles. Only if the molecular species are different is there mixing in the thermodynamic sense.