Gibbs paradox: an urban legend in statistical physics

[autoUFC]

TL;DR Summary

There is no paradox. Gibbs made a mistake. Ehrenfest explained the entropy of mixing of distinguishble elements with the problem of the dog's fleas. However, more than a century later this fact is ignored by most.

Hi,
I recently discovered that there is no real paradox in the question of the mixing of classical distinguishble particles. I was shocked. Most books and all my professors suggest that an extensible entropy could not be defined for distinguishble particles.

I believe that many of you will be skeptical of my claim that no paradox exists. After a careful read of the Wikipedia article on the gibbs paradox I learned that there is no real paradox. Gibbs made a mistake. Van Kampen 1984, and Jaynes 1996 (two references that I found in the wikipedia page) make the point clear. Jaynes goes as far as to suggest that Gibbs was senile when he wrote about the paradox of mixing.
A 2 pages text that is right to the point is
Swendsen, R. H. (2008). Gibbs’ paradox and the definition of entropy. Entropy, 10(1), 15-18.

I would like to know if other people is surprised with that. It seems to me that because physicists are more concerned with Bose and Fermi indistinguishable particles, this misconception regarding the mixing of distinguishble elements has lingered.

 

[vanhees71]

I'm not aware of the claim that an an "extensible entropy" (whatever you mean by that) could not be defined for distinguishable particles. It's of course clear that there must be mixing entropy for distinguishable particles, because it's a real observable physical phenomenon.

That there is no Gibbs paradox in classical statistical physics, is a bold claim. You have to assume extensivity of entropy (in the thermodynamic limit) of a classical many-body theorem to get the correct $1/N!$ factor in the canonical and grand-canonical ensemble to get the entropy right. Quantum theory implies this indistinguishability factor from its very foundations.

 

[autoUFC]

Extensivity does not need to be assumed. Extensivety follows from the correct definition of entropy, where the 1/N! is included by logic, not to obtain an extensive entropy, but due to combinatorial logic. Most people seem to think that this term appears to make classical statistics agree with quantum statistics in a limit. This is not the case, as the articles I mentioned show, and as the dog's flea problem reveals. The N! is no paradox, it is logically needed.

 

[autoUFC]

I guess my point is being lost. I know that quantum mechanics imposes indistinguishable particles. My point is: At the time of Gibbs, one could consider a model of identical particles that are permutable, meaning that exchanging two of them would result in a different state. Gibbs suggested that counting states in this way would not produce the correct form for the entropy. This is not true. The N! Correction is imposed due to logic. In granular media, for instance, there is the Eduards entropy of jamming, it includes the N! term, but for sure grains are distinguishble, and exchanging two grains results in a different state. I could try to explain the logic behind it, but I guess that the references I included do a better job than I would.

 

[vanhees71]

But within classical mechanics, where is the logic to include this crucial factor $1/N!$? In classical mechanics any particle is individually distinguishable from any other, because you can (in principle) follow its trajectory. So there's no "logical reason" to implement this factor other than the extensivity argument (which AFAIK Boltzmann did for the first time).

Of course today we don't have a problem with that, because we know that QT is the right theory of matter and there the indistinguishability is pretty "logical", leading to the existence of bosons and fermions (by a topological argument), and then the factor $1/N!$ of course persists in the classical limit (where both statistics get the classical Boltzmann statistics, including this factor).

 

[andresB]

But within classical mechanics, where is the logic to include this crucial factor $1/N!$? In classical mechanics any particle is individually distinguishable from any other, because you can (in principle) follow its trajectory. in the classical limit (where both statistics get the classical Boltzmann statistics, including this factor).

According to the cited paper, it comes naturally from Boltzmann definition of entropy. Which is different from Gibbs definition of entropy in terms of volume of phase space.

 

[Stephen Tashi]

The link to the Jaynes paper given by the Wikipedia article is: http://bayes.wustl.edu/etj/articles/gibbs.paradox.pdf and, at the current time, the link works.

 

[Stephen Tashi]

I'm not aware of the claim that an an "extensible entropy" (whatever you mean by that) could not be defined for distinguishable particles.

@andresB
Is the general idea of an "extensible" property that it is a numerical quantity P of matter such that P(M1 + M2) = P(M1) + P(M2), where M1+M2 indicates combining masses M1 and M2 physically?

 

[vanhees71]

You mean extensive! Sure, that's the point. If you assume that the particles are distinguishable (and in my opinion nothing in classical mechanics make particles indistinguishable) then by calculating the entropy according to Boltzmann and Planck, you get a non-extensive expression for the entropy of a gas consisting of (chemically) "identical" particles which leads to Gibbs's paradox. This paradox can only be resolved when assuming the indistinguishability of particles in the sense that you have to count any configuration which results from a specific configuration by only exchanging particles as one configuration, which leads to the inclusion of the crucial factor $1/N!$ in the canonical partition sum:
$$Z=\frac{1}{N!}\int_{\mathbb{R}^{6N}} \mathrm{d}^{6 N} \Gamma \exp[-\beta H(\Gamma)], \quad \beta=\frac{1}{k_{\text{B}} T}.$$
This was of course known by Gibbs and Boltzmann, but I still think there is no argument within fundamental classical physics to include this factor. So the textbooks are right when they say that this is an early hint that there's something not correct in the classical particle model. Today we know that this is indeed true and we need quantum theory and indistinguishability of identical particles as a fundamental property. Anyway, historically quantum theory has been discovered due to such subtle inconsistencies occurring in statistical physics (most famously the failure of the application of classical statistics to the electromagnetic radiation field in a cavity, whose solution led Planck to introduce ("in an act of desperation") the light-quantum hypothesis.

 

[Stephen Tashi]

Jaynes goes as far as to suggest that Gibbs was senile when he wrote about the paradox of mixing.

What Jaynes wrote is:

In reading Gibbs, it is important to distinguish between early and late Gibbs. His Heterogeneous Equilibrium of 1875 is the work of a man at the absolute peak of his intellectual powers; no logical subtlety escapes him and we can find no statement that appears technically incorrect today. In contrast, his Statistical Mechanics of 1902 is the work of an old man in rapidly failing health, with only one more year to live. Inevitably, some arguments are left imperfect and incomplete toward the end of the work.

It's interesting that Jayne's own book Probability Theory, The Logic of Science was left incomplete due to Jayne's death.

 

[andresB]

You mean extensive! Sure, that's the point. If you assume that the particles are distinguishable (and in my opinion nothing in classical mechanics make particles indistinguishable) then by calculating the entropy according to Boltzmann and Planck, you get a non-extensive expression for the entropy of a gas consisting of (chemically) "identical" particles which leads to Gibbs's paradox.

According to Swendsen, a non-extensive entropy does not work even for distinguishable particles

https://aapt.scitation.org/doi/10.1119/1.2174962
https://www.mdpi.com/1099-4300/10/1/15

 

[autoUFC]

These two sentences from vanshee are exactly what I meant when I wrote the this paradox is a Urban legend. In fact, there is no problem with the entropy of in the classical model. The N! term is imposed by logic. Gibbs was right in believing identical particles are impermutable (meaning that exchanging two of them leads to the same state). However, he was right by coincidence. There is no hint of something wrong in the classical particle model. These are the misconceptions that I see as urban legend. Not long ago I was one of the believers in these mistical arguments,
As van Kampen calls them.

"If you assume that the particles are distinguishable (and in my opinion nothing in classical mechanics make particles indistinguishable) then by calculating the entropy according to Boltzmann and Planck, you get a non-extensive expression for the entropy."

"textbooks are right when they say that this is an early hint that there's something not correct in the classical particle model."

 

[vanhees71]

According to Swendsen, a non-extensive entropy does not work even for distinguishable particles

https://aapt.scitation.org/doi/10.1119/1.2174962
https://www.mdpi.com/1099-4300/10/1/15

But for distinguishable particles (gases) there is measurable mixing entropy!

 

[HPt]

Hi, I'm the author of the paper Eur. J. Phys. 35 (2014) 015023 (also freely available at arXiv). As I explicity demonstrate in my paper, there is a paradox that manifests as a contradiction to the second law of thermodynamics. This contradiction does not only arise for classical distinguishable particles, but, as demonstrated in my paper, also within the quantum realm. The reason for this is that also quantum particles may be distinguishable: As an example, think of a gas of buckyball molecules where each buckyball is made up of two disinct carbon isotopes in a way that no two buckyballs are identical. Therefore, the claim that the Gibbs Paradox exposes an inconsistency of classical statistical mechanics or that the Gibbs Paradox is resolved by (the indistinguishability of identical particles in) QM is false.

 

[hutchphd]

The different Buckyballs are then, well, different. How does this show anything?? Two hydrogen atoms are identical (and untraceable in QM) A deuterium atom and hydrogen atom are not identical. Am I misunderstanding something here?

 

[HPt]

The different Buckyballs are then, well, different. How does this show anything?? Two hydrogen atoms are identical (and untraceable in QM) A deuterium atom and hydrogen atom are not identical. Am I misunderstanding something here?

You can make a gas out of pairwise different buckyballs and show that this gas (although treated strictly quantum-mechanically) suffers from the Gibbs Paradox just like a gas of classical distinguishable particles. I invite you to read my paper where I do exactly this.

 

[hutchphd]

Pairwise different? Sorry I do not know the term. Perhaps the learning curve is large for me here.

 

[HPt]

Pairwise different? Sorry I do not know the term. Perhaps the learning curve is large for me here.

Pairwise different means that you can choose any two of them (i.e., any pair) and they are always different.

 

[vanhees71]

Hi, I'm the author of the paper Eur. J. Phys. 35 (2014) 015023 (also freely available at arXiv). As I explicity demonstrate in my paper, there is a paradox that manifests as a contradiction to the second law of thermodynamics. This contradiction does not only arise for classical distinguishable particles, but, as demonstrated in my paper, also within the quantum realm. The reason for this is that also quantum particles may be distinguishable: As an example, think of a gas of buckyball molecules where each buckyball is made up of two disinct carbon isotopes in a way that no two buckyballs are identical. Therefore, the claim that the Gibbs Paradox exposes an inconsistency of classical statistical mechanics or that the Gibbs Paradox is resolved by (the indistinguishability of identical particles in) QM is false.

I'm sorry to say that I'm confused. If in the two partial volumes the gases are of distinguishable kinds (i.e., gases of different molecules/atoms) then you must get a mixing entropy. So there's nothing paradoxical in this situation, because you must get a mixing entropy and indeed you can measure it.

Where the paradox comes into play is if the gases are of the same kinds of molecules/atoms. Then there must not be a mixing entropy, and you get rid of this only by assuming that the molecules/atoms are indistinguishable in the quantum-mechanical sense (also otherwise using classical counting of "complexions"), i.e., you have to multiply the number of classical microstates by $1/N!$. Then using the Boltzmann-Planck formula you get the correct result that no mixing entropy occurs and also that the entropy of a single gas is extensive (Sackur-Tetrode formula for the entropy of an ideal gas is the exactly calculable example).

 

[vanhees71]

The different Buckyballs are then, well, different. How does this show anything?? Two hydrogen atoms are identical (and untraceable in QM) A deuterium atom and hydrogen atom are not identical. Am I misunderstanding something here?

Of course in this case the molecules/atoms are different, because they have different masses, i.e., they are distinguishable and putting a gas of $\mathrm{D}_2$ molecules into one half of the box and $\mathrm{H}_2$ molecules at the same temperature and pressure on the other side and then remove the diaphragm and wait until the gas is equilibrated, i.e., completely mixed you gain the mixing entropy. The mixing entropy in that case is
$$\Delta S=k_{\text{B}} N \ln 2.$$

https://en.wikipedia.org/wiki/Entropy_of_mixing

If in both halves of the volume are the molecules (i.e., only $\mathrm{D}_2$ or only $\mathrm{H}_2$) there's no mixing entropy.

 

[autoUFC]

The problem is with semantics. Let's us define impermutable are two particles that are identical, and when one exchange two of them the state does not change. Quantum particles are impermutable. At the time of Gibbs, one could think of permutable identical particles. Meaning that, although identical, exchanging two particles counts was two states. That would apply, for instance, to the counting of states for the jamming entropy of macroscopic grains, or the buckballs of Hjalmar. Precisely, these exemples are of near identical particles, but the classical particle model considers particles as perfectly identical but permutable.

What is suprinsing is that textbooks suggest, erroneously, that the classical model predics increase in entropy due to the mixing of identical permutable particles, when in fact there is none.

 

[vanhees71]

Of course quantum particles are impermutable but classical particles are not. That's the problem with classical statistical mechanics concerning the Gibbs paradox. It's resolved through quantum mechanics and you can "repair" the results from classical statistics by simply taking over the impermutability from quantum mechanics. I don't think that you can argue purely within classical mechanics that indistinguishable particles are impermutable. For that you need to invoke the arguments from quantum theory.

If, however, the particles are distinguishable and be it only by a "minimal difference" as in the examples with the buckyballs or gases only disginguished by having different isotopes of their atoms within their molecules, then you have the mixing entropy.

 

[HPt]

If in the two partial volumes the gases are of distinguishable kinds (i.e., gases of different molecules/atoms) then you must get a mixing entropy.

No, that's only the case if you know which particle is in which partial volume. Consider the following simple thought experiment: Take a volume, then put 1,000,000 different buckyballs in it, and now divide this volume into two equal halfs. After that you have two partial volumes, each containing a gas of approximately 500,000 buckyballs (but you don't know which 500,000 of the initial 1,000,000 buckyballs is in which partial volume). Now, since dividing the volume didn't decrease the entropy, removing this division obviously won't increase it. After all, you have exactly the same situation as before you divided the volume in the first place.
As an aside: If you knew which 500,000 buckyballs are on the left partial volume and which are on the right (say, you have measured each particle of each partial volume), then removing the partition between the two partial volumes would increase the entropy (because you would have lost the information about which particle is located in which partial volume).

 

[Stephen Tashi]

I don't know which thread participants have read Jayne's paper http://bayes.wustl.edu/etj/articles/gibbs.paradox.pdf , but the characterization of that paper as "Gibbs made a mistake" is incorrect. The paper says that Gibbs explained the situation correctly, but in an obscure way.

The paper also says that classical mechanics and QM face similar difficulties in defining an extensive entropy, so it favors @HPt 's view.

 

[vanhees71]

No, that's only the case if you know which particle is in which partial volume. Consider the following simple thought experiment: Take a volume, then put 1,000,000 different buckyballs in it, and now divide this volume into two equal halfs. After that you have two partial volumes, each containing a gas of approximately 500,000 buckyballs (but you don't know which 500,000 of the initial 1,000,000 buckyballs is in which partial volume). Now, since dividing the volume didn't decrease the entropy, removing this division obviously won't increase it. After all, you have exactly the same situation as before you divided the volume in the first place.
As an aside: If you knew which 500,000 buckyballs are on the left partial volume and which are on the right (say, you have measured each particle of each partial volume), then removing the partition between the two partial volumes would increase the entropy (because you would have lost the information about which particle is located in which partial volume).

In the Gibbs paradox setup with different kinds of gases you have a volume divided by a diaphragm and you put one kind of gas in the each of the compartments at the same temperature and pressure. Then you take out the diaphragm and wait till both gases are completely mixed. Then you get an increase of entropy, because you have a irreverible change of state.

A paradox only ockurs if you do the same with identical gases at the beginning, because then there's no change at all if one takes the indistinguishability of the particles into account, because this indistinguishability is generically quantum and contradicts classical physics.

 

[hutchphd]

The paper also says that classical mechanics and QM face similar difficulties in defining an extensive entropy, so it favors @HPt 's view.

Very nice reference. I am particularly pleased that the questions I had in my head were in fact the appropriate questions ( l confess an irrational fear of thermodynamics from my undergraduate days). It seems to me that the remaining issues all have to do with defining an absolute Entropy: is it ever necessary to look at other than changes in S? In what context is unknown information relevant if it remains unknown (but knowable in principal). Seems to me never.
There a echoes of "hidden variables" here which also worry me...

 

[autoUFC]

A paradox only ockurs if you do the same with identical gases at the beginning, because then there's no change at all if one takes the indistinguishability of the particles into account, because this indistinguishability is generically quantum and contradicts classical physics.

No paradox exists. As you say nothing change. That is exactly what is predicted by the statistical mechanics of classical particles, that was the only partcle model at the time of Gibbs. Mixing of identical permutable particles does not increase entropy.

 

[vanhees71]

It's predicted only if I take the indistinguishability of the classical particles from quantum theory into consideration. Then and only then indeed you get the correct result that the mixing of identical (I prefer to call them indistinguishable) particles does not increase entropy, while the mixing of distinguishable particles does indeed increase the entropy.

I don't know, why one would insist too much on deriving statistical physics (i.e., mostly the Boltzmann equation, from which everything else follows, including equilibrium thermodynamics) purely from classical mechanics. We all know that the only appropriate theory of matter is quantum theory anyway. This tells you that you have count identical particles as indistinguishable when counting the number of microstates consistent with a given macrostate in applying the Boltzmann-Planck equation. For details see Sects. 1.5 (for classical statistics) and 1.8 (for quantum statistics) in

https://itp.uni-frankfurt.de/~hees/publ/kolkata.pdf

 

[andresB]

It's predicted only if I take the indistinguishability of the classical particles from quantum theory into consideration. Then and only then indeed you get the correct result that the mixing of identical (I prefer to call them indistinguishable) particles does not increase entropy, while the mixing of distinguishable particles does indeed increase the entropy.

But that doesn't seems to the problems with classical statistics of distinguishable particles
https://www.researchgate.net/publication/237695485_Statistical_mechanics_of_colloids_and_Boltzmann's_definition_of_the_entropy

 

[andresB]

Not sure how much relevant are these two recent articles

https://iopscience.iop.org/article/10.1088/1367-2630/abc602
https://arxiv.org/abs/2006.12482

But at the very least they seem quite interesting.

 

[autoUFC]

I don't know, why one would insist too much on deriving statistical physics purely from classical mechanics.

As mentioned. Granular media, coloidal systems, dogs fleas, fullorene balls, are systems of permutables classical particles. They surely are not identical, but are so similar that one should consider them identical. Not identical particles in the quantum meaning that exchanging two of then does not results in a different state, but in the sense that one can not tell two of them apart. For all these systems is nessessary and possible to define a consistent entropy.
Also, there is the mathematical combinatorial problem of sets of identical elements. For instance, understanding the logic that demands the introduction of the N! correction is needed to deduce the Poisson distribution from the maximum entropy principle.
https://math.stackexchange.com/questions/2241655/maximum-entropy-principle-for-poisson-distribution

 

[autoUFC]

[No increase in entropy when mixing classical identical permutable particles is] predicted only if I take the indistinguishability of the classical particles from quantum theory into consideration.

Not true. That is an urban mith.

 

[vanhees71]

As mentioned. Granular media, coloidal systems, dogs fleas, fullorene balls, are systems of permutables classical particles. They surely are not identical, but are so similar that one should consider them identical. Not identical particles in the quantum meaning that exchanging two of then does not results in a different state, but in the sense that one can not tell two of them apart. For all these systems is nessessary and possible to define a consistent entropy.
Also, there is the mathematical combinatorial problem of sets of identical elements. For instance, understanding the logic that demands the introduction of the N! correction is needed to deduce the Poisson distribution from the maximum entropy principle.
https://math.stackexchange.com/questions/2241655/maximum-entropy-principle-for-poisson-distribution

Not true. That is an urban mith.

It's only a claim that this is an urban myth, but it's not convincing to just claim this. I think you need QT to argue for this factor $1/N!$.

Also there should be mixing entropy when the constituents of the systems in the two compartments of the Gibbs-paradox setup are distinguishable (and be it only by a seemingly "small" difference, e.g., if you have the same gas but with atoms of different isotopes). I don't know, whether there are experiments which measure such mixing entropies though. In any case I'd be interested in corresponding papers.

I've no clue what the Poisson distribution has to do with all this.

 

[hutchphd]

I don't know, whether there are experiments which measure such mixing entropies though. In any case I'd be interested in corresponding papers.

Wouldn't this constitute a "test" for the existence of hidden variables (quantum or classical)?

 

[HPt]

I think you need QT to argue for this factor 1/N!.

No, QT is not needed. In section 4 of my paper I show that there is no entropy increase when mixing distinguishable identical classical particles.