Boltzmann vs. Gibbs entropy, negative energy

[tom.stoer]

This article insinuates that the physics community had forgotten Gibbs entropy long ago and has used Boltzmann entropy since. Isn't this nonsense? For me it was always clear that Boltzmann entropy is problematic.

http://web.mit.edu/newsoffice/2013/its-a-negative-on-negative-absolute-temperatures-1220.html

 

[atyy]

I guess it's http://www.nature.com/nphys/journal/v10/n1/full/nphys2815.html.

Is this the preprint version http://arxiv.org/abs/1304.2066?

Eq 5, if I am reading correctly, is something like ln(number of states), which I do think is the standard definition of the microcanonical entropy. So I think they are right in saying the Boltzmann definition is the one usually assumed for the microcanonical ensemble.

 

[tom.stoer]

Hm. There are many lectures discussing why Gibbs' definition encompasses Boltzmann's definition. But I agree, some lectures (and even textbooks) miss these facts.

 

[atyy]

Yeah , eye opening for me. Does the Gibbs definition have a quantum version?

 

[tom.stoer]

The von-Neumann entropy

$S_N = -k\,\text{tr}(\rho\,\ln\rho)$

with a density operator ρ is the basis for a quantum version of the Gibbs entropy. Note that no temperature has been introduced so far.

The Gibbs entropy follows for a special density operator.

$\rho=e^{-\beta H}$

 

[atyy]

The von-Neumann entropy

$S_N = -k\,\text{tr}(\rho\,\ln\rho)$

with a density operator ρ is the basis for a quantum version of the Gibbs entropy. Note that no temperature has been introduced so far.

The Gibbs entropy follows for a special density operator.

$\rho=e^{-\beta H}$

But isn't that only for the canonical ensemble? For the microcanonical ensemble, doesn't one use

$\rho=\frac{\delta(H-E)}{\omega(E)}$

 

[tom.stoer]

But isn't that only for the canonical ensemble? For the microcanonical ensemble, doesn't one use

$\rho=\frac{\delta(H-E)}{\omega(E)}$

But this is equivalent to a ρ where only the states of a fixed E do contribute, the general expression introduced by von Neumann is still valid (I agree that the the second formula holds only for the canonical ensemble)

You have

$\rho = \omega^{-1}(E) \, \delta(H-E) = P_E \to \sum_n p_n |n\rangle\langle n| = \omega^{-1}(E) \, \sum_{n; E_n = E} |n\rangle\langle n|$

 

[atyy]

But this is equivalent to a ρ where only the states of a fixed E do contribute, the general expression introduced by von Neumann is still valid (I agree that the the second formula holds only for the canonical ensemble)

You have

$\rho = \omega^{-1}(E) \, \delta(H-E) = P_E \to \sum_n p_n |n\rangle\langle n| = \omega^{-1}(E) \, \sum_{n; E_n = E} |n\rangle\langle n|$

Yes, I agree the standard method for the microcanonical ensemble is to use the microcanonical density matrix in the von Neumann entropy (which I usually think of as the generalization of the classical Gibbs entropy). I'm not sure this is correct, but it seems that if I work in the energy basis, and there are $N$ discrete states of energy $E$, then the microcanonical ensemble gives $p(E)=1/N$ and the von Neumann entropy is $\sum_{N} - \frac{1}{N}\ln{\frac{1}{N}} = \ln{N}$. So the microcanonical entropy is something like log(number of states).

But that seems closer to Eq 5 in http://arxiv.org/abs/1304.2066, which they call the "Boltzmann entropy". Their Eq 6, which they call the "Gibbs entropy" seems different from what one would get from the microcanonical ensemble and the von Neumann entropy or what is usually called the Gibbs entropy.

 

[vanhees71]

The equilibrium distribution is always maximizing the entropy under the given contraints, which is defined as the von Neumann entropy in quantum statistical mechanics. This can be founded on information-theoretical ideas by Shannon and Jaynes.

Anyway, assuming that the correct entropy is given by the von Neumann entropy. The constraint for the microcanonical ensemble, applicable to a closed system, is that energy is precisely constant, i.e., the Statistical Operator must be given by
$$\hat{R}_{\text{can}}=\sum_{j} P_j |E,j \rangle \langle E,j|,$$
where for simplicity I've assumed that the energy eigenspace at the given eigenvalue $E$ of the Hamiltonian is spanned by a discrete set of orthonormalized eigenvectors.

The von Neumann entropy is then given by
$$S=-\mathrm{Tr} \; (\hat{R} \ln \hat{R})=-\sum_{j} P_j \ln P_j.$$
To maximize the entropy under the constraint that
$$\sum_j P_j=1$$
we must have (with $\lambda$ as the corresponding lagrange multiplyer
$$\frac{\partial S}{\partial P_j}=-\ln P_j -1-\lambda=0.$$
From this we get
$$P_j=\text{const}$$
and from the constraint
$$P_j=\frac{1}{\omega(E)},$$
where $\omega(E)$ is that dimension of the eigenspace of the Hamiltonian with eigenvalue $E$, and that we wanted to prove.

 

[atyy]

@vanhees71, I agree (except the part about Jaynes :tongue2:), but I think what people usually call the Boltzmann-Gibbs-Shannon entropy, and for which the von Neumann entropy is the quantum generalization, isn't what the authors of http://arxiv.org/abs/1304.2066 are calling the "Gibbs entropy" (Eq 6).

 

[tom.stoer]

I absolutely agree with vanhees71; this is the standard method to define the microcanonical ensemble; and eq. (1) expressed exactly this statement: the density operator of the microcanonical ensemble is

$\rho = \omega^{-1}(E) \, \delta(H-E)$

where δ is the projector to the energy eigenspace and ω is the dimension.

What I do not see is how and why from this unique (!) definition of ρ two inequivalent "definitions" S_B and S_G can be "derived".

 

[vanhees71]

The Boltzmann entropy belongs to the canonical ensemble, which applies to a system that can exchange energy with a reservoir. Then only the mean energy is fixed. The maximum-entropy principle (which can also derived from kinetic theory, which follows from many-body quantum (field) theory by applying appropriate gradient-expansion approximations to the Kadanoff-Baym equations from the 2PI-Baym functional formalism, if you don't like the information-theoretical approach a la Jaynes) then gives the corresponding canonical statistical equilibrium operator
$$\hat{R}_{\text{can}}=\frac{1}{Z} \exp(-\beta \hat{H}), \quad Z=\mathrm{Tr} \; \exp(-\beta \hat{H}).$$
In case of the grand canonical ensemble, you can also exchange one or more conserved charge-like quantities (or, in the non-relativistic case the particle number), which leads to the grand-canonical ensemble
$$\hat{R}_{\text{gc}}=\frac{1}{Z} \exp(-\beta \hat{H} + \alpha \hat{Q} ).$$
The Nature paper by Dunkel and Hilbert shows that the canonical (or grand canonical) ensemble doesn't give a good description for small systems or for systems with bounded Hamiltonians allowing occupation inversion. I think it's a nice very clearly written paper, but I don't think that this is something so new. It's clear that the canonical and grand canonical ensembles work for really large macroscopic open systems, where the fluctuations of the relevant quantities (like energy and other conserved quantities for equilibrium states) are small compared to their average (see, e.g., Landau+Lifshitz vol. 5 and Sommerfeld vol. 5).

 

[atyy]

Yes, it's not new that the microcanonical ensemble with the usual Boltzmann-Gibbs-Shannon-von Neumann entropy has problems for small systems or systems with a finite number or states. I think what they claim is that a different definition of entropy which they call the "Gibbs entropy" or "Hertz entropy" (Eq 6), does work for the microcanonical ensemble even for small systems or systems with a finite number of states, and is more general than the usual BGSvN entropy for the microcanonical ensemble.

So the usual picture is we use BGSvN entropy in both microcanonical and canonical ensembles. I think they propose the BGSvN entropy be used in the canonical ensemble, but that their "Gibbs entropy" or "Hertz entropy" should be used for the microcanonical ensemble.

 

[dextercioby]

So the Jaynes is the same Jaynes who wrote papers rebuting QED, I presume. But if he really came up with the axiomatical formulation of equilibrium statistical mechanics (which uses a variational principle for the statistical entropy), then he should get his name cleared.

 

[tom.stoer]

The Nature paper by Dunkel and Hilbert shows that the canonical (or grand canonical) ensemble doesn't give a good description for small systems or for systems with bounded Hamiltonians allowing occupation inversion.

...

It's clear that the canonical and grand canonical ensembles work for really large macroscopic open systems, where the fluctuations of the relevant quantities ... are small compared to their average.

This is clear.

What is not clear to me is how something like (5) can be "defined" or "derived" from (1). I don't understand that.

In the following they show that quantities derived from (5) are partially partially physically wrong or inconsistent.

On page 8 they write

This contradiction corroborates that S_B cannot be the correct entropy for quantum systems

This was clear to me since my first course in statistical mechanics.

So my summary is that they write down (5) which I don't understand and which I cannot relate to the density operator (1), and that they derive inconsistencies which do not come as a surprise.

 

[atyy]

What is not clear to me is how something like (5) can be "defined" or "derived" from (1). I don't understand that.

I believe their Eq 5 is something like what you get from the microcanonical ensemble and the von Neumann entropy. I posted a rough sketch in post #8 that the microcanonical ensemble and the von Neumann entropy gives an entropy that is log(number of states). Their Eq 5 is log(sum over states with energy E), so it's essentially the same. So their Eq 5 is what you and vanhees71 and most people call the Gibbs entropy or the Boltzmann-Gibbs-Shannon-von Neumann entropy (-∫plnp).

Their Eq 6 is the unconventional proposal. Although they call it the "Gibbs entropy", it is not what you call the Gibbs entropy.

 

[marmoset]

So the Jaynes is the same Jaynes who wrote papers rebuting QED, I presume. But if he really came up with the axiomatical formulation of equilibrium statistical mechanics (which uses a variational principle for the statistical entropy), then he should get his name cleared.

I know this comment was tongue in cheek, but for the record Jaynes' work on QED led to him proposing the now extremely widely used Jaynes-Cummings model, which is "of great interest in atomic physics, quantum optics, and solid-state quantum information circuits, both experimentally and theoretically" (quote from http://en.wikipedia.org/wiki/Jaynes-Cummings_model).

Edit: If you're interested Jaynes' papers on statistical mechanics and information theory can be downloaded here http://bayes.wustl.edu/etj/node1.html (the original papers are items 9 and 10 on the page linked to).

 

[tom.stoer]

atty, if I understand you correctly then for fixed E (1) and (5) contain a sum

$\sum_{n,E_n = E}\ldots$

which agrees with (1) and which therefore agrees with the standard definitions for both ρ and S.

If I understand (6) correctly then due to the step function the sum becomes

$\sum_{n,E_n \le E}\ldots$

But then I don't understand the following:

ρ from (1) restricts the sum to the subspace E, therefore the sentence "Given the MCE operator (1) , one can find two competing ..." cannot be correct. (6) cannot be derived from (1) simply b/c the subspace in the sum is different.

 

[atyy]

atty, if I understand you correctly then for fixed E (1) and (5) contain a sum

$\sum_{n,E_n = E}\ldots$

which agrees with (1) and which therefore agrees with the standard definitions for both ρ and S.

If I understand (6) correctly then due to the step function the sum becomes

$\sum_{n,E_n \le E}\ldots$

But then I don't understand the following:

ρ from (1) restricts the sum to the subspace E, therefore the sentence "Given the MCE operator (1) , one can find two competing ..." cannot be correct. (6) cannot be derived from (1) simply b/c the subspace in the sum is different.

Yes, that is my understanding too - Eq (6) cannot be derived from (1) and the von Neumann entropy. Eq (6) is their proposal for a new definition of entropy in the microcanonical ensemble, that they suggest will to rectify the failure of the standard von Neumann entropy in the microcanonical ensemble for small systems or a system with finite range of energy.

 

[tom.stoer]

OK, I think I got it; was essentially a confusion regarding Gibbs-Boltzmann-von-Neumann-... entropies ;-)

 

[vanhees71]

Well, I've to look that issue up again, but the microcanonical ensemble is usually defined as follows.

Let $|n \rangle$ be a complete set of orthonormal energy eigenvectors. Then the microcanonical ensemble is defined by
$$\hat{R}=\frac{1}{\omega(E,\Delta E)} \sum_{n, E_n \in (E-\Delta E,E)} |n \rangle \langle n|$$
with
$$\omega(E,\Delta E)=\sum_{n,E_n \in (E-\Delta E,E)}.$$
This is the maximum entropy under the constraint that the energy of the system is strictly in an interval $$(E-\Delta E,E)$$.

The entropy is given by the von Neumann expression
$$S_{\text{MC}}=-\mathrm{Tr} \; \hat{R} \ln \hat{R}=\ln \omega(E,\Delta E).$$

The definition of the entropy by Dunkel an Hilbert also refers to the von Neuman entropy maximized under the constraint that the energy is strictly below the value $E$, i.e., the Statistical operator is given by
$$\hat{R}_{\text{DH}}=\frac{1}{\Omega(E)} \sum_{n, E_n \leq E} |n \rangle \langle n| = \sum_{n} \Theta(E-E_n) |n \rangle \langle n|$$
with
$$\Omega(E)=\sum_n \Theta(E-E_n), \quad S_{\text{DH}}=\ln \Omega(E).$$
Then of course, the temperature is strictly positive, if defined by the thermodynamic relation
$$\beta=\frac{1}{T} = \frac{\partial S}{\partial E},$$
where all the external parameters (like the volume of a container of a gas) are kept constant when taking the derivative, i.e., for a gas $S=S(E,V)$. In the thermodynamic limit the usual definition of the of the microcanonical and the Dunkel-Hilbert definition of the thermodynamic quantities coincide asymptotically for large systems.

BTW: I guess you refer to the arXiv version of the paper, which is a bit more detailed than the published version in Nature.

I've just found the following inteteresting reference about this issue, where this is explained in much more detail:

R. B. Griffiths, Microcanonical Ensemble in Quantum Statistical Mechanics, Journal of Mathematical Physics 6, 1447
http://dx.doi.org/10.1063/1.1704681

 

[tom.stoer]

Thanks for your summary.

This is what I understood from the paper (arxiv version) as well, but I see a severe problem with the expectation value if the energy. For fixed E we find

$$\langle H \rangle = \text{tr}(\rho H) = \frac{1}{\Omega(E)}\sum_{n,\,E_n\le E}E_n < E$$

That means that a microcanonical ensemble defined via the energy E has an energy <H> which is strictly smaller than E. What does that mean?

 

[atyy]

Thanks for your summary.

This is what I understood from the paper (arxiv version) as well, but I see a severe problem with the expectation value if the energy. For fixed E we find

$$\langle H \rangle = \text{tr}(\rho H) = \frac{1}{\Omega(E)}\sum_{n,\,E_n\le E}E_n < E$$

That means that a microcanonical ensemble defined via the energy E has an energy <H> which is strictly smaller than E. What does that mean?

I think they suggest that Eq (1) is still valid, so the density matrix for the microcanonical ensemble is not changed. So I think one should still get Tr(ρH)=E.

I believe they suugest that it is the von Neumann entropy formula that is wrong, and they replace it with their Eq (6), ie. their suggestion is that everwhere that one usually uses the von Neumann formula for entropy S, to replace it with their Eq (6). But all other equations are unchanged.

OK, I think I got it; was essentially a confusion regarding Gibbs-Boltzmann-von-Neumann-... entropies ;-)

Yes, they chose very confusing terminology! They could have called Eq (6) Gibbs-Hertz entropy and Eq (5) Boltzmann-Gibbs-von Neumann entropy.

 

[tom.stoer]

I think they suggest that Eq (1) is still valid, so the density matrix for the microcanonical ensemble is not changed. So I think one should still get Tr(ρH)=E.

I believe they suggest that it is the von Neumann entropy formula that is wrong, and they replace it with their Eq (6), ie. their suggestion is that everwhere that one usually uses the von Neumann formula for entropy S, to replace it with their Eq (6). But all other equations are unchanged.

So for <H> you use ρ, but for other expectation values you don't? For <H> only states with E do contribute, but for thermodynamical quantities states with smaller E are required? How can a state |n> with E_n < E contribute to entropy if it is never occupied, i.e. if p_n = 0?

That's cheating.

I think they suggest ... I believe they suggest ...

a lot of guesswork ...

 

[atyy]

So for <H> you use ρ, but for other expectation values you don't? For <H> only states with E do contribute, but for thermodynamical quantities states with smaller E are required? How can a state |n> with E_n < E contribute to entropy if it is never occupied, i.e. if p_n = 0?

That's cheating.

No, I think they use ρ for all expectation values. Above Eq 3, they state "For brevity, we denote averages of some quantity F with respect to the MC density operator by <F>  = Tr[Fρ]."

If I understand correctly, the difference with the standard approach is they do not use the von Neumann entropy formula, but instead use their Eq 6.

 

[tom.stoer]

atty, if they use δ(H-E) as density operator but their definition of entropy, then there are states which are strictly unoccupied (their probability is strictly zero), but which do contribute to entropy. That's more than strange.

Suppose you have two systems with identical spectrum in an energy range E > E₀ but different spectrum below E₀. Now using their approach you can have ensembles at E > E⁰ with identical qm properties, i.e. <A> = tr(ρA), but different thermodynamical properties resulting from states which are not occupied in these ensembles. In addition it makes a difference whether a state is strictly unoccupied or whether it does not exist at all.

 

[atyy]

atty, if they use δ(H-E) as density operator but their definition of entropy, then there are states which are strictly unoccupied (their probability is strictly zero), but which do contribute to entropy. That's more than strange.

Suppose you have two systems with identical spectrum in an energy range E > E₀ but different spectrum below E₀. Now using their approach you can have ensembles at E > E⁰ with identical qm properties, i.e. <A> = tr(ρA), but different thermodynamical properties resulting from states which are not occupied in these ensembles. In addition it makes a difference whether a state is strictly unoccupied or whether it does not exist at all.

Yes, I agree it's strange. Your second remark also seems right to me.

 

[atyy]

Assuming their theory is consistent, I assume its correctness should be checked by experiment?

I have to confess I usually imagine small systems are not governed by thermodynamics. Why should an isolated spring and mass system obey thermodynamics?

 

[tom.stoer]

Assuming their theory is consistent, I assume its correctness should be checked by experiment?

I don't say it's inconsistent. All what I am saying is that it's strange, it violates my understanding of physics

I have to confess I usually imagine small systems are not governed by thermodynamics. Why should an isolated spring and mass system obey thermodynamics?

I agree.

 

[tom.stoer]

I found some papers discussing the Hertz entropy based on Θ(E-H) and possible new definitions of entropy replacing von Neumann's.

https://sites.google.com/site/entropysurfaceorvolume/home/references
http://arxiv.org/pdf/0709.1071.pdf
http://arxiv.org/pdf/1301.0233v2.pdf
http://arxiv.org/pdf/0803.0282v1.pdf

 

[Jano L.]

This call for universal redefinition of entropy of a closed system seems very strange to me too. For a typical system of thermodynamics enclosed in a box, the area of available phase space hypersurface will grow with energy (think of ideal gas). The old definition of entropy for most many-particle systems seems consistent with classical thermodynamics and is well motivated in probabilistic ideas (accessible states...)

For a system of states whose density of states decreases with energy, this standard definition gives entropy that does not grow with energy. For the authors this is somehow sign of a defect of the concept and they propose another definition of statistical entropy $S_{new} (E)$ that maintains monotonicity, and so they say is generally better.

But there is no theoretical justification for such requirement when the systems have decreasing density of states. It is not clear that such systems fall under the purview of classical thermodynamics at all. If not, it makes no sense to impose requirement of thermodynamic properties upon statistical entropy and it is better to stay with the old definition for its other well-known nice properties.

For example, take multi-level system $\mu$ with decreasing density of states $D(E)$ and highest energy $E_h$.

The standard statistical entropy $S(E)$ decreases with energy for this system, and when the energy is $E_h$, it is minimum. Now let us connect $\mu$ thermally to a large system $B$ with ordinary density of states (increasing with energy) and Boltzmann temperature $T$. Since in equilibrium the total system will have such occupation probabilities as to maximize total multiplicity for total energy of the sum system, the $\mu$ system will most probably transfer some heat to $B$, and itself fall down to lower average energy (and higher standard entropy).

In other words, the system $\mu$ behaves as universal source of heat, regardless of the temperature $T$ of the receptor. Such systems do not fall under classical thermodynamics. They do not have thermodynamic temperature (in the original sense).

It may be interesting to study such systems with methods of statistical physics, and even define some new statistical entropy concept and new temperature concept for them for convenience. It would not be the first case (consider how many entropies are there already). Perhaps it is even possible to use thermodynamic $formalism$ for closed collection of such systems.

But it seems doubtful that this necessitates change of the standard statistical entropy for systems it was originally designed for.