# Non-sensical negative entropy? (grand canonical ensemble)

1. Jan 13, 2012

### nonequilibrium

Hello,

I was investigating a system with N indistinguishable particles, each of which can have an energy $\pm \epsilon$, and using the grand canonical ensemble, i.e. $\Xi = \sum_{N=0}^{\infty} e^{\beta \mu N} Z_N$.

But my entropy formula is $S = \left( \textrm{a couple of \sim N  positive terms } \right) - N \ln N$. Not only is this formula not extensive, it also indicates that the entropy will get (arbitrarily) negative for large N! (Also, the formula depends on the temperature, but I'm keeping that constant.)
Note: to avoid confusion, the N that appears in the formula is actually $\langle N \rangle$.

Must this be a calculation error? The calculation is not long and I've looked through it carefully and everything is straight-forward... I'm quite confused at this point!

2. Jan 13, 2012

### Ken G

I think the more fundamental way to think of entropy is the sum of -p*ln(p) over all the possible states and their probability p. Since p<1, this is positive. Since p is proportional to N, this can look a lot like -N*ln(N) to within overall constants and terms proportional to N. I'm not sure the resolution of your specific issue, but I suspect it would be resolved by thinking in terms of -p*ln(p) rather than -N*ln(N).

3. Jan 13, 2012

### maverick_starstrider

Without knowing the specific system I can't say for sure, however, one can definitely have systems whose ground-state degeneracy is a function of N (i.e. entropy is extrinsic). These occur in what are called frustrated system, a prototypical example is spins on a kagome lattice.

4. Jan 13, 2012

### Ken G

I think there may be a confusion between the terms "extensive" and "extrinsic." The latter just means depends on N-- which entropy usually does. But "extensive" means that the entropy of two systems is the sum of their individual entropies.

5. Jan 13, 2012

### nonequilibrium

@ Ken G:

Are you referring to the Gibbs entropy? I'm not sure why: I cannot choose the entropy function myself, it follows from the grand canonical ensemble ($\Xi$, as described above) and the fact that the relevant free energy is $\Phi = E - TS + \mu N$ where $\Phi = -k_B T \ln \Xi$. In other words: I've solved for S.

@ maverick:

I've described my system in my original post; did you read over it, or do you think my description is not sufficient?

6. Jan 13, 2012

### Ken G

I'm trying to connect the entropy to a physically meaningful parameter that seems like it should be positive. Is it not true that the entropy function has physical effects only in terms of its changes, like energy, so wouldn't matter if it is positive or negative as long as the changes are correct? It seems to me the physical issue behind a grand canonical ensemble is that the creation of additional particles has an energy cost, which must be drawn from some reservoir at T and that has entropy consequences, but it also gives the system access to more states (the sum over -p ln(p) is a larger sum), and the combined system will always maximize its expected uncertainty because uncertainty is associated with likeliness, within the external conservation constraints. So it always has to boil down to maximizing the sum of -p ln(p), no matter how that result is derived. That will always be a positive quantity, so I'm wondering if there is not a way to recast the entropy function you are using in terms of a sum over -p ln(p), and assert that any physical system will maximize that quantity subject to the constraints.

7. Jan 13, 2012

### nonequilibrium

I understand your general way of thinking, but it's not even that it is negative (if it were negative by a constant I wouldn't worry as much), but how it is negative. There are two weird things:

1) the N ln N term is not extensive
2) more importantly, the - N ln N suggests that the larger you make the system (and don't forget I'm increasing E proportionally) the lower the entropy is... I can't make sense out of that.

Peculiar is that when I calculate it for distinguishable particles, I don't get this mess!

8. Jan 13, 2012

### Ken G

9. Jan 13, 2012

### Dickfore

@OP:

Would you mind showing us the steps of your calculation so that we can independently check your derivaion?

10. Jan 13, 2012

### Andy Resnick

I wonder if the problem is here- you allow particles to be either in a free (positive energy) or bound (negative energy) state. Perhaps if you instead use $E \pm \epsilon, \epsilon < E$ the problem goes away?

11. Jan 13, 2012

### nonequilibrium

Thank you both for the comments.

@ Ken G: I'm familiar with the Gibbs paradox and the Gibbs factor, but I don't think that it's the answer rather than the "problem": I know it's the Gibbs factor in the grand canonical ensemble that is giving me this, but sadly the Gibbs factor has to be included (as the wiki page says).

@ Dickfore: Sure!

So the grand canonical partition function for the system described in the original post is $\Xi = \sum_{N=0}^{+\infty} \frac{ e^{\beta \mu N} Z_N }{N!} = \sum_{N=0}^{+\infty} \frac{e^{\beta \mu N} Z_1^N}{N!} = e^{e^{\beta \mu} Z_1}$ where $Z_1 = e^{\beta \varepsilon} + e^{- \beta \varepsilon} = 2 \cosh{\beta \varepsilon}$ is the canonical partition function for a one particle system. Define for future ease $y=e^{\beta \mu} Z_1$, then $\log \Xi = y$.

We also know that the grand canonical potential $\Phi = E - TS - \mu N$ and from statistical mechanics $\Phi = -k_B T \log \Xi$, hence: $\boxed{S}=\frac{E}{T} + k_B \log \Xi - \frac{\mu}{T} N =\boxed{ \frac{E}{T} + k_B y - \frac{\mu}{T} N }$.

It would be nice to express this expression for S in terms of N, so I calculate N:
$\langle N \rangle = \frac{1}{\beta} \frac{\partial}{\partial \mu} \log \Xi = y$ which gives that the middle term in the boxed expression (i.e. k_b y) goes like N.

Now to rewrite the last term $\frac{\mu}{T} N = k_B \left( \beta \mu \right) N$ note that $\beta \mu = \log e^{\beta \mu} = \log y - \log Z_1 = \log N - \log Z_1$.

Using these two rewritings, we get that $S = \frac{E}{T} + k_B N - k_B N \log N + k_B N \log Z_1$.
(Note that $Z_1$ only depends on temperature.)

So every term except the third (-N log N) scales like N.

EXTRA CALCULATION: Let's be a bit more careful and check whether E scales as N:
$E = - \frac{\partial}{\partial \beta} \log \Xi |_{\beta\mu = \textrm{ constant }} = - e^{\beta \mu} \frac{\partial}{\partial \beta} Z_1 = - e^{\beta \mu} 2 \varepsilon \sinh \beta \epsilon = - \epsilon y \tanh \beta \epsilon = - \epsilon N \tanh \beta \epsilon$
It does.

EDIT: Resnick, thanks for your post, it appeared while I was writing my post. I don't think it would matter. So say I used $\varepsilon_0 \pm \varepsilon$, then the only thing that changes in the calculation is $Z_1 \to e^{-\beta \varepsilon_0} Z_1$ and the only place where I use the explicit form of Z_1 is in the last line to show E goes as N, and I've redone the calculation with this adapted Z_1 and it doesn't change this fact.

Last edited: Jan 13, 2012
12. Jan 13, 2012

### Ken G

I believe there is a problem with this expression. If you just give each particle one possible state, of energy 0 (if we chose something else it would just show up in the chemical potential), then you would have Z1 = 1, and Z1N = 1. Yet you divide by N!, so your result for the number of ways the system of N indistinguishable particles can be arranged in one energy state is less than 1! That isn't right, there is 1 state, not 1/N! states, for N indistinguishable particles all at energy 0.

13. Jan 13, 2012

### maverick_starstrider

On a first attempt I get

$$S = - k y + \beta \mu y + \frac{\beta \epsilon y}{\tanh (\beta \epsilon)} = \langle N \rangle \left( \frac{\mu}{kT} + \frac{\epsilon}{kT(\frac{\epsilon}{kT} + \frac{(\epsilon)^3}{3(kT)^3} + \ldots)} - k \right)$$

Which may be the same as yours (up to the error you made for E, it's 1/tanh). I don't see the problem with this. S is extensive, it goes to infinity as T -> 0 and it's never negative. Did I mess up somewhere?

Last edited: Jan 13, 2012
14. Jan 13, 2012

### maverick_starstrider

He's making a common approximation that tries to undo double counting with indistinguishable particles, it over estimates though because not all states are over counted

15. Jan 13, 2012

### nonequilibrium

Thank you all three for the replies. I don't consider the matter settled though, please read on.

@ Ken G & last post of maverick: I understand your reasoning Ken, but are you sure that it's correct? You regard Z as something that counts the number of ways of something, but I think this is incorrect. $\Omega$ in the isolated system expression $S=k \ln \Omega$ does this, and Z has been derived from it. More specifically, the formula $Z = \sum e^{\beta E_n}$ was derived from it for distinguishable particles, and when you deal with indistinguishable particles you divide $\Omega$ by N! and I think this factor (in the derivation) transfers unharmed to Z, i.e. the derivation shows that for indistinguishable particles one needs to devide Z by N!. Do you object to this chain of reasoning? It seems more exact, to me, than yours (although less intuitive, granted).

EDIT: I checked the derivation of Z from $\Omega$ and indeed the N! factor transfers unscathed.

EDIT: Upon further reflection, I've changed my mind! I'll make a new post, in case you've already read this post (and thus won't notice the important EDIT).

@ first post of maverick:
How do you get 1/tanh? At what step of my derivation of E did I make a mistake?

But that is a side-note, more importantly: I agree with your formula, but the difference is that you leave $\mu$ as it is, which seems sensical, I agree, but look at my calculation (the relevant part starts from "Now to rewrite the last term..."), but if you just want to know the result: I got that $\mu \sim \log N$. This small remark introduces the non-extensiveness which eventually leads to a negative entropy. This proportionality might surprise you -- it surprised me at least. Hopefully you might find an error in my reasoning.

Last edited: Jan 13, 2012
16. Jan 13, 2012

### kith

I'm not very used to the grand canonical ensemble and haven't looked at your calculation in detail, but maybe my thoughts are still helpful.

The grand canonical ensemble corresponds to a physical system with fixed temperature T and fixed chemical potential µ (along with other system parameters like V). The microscopic structure of the system determines the grand canonical partition function ZG(T,µ) and the grand canonical potential Φ(T,µ). Either of them determines all thermodynamical quantities.

If you're talking about S and N in this context, you are always talking about S(T,µ) and N(T,µ). So in order to change N, you have to change the system parameters T and µ. This makes the relationship between S and N non-obvious, I would say.

For the canonical ensemble, the situation is different. Here, we have T and N as system parameters. From the partition function Z(T,N) we get the entropy S(T,N) as a function of N. This means, we can change N without changing other system parameters, which should lead to extensive behaviour.

17. Jan 13, 2012

### nonequilibrium

Indeed I started with S(T,µ), but as I was interested in the dependence on N, I rewrote it in S(T,N). I rewrote any dependence on µ as a dependence on N.
But when interpreting S(T,N) I'm assuming that I can change N while keeping T constant. In other words I'm assuming that to, say, double N, I only need to let µ change. Maybe you say that this is impossible?

18. Jan 13, 2012

### nonequilibrium

Ken G: I must say I revoke my "proof". I still stand by my proof, but the N! is actually already wrong with $\Omega$, hence Z inherits its mistakes. Maverick is correct: dividing by N! overshoots, in exactly those cases you described: all the particles are in the same state, which is counted once even for distinguishable particles. My apologies for answering too soon about your objection. Thanks for making me realize this!

Do you think this might be the reason for my odd result?

EDIT: It might very well: I'm dividing by N! and then taking the log, which according to Stirling's approximation gives -N ln N, exactly the unwelcome term that I'm complaining about. (This also suggests I'd get the same weird answer with a regular canonical ensemble, also divided by N!)

19. Jan 13, 2012

### kith

No, I suggested that the µ change could be the reason for the apparent non-extensiveness of S. But I only just realized, that µ also changes in the canonical case, so there should probably be no difference between both cases due to this.

20. Jan 13, 2012

### nonequilibrium

Hey kith, thanks for your input. In any matter, I've eliminated µ completely in the expression for S, so that shouldn't be the problem.

I don't know if you read the post above your head, but I think the problem might be resolved :) It seems the reason is that the N! that one usual divides by is not exact but rather an approximation, and a horribly bad approximation in my system.

21. Jan 13, 2012

### maverick_starstrider

Well if you remove mu you're moving to the canonical ensemble which makes this the two-state paramagnet whose solution is in any stat mech book, the only catch being that they're now indistinguishable particles.

P.S. Nevermind my comment about tanh, I messed up

22. Jan 13, 2012

### nonequilibrium

Well, not exactly the canonical ensemble, but I think I get your point: they should give the same predictions for large N.

So this is all cleared up then :) Feels good. Thank you all your time.

23. Jan 13, 2012

### atyy

http://ocw.mit.edu/courses/physics/8-333-statistical-mechanics-i-statistical-mechanics-of-particles-fall-2007/lecture-notes/lec13.pdf [Broken] comments (after Eq IV.49) that the N! kludge is not needed in the two-state system because the particles are distinguished by being fixed at different points on the lattice.

Last edited by a moderator: May 5, 2017
24. Jan 13, 2012

### maverick_starstrider

Well ya, that's the classic two-state paramagnet but he said they're indistinguishable. All we know is that each individual indistinguishable particle has two possible energy states.

Last edited by a moderator: May 5, 2017
25. Jan 14, 2012

### atyy

Can such a system exist? If it does, what is the correct counting (I guess it should come from quantum stat mech)?