How can the maximum entropy and minimum energy principles be physical?

[EE18]

In Chapter 5 of his famous textbook on thermodynamics, Callen argues for the "equivalence" of the maximum entropy (Max-Ent) principle and the minimum energy (Min-En) principles. I quote from Callen first:

Entropy Maximum Principle. The equilibrium value of any unconstrained internal parameter is such as to maximize the entropy for the given value of the total internal energy.

Energy Minimum Principle. The equilibrium value of any unconstrained internal parameter is such as to minimize the energy for the given value of the total entropy.

As far as I know (though Callen never makes this explicit in what, I think, represents an error or at least an oversight) the Max-Ent principle applies if and only if the (composite) system under investigation is isolated. On physical grounds, an isolated system is always characterized by fixed energy $U$. It's with that background that I ask the following.

Callen first gives a "qualitative" proof (in one direction of the equivalence) which goes as follows:

Assume, then, that the system is in equilibrium but that the energy does not have its smallest possible value consistent with the given entropy. We could then withdraw energy from the system (in the form of work) maintaining the entropy constant, and we could thereafter return this energy to the system in the form of heat. The entropy of the system would increase ($dQ = TdS$), and the system would be restored to its original energy but with an increased entropy. This is inconsistent with the principle that the initial equilibrium state is the state of maximum entropy! Hence we are forced to conclude that the original equilibrium state must have had minimum energy consistent with the prescribed entropy.

Aside from the fact that one seems to be applying the Max-Ent principle to a non-isolated system in this "argument", I have an even bigger objection to the first line, wherein we are supposing that we are at some fixed/constrained entropy!

The "mathematical argument" seems to suffer (at least it seems to me) from that same shortcoming. See here for the very nice answer from Chemomechanics which reproduces Callen's derivation. Translating the result of that proof into words, we seem to have shown that (assuming a simple $U,V,N$ system for simplicity) if on the surface $\psi(S,U,V,N) = 0$ we are at a point such that $S$ is locally maximum at the given $U$, then that point is also locally a $U$ minimum *for that given $S$* (i.e. the $S$ corresponding to the max entropy).

My problem with the above is that Callen goes on (or seems to go on) to claim that the two principles are in every way equivalent, but this seems absurd given that each principle (on my reading of things) is applicable in completely different contexts. Callen for example analyzes the diathermal piston problem using both principles, which makes no sense to me given that the composite system of two gases is either at fixed energy or at fixed entropy (though I'm not sure how one would arrange the latter) -- but not both!

To adapt a clever (if controversial) phrase, this question can be summed up as "don't the Max-Ent and Min-En principles have nonoverlapping magisteria and, if so, how does it makes sense to talk about their being equivalent?" I am hoping someone can elucidate exactly in what sense these are equivalent. There have been many questions asked on this site to that effect but none which suitably address this last point, I think.

 

[vanhees71]

Just take
$$\mathrm{d} U=T \mathrm{d} S + X_i \mathrm{d} x^i,$$
where $X_i$ are the "generalized forces" and $x^i$ the external paramaters (e.g., for the usual example of a gas in a given container volume you have $X_1=-p$ and $x^1=V$).

Then for a closed system the energy is conserved, i.e., you have $\mathrm{d} U=0$, which implies that $S$ must be stationary wrt. variations of the $x^i$. Further from the 2nd Law it's clear the $S$ is never decreasing, i.e., the equilibrium must be the maximum entropy under the given external constraints (here energy conservation).

If you make the entropy constant the equilibrium condition is again the $\mathrm{d} U$ is stationary wrt. the variation of the $x^i$. That $U$ takes a minimum is explained by the quote of Callen.

 

[TSny]

Callen for example analyzes the diathermal piston problem using both principles, which makes no sense to me given that the composite system of two gases is either at fixed energy or at fixed entropy (though I'm not sure how one would arrange the latter) -- but not both!

For the diathermal piston problem, most of the points of the entropy surface in configuration space (Callen Fig 4.2 shown below) represent “constrained states” where the piston would have to be adiabatic to keep the system in that state.

If the piston is fixed in place and impermeable to the substances in the subsystems, then a constrained state would be characterized by the energies of the subsystems. To change the system from one constrained state to another, we would need to interact with the system to adjust the energies of the subsystems. Once set up, the system will remain in the constrained state forever if the system is isolated and the piston remains adiabatic. While in this state, we can imagine that the piston somehow becomes diathermal. The subsystems will now exchange energy and the composite system will move to some “unconstrained” equilibrium state $A$. The entropy maximum and energy minimum principles describe properties of the unconstrained state $A$.

If I’m understanding these principles, then I think they could be worded in the following way.

Entropy Maximum Principle. Let $\Omega_{U_A}$ be the set of states of the composite system that includes the unconstrained equilibrium state $A$ as well as all the constrained states that have the same energy, $U_A$, as state $A$. This is the set of states colored blue in the diagram below. Then within this set, $A$ is the state that has the maximum entropy.

Energy Minimum Principle. Let $\Omega_{S_A}$ be the set of states of the composite system that includes the unconstrained equilibrium state $A$ as well as all the constrained states that have the same entropy, $S_A$, as state $A$. This is the set of states colored green in the diagram below. Then, within this set, state $A$ has the minimum energy.

The isolated, composite system could never move on its own from one of the constrained states in $\Omega_{S_A}$ to a neighboring constrained state in $\Omega_{S_A}$. That would violate conservation of energy for the isolated system. But there's nothing in the statement of the energy minimum principle that requires the system to be able to move on its own between these two states.

 

[EE18]

But there's nothing in the statement of the energy minimum principle that requires the system to be able to move on its own between these two states.

I think this sentence is headed towards answering my confusion, but I'm not sure it's quite there.

In theory, each of these principles should be (in particular) operational "recipes" for determining the equilibrium state of a system given certain initial information, right? Certainly, that's how I've used the second law (entropy max postulate) up until this point is Callen. I'm therefore not sure it makes sense to say something like "Let $\Omega_{S_A}$ be the set of states of the composite system that includes the unconstrained equilibrium state A as well as all the constrained states that have the same entropy, $S_A$, as state A", since we don't even know what state A is a priori -- after all, we're trying to find it! Does that gripe make sense or am I being crazy here?

The entropy maximum principle makes sense to me if it is phrased as follows :
For a system which is restricted to have the same energy (as its initial state), the equilibrium value of any unconstrained internal parameter is such as to maximize the entropy for the given value of the total internal energy.

By analogy, one would think the minimum energy principle ought to be phrased as follows:
For a system which is restricted to have the same entropy (as its initial state), the equilibrium value of any unconstrained internal parameter is such as to minimize the energy for the given value of the total entropy.

But if these are correct, then (1) they are certainly not equivalent, since they have different domains of applicability, and (2) I'm not sure how the proofs actually prove one from the other. Maybe I'll need a closer reading of Callen again to see if his proof can be adapted to prove the more specific claims I've given here.

What do you think?

 

[Lord Jestocost]

From chapter 4 “THERMODYNAMIC EQUILIBRIUM“ of Nicholas W. Tschoegl’s book “Fundamentals of Equilibrium and Steady-State Thermodynamics” one reads:

“Chapters 2 and 3 developed the basic armamentarium of the theory of equilibrium thermodynamics. We are now ready to address its central problem: the conditions of thermodynamic equilibrium (§ 1.11). In particular, we consider the conditions under which an isolated composite system returns to a state of equilibrium after the lifting of an internal constraint (the removal of a barrier)….

‘At equilibrium the value of any unconstrained parameter of an isolated thermodynamic system is such that the entropy is maximized at constant internal energy’.

The entropy maximum principle thus characterizes the equilibrium state as one of maximum entropy for a given total internal energy….

The energy minimum principle characterizes the equilibrium state as one of minimum energy for a given total entropy. It reads:

'At equilibrium the value of any unconstrained parameter of an isolated thermodynamic system is such that the internal energy is minimized at constant entropy’.

The extremum principles for the internal energy and for the entropy express the condition of equilibrium of the isolated thermodynamic system in the entropy representation and in the energy representation, respectively. They are thus equivalent and may be used interchangeably….”

 

[EE18]

'At equilibrium the value of any unconstrained parameter of an isolated thermodynamic system is such that the internal energy is minimized at constant entropy’.

This is the part which has befuddled me and still makes no sense to me, I'm afraid. How can an isolated system be constant entropy!

 

[TSny]

In theory, each of these principles should be (in particular) operational "recipes" for determining the equilibrium state of a system given certain initial information, right? Certainly, that's how I've used the second law (entropy max postulate) up until this point is Callen. I'm therefore not sure it makes sense to say something like "Let $\Omega_{S_A}$ be the set of states of the composite system that includes the unconstrained equilibrium state A as well as all the constrained states that have the same entropy, $S_A$, as state A", since we don't even know what state A is a priori -- after all, we're trying to find it! Does that gripe make sense or am I being crazy here?

In section 2-4, Callen shows how to use the entropy maximization principle to find the equilibrium state $A$ for the diathermal piston problem. On page 136 in section 5-1, he uses the energy minimization principle to solve the same problem.

The entropy maximum principle makes sense to me if it is phrased as follows :
For a system which is restricted to have the same energy (as its initial state), the equilibrium value of any unconstrained internal parameter is such as to maximize the entropy for the given value of the total internal energy.

By analogy, one would think the minimum energy principle ought to be phrased as follows:
For a system which is restricted to have the same entropy (as its initial state), the equilibrium value of any unconstrained internal parameter is such as to minimize the energy for the given value of the total entropy.

But if these are correct, then (1) they are certainly not equivalent, since they have different domains of applicability,

I think the two principles are equivalent. I believe that the energy minimization principle is not implying that the entropy stays constant while the isolated system approaches the unconstrained equilibrium state $A$. In my opinion, the principle says something about state $A$ that has nothing to do with how the system got to state $A$. If state $A$ is an equilibrium state for the diathermal piston, then $A$ will satisfy both principles. I don't understand your claim that the two principles have "different domains of applicability". As pointed out above, Callen illustrates how to use either principle to solve the same problem. I could be misinterpreting what the two principles are actually saying or I could be misinterpreting why you believe the two principles are not equivalent.

and (2) I'm not sure how the proofs actually prove one from the other. Maybe I'll need a closer reading of Callen again to see if his proof can be adapted to prove the more specific claims I've given here.

You might find it interesting to look at the statement of the two principles as given by Gibbs around 1875. After stating them, he gives his short proof of their equivalence.

 

[EE18]

If state $A$ is an equilibrium state for the diathermal piston, then $A$ will satisfy both principles.

I think this is starting to get me towards understanding. You are saying that, given an isolated system, the (final) equilibrium system is that for which
1) At its value of the energy, $U_0$, it has the maximum entropy or, equivalently
2) At its value of the entropy $S_0$, it has the minimum energy.
I suppose I can accept this as being logically consistent, at least a priori.
The question is how does one operationalize this; in the course of this hopefully I can point out where 2) fails to make sense to me. Suppose I give you an isolated system at some $U_i$ and tell you that some internal constraint has been released. By 1), you can then search in configuration space (as restricted to the $U_i$ constraint) for the state which maximizes $S$ on this plane. Suppose I find that this final state has $S_f>S_i$ But if I use 2) as the method as applied to the same system, how do I proceed? The initial state has some entropy $S_i$ -- do I search on this plane of $S = S_i$ for the minimum energy state? And, if so, since this minimum energy state will have $S = S_i$ (and so cannot be the same state as I just found with the maximum entropy principle), surely (though of course I have erred somewhere since you, Callen, and Gibbs disagree with me!) I have just shown that the minimum energy principle has led me to a different conclusion than the maximum entropy principle?

You might find it interesting to look at the statement of the two principles as given by Gibbs around 1875. After stating them, he gives his short proof of their equivalence.

Thank you very much for sending this to me. Gibbs clearly says at the bottom of the first page that both principles apply to the equilibrium of an *isolated* system, which I doubted, so that at least confirms to me that I am *am* misunderstanding something (as I had thought that the resolution was to point out that only Max Ent applied to isolated systems -- that's what I meant by different domains of applicability). My confusion still remains about how Min En can possibly be used, as well as with the proof. Gibbs also argues that if the state is not max ent, then we can vary the energy at constant entropy -- but how can we vary the energy if the system is isolated!? Fixed energy is surely a constraint on any virtual variation, no?

 

[TSny]

I think this is starting to get me towards understanding. You are saying that, given an isolated system, the (final) equilibrium system is that for which
1) At its value of the energy, $U_0$, it has the maximum entropy or, equivalently
2) At its value of the entropy $S_0$, it has the minimum energy.

Yes, I think those are correct and concise statements of the principles.

I suppose I can accept this as being logically consistent, at least a priori.
The question is how does one operationalize this; in the course of this hopefully I can point out where 2) fails to make sense to me. Suppose I give you an isolated system at some $U_i$ and tell you that some internal constraint has been released. By 1), you can then search in configuration space (as restricted to the $U_i$ constraint) for the state which maximizes $S$ on this plane. Suppose I find that this final state has $S_f>S_i$

OK.

But if I use 2) as the method as applied to the same system, how do I proceed? The initial state has some entropy $S_i$ -- do I search on this plane of $S = S_i$ for the minimum energy state?
And, if so, since this minimum energy state will have $S = S_i$ (and so cannot be the same state as I just found with the maximum entropy principle), surely (though of course I have erred somewhere since you, Callen, and Gibbs disagree with me!) I have just shown that the minimum energy principle has led me to a different conclusion than the maximum entropy principle?

What you say above is correct. Searching the $U = U_i$ plane for maximum entropy will yield an equilibrium state that is actually the equilibrium state that the system would evolve into from the initial state. But, as you say, searching the $S = S_i$ plane for minimum energy will yield a state that is not the equilibrium state that the isolated system would evolve into from the initial state.

The two principles are concerned with entropy and energy planes that contain the final equilibrium state $A$, not the planes containing the initial state $i$. Of course, since the system is isolated, the energy plane containing the initial state will also be the energy plane for the final state.

The two principles state that $A$ is the state of maximum entropy in the $U_A$ plane and state $A$ is also the state of minimum energy in the $S_A$ plane. The principles give you equivalent conditions that must be satisfied in order for $A$ to be the equilibrium state.

Both principles allow you to find, in principle, the final equilibrium state for a given initial state, $i$. For example, in the diathermal piston problem with monatomic ideal gases as the subsystems, Callen shows how both principles lead to the condition that the two subsystems must have the same temperature $T_A$ in the final state. If the system was isolated while evolving from $i$ to $A$, then we know $U_A = U_i$. (Neither principle is needed to know this.) Thus, the value of the final equilibrium temperature can be calculated from $$U_A^{(1)} + U_A^{(2)} = U_A = U_i$$ or $$\frac 3 2 N^{(1)} RT_A + \frac 3 2 N^{(2)} RT_A = U_i$$ from which we can solve for $T_A$.

Thank you very much for sending this to me. Gibbs clearly says at the bottom of the first page that both principles apply to the equilibrium of an *isolated* system, which I doubted, so that at least confirms to me that I am *am* misunderstanding something (as I had thought that the resolution was to point out that only Max Ent applied to isolated systems -- that's what I meant by different domains of applicability). My confusion still remains about how Min En can possibly be used, as well as with the proof. Gibbs also argues that if the state is not max ent, then we can vary the energy at constant entropy -- but how can we vary the energy if the system is isolated!? Fixed energy is surely a constraint on any virtual variation, no?

Right, you can't vary the energy of the system if it is isolated. When the isolated system reaches state $A$ from some initial state $i$, it will remain in that state forever as long as it remains isolated. It could not even vary its state to other states with the same energy.

When Gibbs considers "all possible variations of the state of the system which do not alter its entropy", he is talking about "virtual" variations from state $A$ that would require interacting with the system to actually carry out. To me, this is just a way to say, "Consider all the states in the set $\Omega_{S_A}$ of post #3. The energy minimum principle states that $A$ will have minimum energy in this set.

 

[EE18]

The two principles are concerned with entropy and energy planes that contain the final equilibrium state $A$, not the planes containing the initial state $i$.

Allow me to begin here if you don't mind as I think it starts getting me close to understanding (I know I keep saying that but this time I promise!). If we can, let's once again consider the example I gave, and let's suppose that I am forcing us to use the Min En principle. How would we go about it? Are you saying that we would consider the $U = U_i$ plane (obviously we must use/reference the initial state $i$, since the final equilibrium state $A$ is not known a priori), and the Min En principle tells us that the final equilibrium state is the point on that plane which is *also* a minimum of the energy at their given entropy? How can we know that there is only one such point? Perhaps this is embedded in the proof given by Callen and/or the properties of the fundamental equation $S = S(U,V,N)$, but it slid by me.

The two principles state that $A$ is the state of maximum entropy in the $U_A$ plane and state $A$ is also the state of minimum energy in the $S_A$ plane.

I think what I've written above agrees with this, except that it's not clear to me how we can talk about the $S_A$ plane before knowing what $A$ is! Surely there must be some reference to $U_i$?

 

[TSny]

Allow me to begin here if you don't mind as I think it starts getting me close to understanding (I know I keep saying that but this time I promise!). If we can, let's once again consider the example I gave, and let's suppose that I am forcing us to use the Min En principle. How would we go about it?

In my previous post, I outlined an example illustrating how to find the unknown state $A$ from the known initial state using the energy minimum principle. For that example, you can use the energy minimum principle to show that the final state $A$ must be a state for which the temperatures of the two subsystems are equal. That, along with knowing that $A$ must have the same total energy as the known initial state and knowing the equation of state $U = \frac 3 2 NRT$ essentially solved the problem.

Are you saying that we would consider the $U = U_i$ plane (obviously we must use/reference the initial state $i$, since the final equilibrium state $A$ is not known a priori), and the Min En principle tells us that the final equilibrium state is the point on that plane which is *also* a minimum of the energy at their given entropy? How can we know that there is only one such point? Perhaps this is embedded in the proof given by Callen and/or the properties of the fundamental equation $S = S(U,V,N)$, but it slid by me.

I'm not sure I understand your questions here. Yes, certain properties of the fundamental equation $S = S(U, V, N)$ are required in order for the max and min principles to hold. Geometrically, this means that the entropy surface in Callen's figures must satisfy certain conditions. See his remarks regarding this on page 133.

I think what I've written above agrees with this, except that it's not clear to me how we can talk about the $S_A$ plane before knowing what $A$ is! Surely there must be some reference to $U_i$?

Even when state $A$ is unknown, we can certainly say that it will have a definite energy $U_A$ and a definite entropy $S_A$. We can still make reference to the set of all possible states of the system which have the same entropy $S_A$. Or, following Gibbs, we can talk about all possible variations in the state of the system [starting from $A$] which do not alter its entropy.

Yes, to find the final state $A$ of the system from some given initial state, you need to know some properties of the initial state (such as its energy) and some information about the system itself such as one or more equations of state.

 

[EE18]

In my previous post, I outlined an example illustrating how to find the unknown state $A$ from the known initial state using the energy minimum principle. For that example, you can use the energy minimum principle to show that the final state $A$ must be a state for which the temperatures of the two subsystems are equal. That, along with knowing that $A$ must have the same total energy as the known initial state and knowing the equation of state $U = \frac 3 2 NRT$ essentially solved the problem.

I am virtually certain I understand you, but just want to be crystal clear so I don't waste all of your time and help by not understanding in the end! To confirm, it seems in the above that you are agreeing with my earlier statement: "Are you saying that we would consider the $U = U_i$ plane (obviously we must use/reference the initial state $i$, since the final equilibrium state $A$ is not known a priori), and the Min En principle tells us that the final equilibrium state is the point on that plane which is *also* a minimum of the energy at their given entropy?" The Min En principle at fixed $S$ ends up giving us $$0 = dU_1 + dU_2 = T_1dS_1 +T_2 dS_2 = (T_1 - T_2)dS_1 \implies T_1 = T_2$$ on using closure ($dS_2 = -dS_1$) too.

I'm not sure I understand your questions here. Yes, certain properties of the fundamental equation $S = S(U,V,N)$ are required in order for the max and min principles to hold.

This is the crux so I am going to reiterate what I think is going on in the hopes of getting confirmation from you. I said "Are you saying that we would consider the $U = U_i$ plane (obviously we must use/reference the initial state $i$, since the final equilibrium state $A$ is not known a priori), and the Min En principle tells us that the final equilibrium state is the point on that plane which is *also* a minimum of the energy at its given entropy? How can we know that there is only one such point?" To reiterate, is your point about the Min En principle that on the $U = U_i$, there will be one and only one point (call it $A$ as you have been doing) where this energy $U = U_i = U_A$ is also the minimum energy $U$ for A's given value of the entropy, $S = S_A$. Put differently, there is no *other* point B which is such that $U = U_i = U_A = U_B$ and B is also such that $U_B$ is the minimum energy $U$ for B's given value of the entropy, $S = S_B \neq S_A$?

 

[TSny]

This is the crux so I am going to reiterate what I think is going on in the hopes of getting confirmation from you. I said "Are you saying that we would consider the $U = U_i$ plane (obviously we must use/reference the initial state $i$, since the final equilibrium state $A$ is not known a priori), and the Min En principle tells us that the final equilibrium state is the point on that plane which is *also* a minimum of the energy at its given entropy?

Yes. Note that since $U_A = U_i$, the $U_A$ plane must be the same as the $U_i$ plane.

How can we know that there is only one such point?" To reiterate, is your point about the Min En principle that on the $U = U_i$, there will be one and only one point (call it $A$ as you have been doing) where this energy $U = U_i = U_A$ is also the minimum energy $U$ for A's given value of the entropy, $S = S_A$. Put differently, there is no *other* point B which is such that $U = U_i = U_A = U_B$ and B is also such that $U_B$ is the minimum energy $U$ for B's given value of the entropy, $S = S_B \neq S_A$?

I don't believe that there is a state $B$ (which is distinct from $A$) with $U_B = U_A$ and that has the property that the energy of $B$ is minimum among all states which have entropy $S_B$. If $B$ did have this property, then $B$ would satisfy the energy minimum principle. But, the energy minimum principle is equivalent to the entropy maximum principle. Therefore, state $B$ would satisfy the entropy maximum principle. The entropy maximum principle is assumed to be true (Postulate II, page 27). Hence, $B$ would be an equilibrium state distinct from equilibrium state $A$. But I think it is assumed that equilibrium states are unique. That is if a constraint is released in a composite system, the system will approach a unique equilibrium state.

 

[EE18]

Yes. Note that since $U_A = U_i$, the $U_A$ plane must be the same as the $U_i$ plane.I don't believe that there is a state $B$ (which is distinct from $A$) with $U_B = U_A$ and that has the property that the energy of $B$ is minimum among all states which have entropy $S_B$. If $B$ did have this property, then $B$ would satisfy the energy minimum principle. But, the energy minimum principle is equivalent to the entropy maximum principle. Therefore, state $B$ would satisfy the entropy maximum principle. The entropy maximum principle is assumed to be true (Postulate II, page 27). Hence, $B$ would be an equilibrium state distinct from equilibrium state $A$. But I think it is assumed that equilibrium states are unique. That is if a constraint is released in a composite system, the system will approach a unique equilibrium state.

Thank you! I am going to consider this matter resolved in my head, and that is all thanks to your very careful and deliberate discussion with me. In my notes I will write up that which we have discussed, with the crux being that Min En says (effectively) that there is one point A on $U = U_i$ which is such that $U = U_A = U_i$ is the lowest energy in its given slice $S = S_A$.

I really can't tell you how much I appreciate it! As someone who does self-study, sites like this one are my only way to glean insights from folks who know much more than me, especially when I'm stuck; so, thank you again!