What is the connection between Maxwell's Demon and Liouville's Theorem?

[Ken G]

Now I understand where the idea that high energy photons are needed came from. This is a good point, the cannonball gas without photons is not in equilibrium, and Maxwell's demon is not operating on equlibrated gases.

I actually don't see that at as a good point, and I'm not sure those source aren't getting hung up on fairly irrelevant issues. The concept of equilibrium must be an effective, not literal, concept in physics-- we must be able to talk about equilibrium of what we care about, not necessarily equilibrium of everything. The second law is useless if we interpret it as a law that only applies in strict equilibrium, because no such situation exists anywhere. The patent office would have to reopen all the cases of perpetual motion machines that they refused to consider on the grounds that they violated the second law, if the inventor says "but my invention works in the corner of a room, where there is no strict thermodynamic equilibrium". The second law does not say that "entropy increases in strict thermodynamic equilibrium", because in strict equilibrium nothing changes, including entropy. Instead, the second law is about characterizing small deviations from equilibrium, and only in regard to whatever is the critical mode of behavior that is under study. For example, the universe is bathed in a very low-temperature neutrino bath, and no gas at normal temperature is in equilibrium with that. Saying that non-equilibrated photons in the cannonball gas invalidates the second law would be like saying that neutrinos always invalidate it. But actually this is only true in situations where the thermal neutrinos matter in some way, and it would only matter for the Demon and the cannonballs if thermal photons were somehow being used in the apparent entropy violation.

I don't know how those sources are arguing their points, but the summaries sound wholly unconvincing to me. I see the situation as much simpler than worrying about whether brains remember or forget, or whether they are or are not bathed in hot photons or hot neutrinos. It is simply that if you want to make a decision regarding some information, you have to be able to process that information, and information processing requires an environment. That environment will necessarily increase its entropy to at least "copy" the information being processed, and even then only if it is perfectly efficient. So the Demon will always generate more entropy than it destroys, and it will never matter whether the Demon is being bathed in photons or if it has a memory. Saying more probably requires having some specific model for how the Demon manages to "think", but much of the purpose of thermodynamics is being able to say things independently of the details of that kind of model, just as we don't have to talk about the mass of the cannonballs or how much they compress before they elastically rebound.

Landauer says there is an entropy cost in erasing information. Bennett agrees, and says this saves the second law in the case of a forgetful Maxwell's demon. I accepted this in my analysis of KenG's "coin entropy" scenario. I tend to think this is true. If you forget information, then there is missing information, which is information entropy, which is thermodynamic entropy.

Yes, I agree that one cannot get rid of entropy by forgetting. But what this says is that remembering or forgetting is irrelevant. One cannot argue the Demon is cyclic or in steady-state by letting it forget, because the entropy increase is in the Demon's environment, not its memory. One can treat the memory as part of the environment, but if forgetting occurs, then it's not the whole environment. All violations of the second law invariably boil down to not considering the whole environment, and I'd say that's just what is going on here.

 

[Rap]

I actually don't see that at as a good point, and I'm not sure those source aren't getting hung up on fairly irrelevant issues. The concept of equilibrium must be an effective, not literal, concept in physics-- we must be able to talk about equilibrium of what we care about, not necessarily equilibrium of everything.

Agreed - I didn't mean to imply that lack of equilibrium with the photons was deal-killing, only that it was something that needed to be thought about, and hopefully rejected. I tend to agree with you - it can be rejected.

It is simply that if you want to make a decision regarding some information, you have to be able to process that information, and information processing requires an environment. That environment will necessarily increase its entropy to at least "copy" the information being processed, and even then only if it is perfectly efficient. So the Demon will always generate more entropy than it destroys, and it will never matter whether the Demon is being bathed in photons or if it has a memory.

You are assuming that information entropy and thermodynamic entropy are equivalent via Boltzmann's constant, which I agree with, but apparently others do not.

Yes, I agree that one cannot get rid of entropy by forgetting. But what this says is that remembering or forgetting is irrelevant.

Yes.

But let me ask, just to be very clear - Do you think that a demon could successfully reduce the entropy of the two systems, with the idea that the second law is not violated due to the increase in entropy of the demon?

 

[Ken G]

You are assuming that information entropy and thermodynamic entropy are equivalent via Boltzmann's constant, which I agree with, but apparently others do not.

Yes, but to me that's the point of the Demon-- to show that they must be equivalent. Otherwise, we can substitute one for the other, and it's bye-bye second law. Perhaps those who don't see the equivalence don't think the second law is correct, but I'm not buying those wares. I hold that if one has a model for how the information is actually being processed, including the environment that is doing it, then the equivalence between information and thermodynamic entropy becomes clear-- it all boils down to the need for spontaneous results to be those which have a higher probability of occurring, and the sole reason they have a higher probability is that they represent larger entropy.

But let me ask, just to be very clear - Do you think that a demon could successfully reduce the entropy of the two systems, with the idea that the second law is not violated due to the increase in entropy of the demon?

Yes, the Demon can separate hot and cold gases, and it can do it by increasing the entropy even more in whatever environment the Demon uses for a brain, but the energy scale of the Demon can be vastly less than the energy scale of the gas. The latter would just mean that the Demon has a lower effective temperature. Hence the Demon cannot be born from the thermodynamic equilibrium it is disrupting, it needs some separate existence that must also be analyzed-- not treated like some magical process of intention, and not just some program and memory, but a whole thermodynamic environment of its own. But the Demon can be used to extract heat from a gas-- indeed there are many ways to do that, which violate neither the first nor second laws.

 

[AdrianMay]

OK let me quote from that paper.

First the top line: Bennet, 2011.

Then we have:

One of the main objections to Landauer’s principle,
and in my opinion the one of greatest merit, is that raised
by Earman and Norton [3], who argue that since it is not
independent of the Second Law, it is either unnecessary
or insufficient as an exorcism of Maxwell’s demon. I will
discuss this objection further in the third section.

So then we excitedly skip to the third section:

Earman and Norton have pointed out with some jus-
tice that Landauer’s principle appears both unnecessary
and insufficient as an exorcism Maxwell’s demon, be-
cause if the Demon is a thermodynamic system already
governed by the Second Law, no further supposition
about information and entropy is needed to save the
Second Law. On the other hand, if the Demon is not
assumed to obey the Second Law, no supposition about
the entropy cost of information processing can save the
Second Law from the Demon.
I would nevertheless argue that Landauer’s princi-
ple serves an important pedagogic purpose ...

That means he agrees. Pedagogic purposes are not logic.

You guys know me! Would I ever presume to question what a giant like Bennet says ?

 

[lugita15]

Yes, the Demon can separate hot and cold gases, and it can do it by increasing the entropy even more in whatever environment the Demon uses for a brain, but the energy scale of the Demon can be vastly less than the energy scale of the gas.

One of the formulations of the second law is the amount of usable energy in the universe can only decrease. Usable energy is essentially the kind of energy that a low-entropy system has, and entropy is essentially a measure of what percentage of the universe's energy isn't usable. So if Maxwell's demon isn't expending that much energy, is he increasing the amount of usable energy in the universe?

To put it another way, can Maxwell's demon extract more usable energy than a Carnot engine? If we take the fumes coming out of the (ideal) combustion engine of a car, can we feed it into Maxwell's demon and get more energy out? (Let the molecules of the car fumes be cannonballs... It's a giant car, OK?)

 

[AdrianMay]

I also agree that we don't care about equilibrium - that's for PM machines to worry about, not us.

I'd still like to quibble this enormous temperature business. If we don't care about equilibrium, I can use balls from the fridge. They don't have to move very fast so there's no relativistic doppler effect to worry about, so I think they will look to the demon like normal cool balls. He won't see them glowing like a quasar.

You guys keep saying "the entropy of the demon will increase." I'd like to see one piece of evidence for that which does not use the 2nd law as an assumption. This is E+N's point, and apparently Bennet accepts it. Without using the second law, there's no argument saying that there will be any increase of entropy anywhere. All that stuff (whether you think observing or forgetting is associated with +ve or -ve entropy or whatever) sits on top of the assumption that the demon does not break the 2nd law, so you can hardly turn it around and use it to defend the 2nd law.

I think Ken is saying there will be something else defending the 2nd law from the demon if we think hard enough about it, and then all that information theory will be valid. But my mind is blank. Yes we must examine the whole system including the demon's brain, and we'd be home and dry if we could prove (without invoking the 2nd law) that this brain must increase entropy somewhere else. But how? I'm at a loss as to how this should work. Presumably there's some light in the system so he can see what's coming, but I think I can let that light be totally disordered from start to finish, so no problem there. I think I can make his brain a steady state so no problem there either.

I think it might be helpful to get the whole temperature thing out of the way because it's just causing distraction. How about a gas of aromatic molecules in a mixture of left handed and right handed forms. We don't give a monkey's about the temperatures - we just want to sort the two forms. The chemical reflects circularly polarised light in the matching handedness. (A bit of detail is called for here but I think it can work.) So if the demon sees a lot of right-circular light in his vicinity he opens he door. We can use oodles of light because we no longer care about the temperature.

If there are no losses to the environment, the light energy won't get lost but it will tend to turn from a few high energy photons to a large number of low energy photons. But I think we can probably arrange for the interesting mode (the one where the chirality plays a role) to be the lowest frequency mode available. Failing that, we only need to show that we can sort a molecule without degrading a useful photon into two lower energy ones.

If this light stuff fails we can also make a chemical detector, like a nose.

I'm back to square one now - there may be losses, but I don't see how I can equate them with the entropy reduction without invoking the 2nd law.

 

[Rap]

This all seems to be making a mountain out of a molehill. The demon and the two gases are to be considered a single system. If an outside observer looks at the system, measuring only the thermodynamic states of the two systems and the state of the demon - a combination of logical state, mechanical state, and thermodynamic parameters, the outside observer has to ask, how many ways could such a set of measurements occur? Equivalently, how many microstates could yield this observation? The entropy will be k log(W) where W is that number of ways. The second law says that will increase (or stay the same, but probably not).

The only way the demon is going to violate the second law is if the number of ways the demon could give rise to our measured state of the demon is less than the number of ways the entropy of the two gases could have been reduced. I think the second law will hold, so the number of ways will be larger, but part of these ways is a description of the logical state of the Turing-machine demon. That means that information theory must be in part invoked in the calculation of the state of the demon. Information theory applied to the "logical state" of the demon is just a way of figuring the contribution to the calculation of the total number of ways the demon could exist in his final state. If the logical state of the demon is the same as before it made the decision, and the mechanical parts are all in the same place, then heat must have been generated.

This all turns on what we define as the "macrostate of the demon". The crux of the problem is here, I think.

OK let me quote from that paper.

First the top line: Bennet, 2011.

...

So then we excitedly skip to the third section:

Earman and Norton have pointed out with some jus-
tice that Landauer’s principle appears both unnecessary
and insufficient as an exorcism Maxwell’s demon, be-
cause if the Demon is a thermodynamic system already
governed by the Second Law, no further supposition
about information and entropy is needed to save the
Second Law. On the other hand, if the Demon is not
assumed to obey the Second Law, no supposition about
the entropy cost of information processing can save the
Second Law from the Demon.
I would nevertheless argue that Landauer’s princi-
ple serves an important pedagogic purpose ...

That means he agrees. Pedagogic purposes are not logic.

You guys know me! Would I ever presume to question what a giant like Bennet says ?

Ok, I'm still trying to understand these papers, and if I can summarize the above, I read that E&R point out that Landauer's principle is based on the second law and therefore applied to the demon, it will of course save the second law. Bennett agrees. But what does it mean to save the second law? What does it mean to "exorcise the demon"? Does it mean that the demon cannot reduce the entropy of the two gases (i.e. will fail in its purpose) or does it mean that it will (or can sometimes) succeed because the entropy of the demon increases as the entropy of the two gases decreases, yielding a net increase in entropy?

I think they mean the latter.

If the demon can succeed in reducing the entropy of the two gases, then what are the details of how its entropy increases? It seems to me that Landauer is giving an information-theoretic explanation of how this happens, consistent with the second law. Earman and Norton have three objections to Landauer's principle -

1) (pages 14-16) seems to me to be "who cares about the details, Landauer's principle upholds the second law, so everything is fine". Since I care about the details, I reject that.

2 and or 3) (pages 16 - 20) I could not find a delineation of two separate problems here, its all mushed together. I definitely do not understand these pages in detail. To begin with, E&N say Bennett requires that at the end of each decision by the demon, the demon (a Turing machine) must be reset to its original state, which involves erasure and the creation of entropy by Landauer's principle. (I don't think that's necessary, but ok, let's go with that.) They then give a case where erasure supposedly does not happen. They say that Zurek and Caves (Z&C) dealt with this problem by a more complicated definition of entropy involving algorithmic complexity of the Turing machine demon. etc. etc. etc. That's as far as I got.

One question I have is "are they assuming the temperature of the gases and the demon are the same or not?"

 

[Rap]

How about a gas of aromatic molecules in a mixture of left handed and right handed forms. We don't give a monkey's about the temperatures - we just want to sort the two forms.

This would avoid lugita15's problem which needs to be considered, if not just to dispose of it.

Note that the two actually can in principle be separated. There can exist in principle, a membrane that is permeable to one, but not the other. If you have a volume of mixture, you put the membrane at one end and start slowly pushing on it. It gets to the middle, and the pressure on the pushed side is lower than on the other side. You replace the membrane with an impermeable one, let the pressures equilibrate, put in the permeable membrane, and push again. This process will eventually "distill" one type from the other. When you calculate up the total work done by pushing, divide by temperature, you will get the new (lowered) entropy of the system. The entropy (and energy) that the system has lost has gone into the pushing machine. If its your muscles, they will heat up, etc.

One of the formulations of the second law is the amount of usable energy in the universe can only decrease. Usable energy is essentially the kind of energy that a low-entropy system has, and entropy is essentially a measure of what percentage of the universe's energy isn't usable. So if Maxwell's demon isn't expending that much energy, is he increasing the amount of usable energy in the universe?

To put it another way, can Maxwell's demon extract more usable energy than a Carnot engine? If we take the fumes coming out of the (ideal) combustion engine of a car, can we feed it into Maxwell's demon and get more energy out? (Let the molecules of the car fumes be cannonballs... It's a giant car, OK?)

Hmmmm

 

[AdrianMay]

We should be able to express all this in the terms lugita is asking for, but I'm not sure how. If we dodge it with my aromatic gas for now, we have to come back to it. He's quite right that usable energy is an important consequence of this whole entropy thing, but I lost track of that when it turned out that I couldn't make a power station out of my cannonball gas. I thought I would be able to because of the usable energy thing.

If we can clarify this, we might get a line of sight on the business of how much work the demon will have to do, i.e., how much energy needs to be pumped into keep him running, and I think that'll turn out to be the crux of it.

> what does it mean to save the second law?

I think it means to come up with a catch in the demon challenge, without resorting to the second law. The catch ought to be mechanistic or statistical, and it should show that this demon system will observe the 2nd law of its own accord. Then, all that Landauer stuff would rest on the 2nd law and be valid. But that's not happening right now, rather, they are calculating the performance limits of the demon such that it can do all things permitted under the 2nd law but nothing that's forbidden.

> the outside observer has to ask, how many ways could such a set of measurements occur? Equivalently, how many microstates could yield this observation?

Now we have a precise definition, and I think it's the correct one. But I see no reason why the demon needs more states in its entire life cycle than you can count on the fingers of one hand, so how could it contribute anything significant?

 

[Rap]

Now we have a precise definition, and I think it's the correct one. But I see no reason why the demon needs more states in its entire life cycle than you can count on the fingers of one hand, so how could it contribute anything significant?

No not the states in its life cycle, the number of microstates that could give its present macrostate. For a particle gas, classically, a microstate is the specification of the position and momenta of each particle in the gas. The macrostate is three thermodynamic parameters, like temperature, pressure, and number of particles.

For the demon, the microstate is the position and momenta of every molecule that makes up the demon. If the demon is a computer that measures nearby gas particle positions and momenta, makes some calculations using some mechanism, and then opens or shuts a trap door, and generates some heat, then what is its microstate? I think you can describe its microstate by the state of the registers in it and the position and momenta of every particle that would be considered "heat" but for the fact that we now know them. The "state of its registers" is where information theory steps in and contributes knowledge of the number of microstates the demon could be in. If we follow Bennett and require that the demon be in the same logical state after a measurement-trapdoor opening as it was before, then, if the second law holds, the whole calculation process had to generate some heat, and that heat represents the entropy generated by the demon. Landauer's principle is more specific, it says that erasure causes heat. So we want to look at the whole process by which the demon arrives at its conclusion and does what it does, and count up the erasures and add up the heat and hopefully say that it all fits together. I think it is the details of this process that Bennett and E&N are discussing.

This has, of course, ignored the contributions to entropy of the measurement process and the trapdoor opening process, which we are more or less ignoring, and maybe we should not be. Szilard's principle says the measurement process produces entropy, I think Bennett says no, not necessarily, me, I don't know, but if the inquiry into the demon's entropy production fixes things, then it doesn't matter. The trapdoor opening and closing - well, I'm willing to ignore that for the same reason.

The crux of the problem, and the thing that puzzles me, is the macrostate and microstate of the demon. It seems we can make different assumptions here, and get different results, but I expect all those results are in accord with the second law. Entropy is "missing information" and depending on how you define the demon's macrostate, you get different entropies. Its should be all ok nevertheless, though. Even with gases, you can suppose different levels of information knowledge, thus get different macrostates for the same situation, thus get different entropies, but no matter which description you use, the second law still holds.

 

[AdrianMay]

I think I can describe the microstates of my dog-leg protein. If the molecule is in a given macrostate (e.g. open, closed, thinking about it, etc) then the microstates are just the waqys of arranging thermal phonons over the molecule in that configuration and at that temperature. This is just a function of temperature if we're considering a specific configuration. Different configurations have different specific heats.

I think it's that simple. You'd only get inflation of the microstates per macrostate if the temperature was increasing. But we haven't shown that it will, at least, not without invoking the second law as Landauer, Bennet and friends do.

So I don't see a route in that line of thought. It still boils down to finding some mechanical reason why the demon has to get hotter. Then we'd be home and dry. Even if the demon then dissipated his heat into the chambers, we'd still be home and dry because that would still be the sought entropy increase.

But why can't he just be frictionless and flout the second law, leaving Landauer etc without a leg to stand on?

 

[Delta Kilo]

I think I can make his brain a steady state so no problem there either.

I'd like to see that :)

I'm pretty sure it's impossible and this is exactly the reason why the whole setup is not going to work.

If your computations return to back to exactly the same state, they are reversible and there is no particular reason for them to go forward, they are just as likeky to run backwards and release the molecules instead of capturing them.

 

[AdrianMay]

I'd like to see that :)

If your computations return to back to exactly the same state, they are reversible and there is no particular reason for them to go forward, they are just as likeky to run backwards and release the molecules instead of capturing them.

Hmm. Maybe there's something in that.

 

[Ken G]

One of the formulations of the second law is the amount of usable energy in the universe can only decrease. Usable energy is essentially the kind of energy that a low-entropy system has, and entropy is essentially a measure of what percentage of the universe's energy isn't usable. So if Maxwell's demon isn't expending that much energy, is he increasing the amount of usable energy in the universe?

The energy is already usable, because you have a container filled with hot gas (the temperature is associated with the average kinetic energy of the cannonballs, AdrianMay-- that doesn't mean the balls themselves are hot, indeed we can imagine they are at absolute zero internal temperature for all the difference it would make here), sitting in an environment of lower T. If you don't have that, say the whole universe is at that T, then the Demon's environment is also at that T, and then the Demon cannot work (it needs to be at a much lower effective T so it can increase entropy without using up all the energy being made extractable from the gas).

To put it another way, can Maxwell's demon extract more usable energy than a Carnot engine?

A Carnot engine can extract all the energy from a gas, it's all a question of the effective temperature of the brain making the decisions (or in the standard setup, the temperature of the exhaust reservoir). Like all thermodynamic engines, Carnot cycles participate with our intentions only so long as the total entropy increases in the cycle (or stays nearly the same). Frankly, I don't see what Bennett and Landauer etc. are on about, the situation all seems pretty clear. Unless their intention is to delve into the constraints on modeling how the Demon thinks, i.e., on just how it manages to increase the entropy of its environment, that's the only issue that seems subtle. I would say it is perfectly obvious that the Demon obeys the second law, and the only reason we might imagine giving it a "pass" on that law is that we are treating it as magic-- rather than as an actual physical system.

 

[Rap]

The energy is already usable, because you have a container filled with hot gas (the temperature is associated with the average kinetic energy of the cannonballs.

A Carnot engine can extract all the energy from a gas, it's all a question of the effective temperature of the brain making the decisions (or in the standard setup, the temperature of the exhaust reservoir).

I think what he means is that you cannot run an engine between two reservoirs at the same temperature. If the demon causes the two temperatures to differ, then and engine can extract "usable" energy. One of the gas systems is serving as the exhaust reservoir.

Frankly, I don't see what Bennett and Landauer etc. are on about, the situation all seems pretty clear. Unless their intention is to delve into the constraints on modeling how the Demon thinks, i.e., on just how it manages to increase the entropy of its environment, that's the only issue that seems subtle.

Yes, I think that's just what they are doing. I think its very interesting to try to understand how the demon increases its "entropy". I put that in quotes, because unless you can define a microstate and a macrostate, you cannot define entropy, and I am not always sure what constitutes a macrostate of the demon. If you demand that the demon return to its original state, then the macrostate is just a matter of temperatures, that's why they demand it, it simplifies things. But on the other extreme, if you say the demon never erases, just keeps using new memory, then what? Temperature does not change? In order not to violate the second law, the entropy has to be k times the information entropy of the demon's memory? That means the macrostate of the demon is one in which we cannot read its internal logical state and its entropy is the number of ways its memory could be configured given what we know about the two gases at that point. Does the second law therefore imply that there is no way we can communicate with the demon and read its logical state? Forgetful or not, the demon is not thermally connected to the system, yet it is acquiring or creating entropy to match the entropy it is removing, not by thermodynamic means, but by "logical" means.

 

[Ken G]

I think what he means is that you cannot run an engine between two reservoirs at the same temperature. If the demon causes the two temperatures to differ, then and engine can extract "usable" energy. One of the gas systems is serving as the exhaust reservoir.

What I'm saying is that you can run a heat engine between two temperatures, and you can run a Demon to create that temperature difference, and both stages increase entropy (the Carnot cycle can increase it very little). This isn't a problem with free energy, free energy depends on the context. A hot gas inside more of the same hot gas has no free energy, but a hot gas inside a cold or low pressure environment, or connected to a Demon that effectively has access to a cold environment, has lots of free energy. Free energy just means that there is some other class of states that are more numerous (possible just barely more numerous) in which that energy is somewhere else. That is just as true when the Demon is creating a T difference as when we are running an engine off that T difference. They both happen because entropy increases.

Does the second law therefore imply that there is no way we can communicate with the demon and read its logical state? Forgetful or not, the demon is not thermally connected to the system, yet it is acquiring or creating entropy to match the entropy it is removing, not by thermodynamic means, but by "logical" means.

The second law is way easier than that, I really don't know why they want to make it so complicated. It doesn't have anything to do with memory, unless memory is the way the entropy is increasing. If the memory isn't there, then it would just be some other way. The Demon's brain simply doesn't function unless it increases entropy, that's how it works. The entropy increase can be used to make decisions that drop the entropy somewhere else, but it can't operate to drop total entropy, or the behavior simply doesn't happen. More likely configurations do not evolve into less likely ones, that's all the second law says.

 

[lugita15]

The second law is way easier than that, I really don't know why they want to make it so complicated. It doesn't have anything to do with memory, unless memory is the way the entropy is increasing. If the memory isn't there, then it would just be some other way. The Demon's brain simply doesn't function unless it increases entropy, that's how it works. The entropy increase can be used to make decisions that drop the entropy somewhere else, but it can't operate to drop total entropy, or the behavior simply doesn't happen. More likely configurations do not evolve into less likely ones, that's all the second law says.

Ken G, I'm a bit uneasy about the philosophical attitude you seem to have concerning the 2nd law. It is not some iron-clad cosmic law of the universe, like the first law. It is, as you said, a statement that things tend to go from less likely configuration to more likely configurations. You make it sound as if machines cannot even in principle function if they decrease the total entropy of the system, as if the second law is what is driving them. But the second law is an effect, not a cause, of physical behavior, and it is a statistical rule, not a deterministic law; see the fluctuation theorem in statistical mechanics, which gives the probability that the entropy of a system increases or decreases.

For this reason, I think that instead of analyzing Maxwell's demon based on what the second law "permits" it to do, it's more useful to analyze it without assuming the second law, and thus finding out what makes the second law work in this case.

 

[Ken G]

You make it sound as if machines cannot even in principle function if they decrease the total entropy of the system, as if the second law is what is driving them.

Yes, that statement is correct-- no machine can function reliably unless its net effect is to increase entropy. It makes no difference if we imagine this constraint is "driving" them-- it's just a true constraint.

But the second law is an effect, not a cause, of physical behavior, and it is a statistical rule, not a deterministic law; see the fluctuation theorem in statistical mechanics, which gives the probability that the entropy of a system increases or decreases.

I did not say it is a "deterministic" law-- it is a constraint. You can't use it to determine what will happen because you can't always know what is most likely to happen without significant knowledge of the details. However, you can know what will not happen without any knowledge of the details-- the Demon will not work unless he increases entropy. Not having to know the details is the raison d'etre of thermodynamics.

For this reason, I think that instead of analyzing Maxwell's demon based on what the second law "permits" it to do, it's more useful to analyze it without assuming the second law, and thus finding out what makes the second law work in this case.

The reason that is not terribly useful is because you have a different analysis for every different Demon you can construct. The purpose of thermodynamics is to make general statements, not statements tied directly to some particular mechanism (which must be analyzed using mechanics, not thermodynamics). If you want to use mechanics to test thermodynamics, then you need a model of the Demon, but you cannot get "paradoxes" by imagining magical properties of the Demon-- you need an actual mechanism, and when you get that, you will see that the Demon either increases entropy, or doesn't work.

 

[lugita15]

The reason that is not terribly useful is because you have a different analysis for every different Demon you can construct. The purpose of thermodynamics is to make general statements, not statements tied directly to some particular mechanism (which must be analyzed using mechanics, not thermodynamics). If you want to use mechanics to test thermodynamics, then you need a model of the Demon, but you cannot get "paradoxes" by imagining magical properties of the Demon-- you need an actual mechanism, and when you get that, you will see that the Demon either increases entropy, or doesn't work.

But the second law of thermodynamics is a mere statistical consequence of the laws of mechanics. Thus, if the second law tells you that Maxwell's demon cannot reliably reduce total entropy, without actually knowing the inner workings of the demon, it should also be possible to draw the same statistical inference directly from the laws of mechanics. In other words, you are in essence redoing the proof of the second law from Newton's laws in the particular case where a device has the properties Maxwell's demon claims to. It is this kind of analysis that has allowed people to figure out some of the underlying reasons why the second law is an emergent property of mechanical systems, for instance the connection between information and entropy.

 

[Ken G]

But the second law of thermodynamics is a mere statistical consequence of the laws of mechanics.

I would say it must be consistent with any laws of mechanics, but it is independent of those laws, and it doesn't even really require any laws of mechanics be in place. Mechanics are the details, thermodynamics is what you can do without even knowing the details.

Thus, if the second law tells you that Maxwell's demon cannot reliably reduce total entropy, without actually knowing the inner workings of the demon, it should also be possible to draw the same statistical inference directly from the laws of mechanics.

Yes, if you have a model for the Demon. Do we have mechanical models of brains?

In other words, you are in essence redoing the proof of the second law from Newton's laws in the particular case where a device has the properties Maxwell's demon claims to. It is this kind of analysis that has allowed people to figure out some of the underlying reasons why the second law is an emergent property of mechanical systems, for instance the connection between information and entropy.

The "why" of the second law is independent of mechanics, it is thermodynamics. It all boils down to, entropy is our way of counting which collections of configurations contain more equally likely states, and hence are what will happen.

 

[lugita15]

I would say it must be consistent with any laws of mechanics, but it is independent of those laws, and it doesn't even really require any laws of mechanics be in place. Mechanics are the details, thermodynamics is what you can do without even knowing the details..

We can easily have a universe in which the laws of mechanics do not lead to the second law of thermodynamics. For instance, the laws could dictate that systems try to attain a specific ordered state.

Yes, if you have a model for the Demon. Do we have mechanical models of brains?

My point was that if we can use the 2nd law of thermodynamics to conclude that Maxwell's demon cannot reliably decrease the total entropy, and we can conclude this without examining in detail how it works, then we should be able to reach a similar conclusion using Newton's laws, again without looking at a specific model for the demon. If nothing else, we can adapt the proof of the second law of thermodyamics given in e.g. the later chapters of Feynman Lectures volume 1, for the case of a unspecified device which is assumed to have the properties we ascribe to Maxwell's demon. If we do something like this, we can find out some of what make the second law "tick", such as the role of information.

The "why" of the second law is independent of mechanics, it is thermodynamics. It all boils down to, entropy is our way of counting which collections of configurations contain more equally likely states, and hence are what will happen

I agree that the definition of entropy is independent of the laws of physics. But I disagree with your assertion that thermodynamics is independent of mechanics. The fact that entropy tends to increase with time more often than it decreases seems like a contingent fact of the universe. And in fact, even in our universe with our laws of physics, the fluctuation theorem suggests that if entropy gets really high, it is possible in principle for the second law of thermodynamics to go in reverse.

 

[Ken G]

We can easily have a universe in which the laws of mechanics do not lead to the second law of thermodynamics. For instance, the laws could dictate that systems try to attain a specific ordered state.

"Disorder" simply means "more ways of being", which means "more likely", and that's the second law in a nutshell. The sole assumption is that you can just count the ways of being (the number of configurations)-- this is the crux of statistical mechanics, that every individual state is equally likely. That's the only assumption behind the second law, and if it weren't true, it would only mean that we would need to have a more sophisticated concept of what entropy is, beyond just ln(N), if some states were "preferred" by the mechanics. But any mechanics without that property yields the second law, quite generally.

My point was that if we can use the 2nd law of thermodynamics to conclude that Maxwell's demon cannot reliably decrease the total entropy, and we can conclude this without examining in detail how it works, then we should be able to reach a similar conclusion using Newton's laws, again without looking at a specific model for the demon.

And that is what is not true. Newton's laws are about the details, thermodynamics is what you can do without anything like Newton's laws. That's why the main principles of thermodynamics were discovered independently of Newton's laws (like the work of Carnot and Clausius), and sometimes even prior to them (like Boyle's law).

If nothing else, we can adapt the proof of the second law of thermodyamics given in e.g. the later chapters of Feynman Lectures volume 1, for the case of a unspecified device which is assumed to have the properties we ascribe to Maxwell's demon. If we do something like this, we can find out some of what make the second law "tick", such as the role of information.

Right, with no reference to any mechanism or mechanics of the Demon. This is crucial-- the mechanics only serve as informative examples of the second law, they are not part of the derivation of it. The derivation proceeds along the lines I gave above, and with no mention of any laws of mechanics, other than that they do not pick out preferred states. One might thus say that the second law arises from the universe being "non-teleological", which still remains the hardest thing for many to accept about it. It might not even be true-- it's only a law after all! But we won't know until we can really model thought, to see if it brings in some kind of teleology that could motivate essentially "magical" treatments of the Demon.

 

[lugita15]

No, that is exactly what cannot be true, in any theory of mechanics exhibited by large systems. That's pretty much the whole point of thermodynamics! Again, "disorder" simply means "more ways of being", which means "more likely", and that's the second law in a nutshell. The sole assumption is that you can just count the ways of being (the number of configurations)-- this is the crux of statistical mechanics, that every individual state is equally likely. That's the only assumption behind the second law, and if it weren't true, it would only mean that we would need to have a more sophisticated concept of what entropy is, beyond just ln(N).

Are you saying that it is literally impossible to have laws of physics in which all the particles work together to produce a particular ordered state?

And that is what is not true. Newton's laws are about the details, thermodynamics is what you can do without anything like Newton's laws. That's why the main principles of thermodynamics were discovered independently of Newton's laws (like the work of Carnot and Clausius), and sometimes even prior to them (like Boyle's law).

Sure, just like Kepler's laws were discovered before Newton's law of gravitation and the Balmer series was discovered before the Schrodinger's equation. Phenomena of nature can be discovered independently even if they derive theoretically from a common source.

Right, with no reference to any mechanism or mechanics of the Demon. This is crucial-- the mechanics only serve as informative examples of the second law, they are not part of the derivation of it. The derivation proceeds along the lines I gave above, and with no mention of any laws of mechanics.

I was envisioning a different sort of procedure. I'm suggesting doing the statistical mechanics derivation of the second law of thermodynamics from Newton's laws of motion, as outlined in the Feynman lectures and fleshed out by Boltzmann, but restricting the proof to the case where you have a Maxwell's demon with unspecified mechanism. So the rest of the scenario will be analyzed according to mechanics, it is only the demon that is a black box.

If you set F=mv instead of ma, as the ancients imagined, you still get the second law of thermodynamics, without any difference. Indeed, this is the second law in highly dissipative situations, and it's still just thermodynamics.

I don't think this is too surprising; (this part of) Aristotelian physics is just Newtonian physics in the limit of strongly dissipative forces.

 

[Ken G]

I confess that I did some editing of my last post, just after I posted it, so some of the points were clarified and you might want to look at the improved version. I'm not saying it's impossible to have laws that create ordered states, I'm saying that only a very general assumption about the laws is required to rule that out (the assumption needed is that all possible states are equally likely, so none are picked out by the laws as special in some way). I would call that "non-teleological" laws, similar to what we get in relativity where there are no preferred reference frames. The key point about the Demon is that it is not given a pass to violate this rule-- it cannot target specific states, it must "throw darts" like everything else, and what it hits, ultimately, is simply the largest target, i.e., the highest entropy. Anything else is "magic". Now, of course we must include the entire "target", not just the gas and its entropy, so that's why violations of the second law invariably result from not recognizing the full space of possible outcomes that are being affected by the Demon. That's how the Demon works-- by gaining access to some other set of possible outcomes that can mitigate the "smaller target" of the reduced entropy in the gas. So my point is, this is not some technicality about how the Demon functions, it is how the Demon functions, thermodynamically speaking. No magic, no teleology, and you have the second law, regardless of the mechanics.

Now of course, this is the thermodynamics view-- I don't say reality actually contains no magic, and no teleology. It is physics that doesn't have those things, and does well without them, and that seems to be the reason that thermodynamics works so well. We don't have any reason to think a mind, demonic or human, can violate the second law, but we can notice that the second law stems from how our mind analyzes nature, so the law is as much a product of our minds as it is something that is a rule of nature itself. Hence we don't have to say that our minds result from the action of the second law, we can always assert the converse if we prefer. But either way, the two come together, and I would say the responsibility is on any who would claim a Demon can do something that the patent office rejects as plausible.

 

[Rap]

Its the temperature and entropy thing again - When the second law talks about entropy, it is always in the context of temperature. Thermodynamic entropy is information entropy, but not all information entropy is thermodynamic entropy. Thermodynamic entropy always involves temperature.

Lets assume we can ignore the measurement process as far as its contribution to the second law balance sheet is concerned. Measuring the cannonballs in the cannonball gas suggests to me this is true. Let's ignore the trapdoor process as well. Then we have two containers of gas and a demon. In the beginning, the two gas systems are at the same temperature. Some time later, they are not. There has been a thermodynamic entropy reduction, if we ignore the demon. Let's say the second law holds, and the demon is some kind of computer, a Turing machine for simplicity. This means that the demon's thermodynamic (!) entropy must be increased, or, if it must remain at some low temperature, that its increasing entropy is continually dumped outside. The information entropy of the demon's logical state is not the subject of the second law, unless you can define some "logic temperature" and multiply it by k times the info entropy to get an energy. Let's say you cannot. Then it follows that there can exist no Turing machine that accomplishes the demon's purpose without the dissipation of the amount of lost gas entropy as HEAT. If you want to throw a monkey wrench into the workings of the second law, then you have to design a Turing machine that accomplishes the demon's purpose while dissipating less than that lost entropy. If you can design a Turing machine that is as general as possible, which dissipates no heat, you will have surely thrown a monkey wrench into the second law (given the arguable assumptions already made).

This is partly what Bennett and E&N are discussing, trying to figure out the thermodynamics of a Turing machine, or some other universal computer. If you break the computer down into individual logical operations, then one or more of those operations must dissipate heat. The heat dissipation, or absence of it, for various logical operations, is what they are discussing, and I see it as an interesting discussion, given the implications.

They also discuss Szilard's principle which says entropy is lost in the measurement process. As far as I can see, Bennett disagrees and E&N, I don't know.

 

[Ken G]

Its the temperature and entropy thing again - When the second law talks about entropy, it is always in the context of temperature. Thermodynamic entropy is information entropy, but not all information entropy is thermodynamic entropy. Thermodynamic entropy always involves temperature.

Actually I think it's the other way around-- the meaning of temperature stems from thermodynamical entropy. The best way to think about what temperature is is to multiply it by k and say that kT is the energy a system must take in in order to increase its access to undifferentiated states by a factor of e. But one can talk about the latter without ever referring to the former, if you simply don't care about energy (given that it's conserved, we usually do choose to care about it, but we don't have to and we still have a second law). So T comes in not so much with entropy, but when you want to connect entropy to energy. That means it relates to the importance of entropy, because energy measures the consequences of what the entropy is doing. But we still have a second law even if we have never heard of either energy or temperature.

Lets say the second law holds, and the demon is some kind of computer, a Turing machine for simplicity. This means that the demon's thermodynamic (!) entropy must be increased, or, if it must remain at some low temperature, that its increasing entropy is continually dumped outside.

Be careful not to confuse entropy with energy. If the Demon is at a very low T, then a very small energy change can correspond to a huge entropy change, and the Demon can remain at low T yet still increase its entropy without dumping anything to the outside. Indeed, if the Demon is nearly at absolute zero T, then it can increase its own entropy arbitrarily, and still be at nearly absolute zero T because it is taking on very little energy. But I agree that in practice, most Demons are not going to have an effective T that is incredibly small, so they are going to need an environment they can dump non-negligible heat into and increase the entropy of.

If you want to throw a monkey wrench into the workings of the second law, then you have to design a Turing machine that accomplishes the demon's purpose while dissipating less than that lost entropy. If you can design a Turing machine that is as general as possible, which dissipates no heat, you will have surely thrown a monkey wrench into the second law (given the arguable assumptions already made).

But you will also run afoul of the basic axioms of thermodynamics, which will not be possible to avoid. Let's say you do make such a Turing machine, whose functioning is to reduce the entropy in the gas by more than it increases the entropy in the environment in which it is functioning. Then the net result of the action of your machine is to go from more likely configurations to less likely ones. What keeps the machine from running backward? Remember, all the laws of physics involved are time reversible, so if I send t to -t, I get a version of your Turing machine that makes the opposite decisions and causes the gas to come to the same T, and does so in a way that increases entropy. So what is going to make your Turing machine not turn into mine? I claim that is just exactly what it will do. Remember, the Turing machine is not a magical genie, even if you give it a program it can follow that program in either order of time, and it will follow it in whatever order increases entropy, because the arrow of time comes from less likely configurations being replaced by more likely ones. The arrow of time isn't magic either.

 

[lugita15]

Ken G, if you believe that the 2nd law of thermodynamics is so natural, and that it is an obvious consequence of not only Newton's laws but any laws remotely like them, then why is Boltzmann's H-theorem so hard to prove, involving advanced mathematics and clever pieces of reasoning? Your view of the 2nd law as a near-tautology "systems are more likely to evolve into more likely states" should allow for a much easier proof of the theorem.

 

[Rap]

Actually I think it's the other way around-- the meaning of temperature stems from thermodynamical entropy. The best way to think about what temperature is is to multiply it by k and say that kT is the energy a system must take in in order to increase its access to undifferentiated states by a factor of e. But one can talk about the latter without ever referring to the former, if you simply don't care about energy (given that it's conserved, we usually do choose to care about it, but we don't have to and we still have a second law). So T comes in not so much with entropy, but when you want to connect entropy to energy. That means it relates to the importance of entropy, because energy measures the consequences of what the entropy is doing. But we still have a second law even if we have never heard of either energy or temperature.

Hmmm - ok, right.

Be careful not to confuse entropy with energy. If the Demon is at a very low T, then a very small energy change can correspond to a huge entropy change, and the Demon can remain at low T yet still increase its entropy without dumping anything to the outside. Indeed, if the Demon is nearly at absolute zero T, then it can increase its own entropy arbitrarily, and still be at nearly absolute zero T because it is taking on very little energy. But I agree that in practice, most Demons are not going to have an effective T that is incredibly small, so they are going to need an environment they can dump non-negligible heat into and increase the entropy of.

Yes. I am keeping close track of the difference between entropy and energy, and I agree with the above.

But you will also run afoul of the basic axioms of thermodynamics, which will not be possible to avoid. Let's say you do make such a Turing machine, whose functioning is to reduce the entropy in the gas by more than it increases the entropy in the environment in which it is functioning. Then the net result of the action of your machine is to go from more likely configurations to less likely ones. What keeps the machine from running backward? Remember, all the laws of physics involved are time reversible, so if I send t to -t, I get a version of your Turing machine that makes the opposite decisions and causes the gas to come to the same T, and does so in a way that increases entropy. So what is going to make your Turing machine not turn into mine? I claim that is just exactly what it will do. Remember, the Turing machine is not a magical genie, even if you give it a program it can follow that program in either order of time, and it will follow it in whatever order increases entropy, because the arrow of time comes from less likely configurations being replaced by more likely ones. The arrow of time isn't magic either.

Good point. It shows that in order for a Turing machine to have a forward direction, it must generate entropy. Let's assume the Turing machine is made from macroscopic mechanical parts. The arrangement of its mechanical parts is its "logical state". This logical state has nothing to do with thermodynamics, it constitutes a macrostate, it does not factor into the many microstates which give rise to this configuration. The entropy the demon generates is thermodynamic entropy, which is heat (assuming its temperature is not absolute zero). Now, this entropy, by the second law, must be greater than the entropy it removed from the two gases. The problem now is to prove this, hopefully by conceptually breaking down the Turing machine into separate logical operations, and showing which logical operations are generating the entropy, and what is the least amount that they generate, in principle. Then show that when you add them all up for a program that contains the minimum number of entropy-generating logical steps, but which still implements the demon's purpose, that value of entropy is greater than the entropy removed from the two gases.

 

[Ken G]

Ken G, if you believe that the 2nd law of thermodynamics is so natural, and that it is an obvious consequence of not only Newton's laws but any laws remotely like them, then why is Boltzmann's H-theorem so hard to prove, involving advanced mathematics and clever pieces of reasoning? Your view of the 2nd law as a near-tautology "systems are more likely to evolve into more likely states" should allow for a much easier proof of the theorem.

This is certainly a valid challenge, and I'll give it some thought, but on the surface I would say the difference is in understanding what the second law is basically saying, in contrast with how to place that fairly intuitive statement into a more rigorous mathematical structure. In the mean time consider these words by de Broglie:
"When Boltzmann and his continuators developed their statistical interpretation of Thermodynamics, one could have considered Thermodynamics to be a complicated branch of Dynamics. But, with my actual ideas, it's Dynamics that appear to be a simplified branch of Thermodynamics. I think that, of all the ideas that I've introduced in quantum theory in these past years, it's that idea that is, by far, the most important and the most profound."

 

[Ken G]

The problem now is to prove this, hopefully by conceptually breaking down the Turing machine into separate logical operations, and showing which logical operations are generating the entropy, and what is the least amount that they generate, in principle. Then show that when you add them all up for a program that contains the minimum number of entropy-generating logical steps, but which still implements the demon's purpose, that value of entropy is greater than the entropy removed from the two gases.

But what I would say is, once you have accomplished that breakdown, you still don't know which way your machine will function until you either invoke the second law, or claim to have experience in similar machines. Both are determined by the answer to the entropy issue, not the other way around. So our job is not to prove that the machine increases entropy, it is to figure out what the machine will do, given that it increases entropy. The problem with the "Demon" is that we borrow from the function of our brains to give the Demon magical properties, but we only do that because we think we are familiar with thought and decisions-- as soon as you have to provide a machine instead, broken down as you say, then you immediately do not know what that machine will do (in particular, what sense will time have for your machine). That comes from entropy, there is no other way to calculate what the machine will actually do once you build it, unless you invoke experience with similar machines (but then it's not a theoretical analysis any more).

If you do go that latter way, and rely on your experience with machines (say computers) instead of with brains, then the way the second law works in our experience is already built into the analysis. So the choice is, invoke experience and use it to show that the second law is behind that experience (which is what you want to do), or invoke the second law and figure out what will happen without the benefit of experience (which is what the Patent Office does when it refuses to consider perpetual motion machines). But there's no reason to think that using a machine to separate gas into different Ts is going to reduce the entropy, I think it's pretty clear that's a dead duck and the details of the machine (like if it has memory) are not terribly important or even advisable to analyze (since you cannot individually analyze every possible machine, just like the Patent Office cannot). But granted, you want to be convinced it's a dead duck, so for that you will need to invoke a lot of experience in how machines work, and if you see it for a few examples, you can develop the faith you seek in the second law. You can't add to my faith by analyzing a few more examples, you could only have an effect by finding a counterexample (certainly a noble effort, most likely doomed to fail but instructive in how it fails each time).