What is the connection between Maxwell's Demon and Liouville's Theorem?

[AdrianMay]

Hi Folks,

(Skip to next paragraph if you already know what M's D is.) Maxwell's demon was a counterargument to the second law of thermodynamics (and hence the first) involving two chambers of air connected by a trap door which this demon would open and close to let fast molecules go one way and slow ones the other. One chamber gets hotter and can perpetually drive a heat engine.

The catch apparently lies in the assumption that the demon himself can be neglected but in reality one has to consider the entropy associated with his decision. But this argument makes no reference to the size of the air molecules.

What if they were whopping great cannonballs? Can one still claim that the energetic equivalent of whatever bits he has to erase in his head is comparable with the power station you could drive that way? How do we even make the comparison between energy and entropy here?

Now that I think about it, I don't really understand why it would have been a problem in the first place. If I milk kinetic energy out of either chamber, all the cannonballs are going to slow down eventually. But they did teach me in school that breaking the second law was tantamount to breaking the first.

Confused,
Adrian.

 

[AdrianMay]

Another problem with the "erasing bits in his head" argument is that it doesn't mention the temperatures involved, or how fussy he is. The number of bits he has to erase is independent of whether he sets his threshold at 100^C or 1000^C. But surely the entropy reduction is different in the two cases.

 

[Andy Resnick]

You have to erase the bits to ensure the process is cyclic- your end state is the same as the initial state.

 

[AdrianMay]

I know I have to erase the bits, but I don't see how that problem can compare with an entropy reduction that seems to be in totally different units.

 

[Rap]

Hi Folks,

(Skip to next paragraph if you already know what M's D is.) Maxwell's demon was a counterargument to the second law of thermodynamics (and hence the first) involving two chambers of air connected by a trap door which this demon would open and close to let fast molecules go one way and slow ones the other. One chamber gets hotter and can perpetually drive a heat engine.

The catch apparently lies in the assumption that the demon himself can be neglected but in reality one has to consider the entropy associated with his decision. But this argument makes no reference to the size of the air molecules.

What if they were whopping great cannonballs? Can one still claim that the energetic equivalent of whatever bits he has to erase in his head is comparable with the power station you could drive that way? How do we even make the comparison between energy and entropy here?

Now that I think about it, I don't really understand why it would have been a problem in the first place. If I milk kinetic energy out of either chamber, all the cannonballs are going to slow down eventually. But they did teach me in school that breaking the second law was tantamount to breaking the first.

Confused,
Adrian.

I think this is a good question, and I don't know the answer (yet). A few points though - if you have a gas of cannonballs, say they have a mass of 1 kilogram each, and let's assume they collide elastically with each other, so that their internal thermodynamics doesn't matter, then their kinetic energy will be 3kT/2=mv^2/2 where v is their velocity, m is their mass, T is their temperature. If the cannonballs are moving at an average velocity of 1 meter per second, then their temperature will be T=mv^2/3k = 2.4 x 10^22 Kelvin.

Also, entropy is "missing information" and may be different for different people. If person A knows only the temperature, pressure, volume, etc of a gas, then A will calculate some entropy for that gas, but if person B knows the position and velocity (speaking classically, now) of every particle in that gas, then they will say there is no entropy. Person A will write the second law of thermodynamics as dU=TdS-PdV+µdN (µ is chemical potential, you have to include that when possibly changing the number of particles). Person B will write the second law as dU=\sum m_iv_i^2/2 where the index i runs over all particles. It won't be the second law, really, it will be a mechanical equation.

I guess I don't understand how the energy expended by the demon can compare to the energy of one "hot" cannonball being allowed to pass through the door. The demon can use photons to determine the position and energy of a cannonball, but the energy used is negligible compared to the cannonball. Same goes for the energy needed for the demon to record the information. Can we say the energy required to open and shut the door is negligible?

So this means that energy is flowing from one side to the other. The entropy of the system may be increasing, but the temperature of demon's process is way lower than the temperature of the cannonballs.

 

[Andy Resnick]

What if they were whopping great cannonballs? Can one still claim that the energetic equivalent of whatever bits he has to erase in his head is comparable with the power station you could drive that way? How do we even make the comparison between energy and entropy here?

There are two ways to make the comparison. The first is to associate entropy with the transfer of information: in this case, the information associated with measuring the velocity of a cannonball and deciding to open a door (or not). Shannon was the first (AFAIK) to work out that the transfer of 1 bit of information is equivalent to kT ln(2) units of energy, or a change in entropy of k ln(2).

Alternatively, there is the information required to specify the microstate itself- this has been worked out in terms of algorithms, and the most straightforward definition is the Kolmogorov complexity. The two measures are somewhat related (for example, the amount of information required to transmit instructions for constructing a power plant is related to the information required to actually construct the power plant), but Shannon's result is better understood and is the basis for several data compression algorithms.

Your example of a gas of cannonballs a very interesting one (especially if gravity is not important)- defining the temperature of such a gas is not so obvious: see, for example, the hard-sphere phase transition. The transition between fluid and crystal occurs at a particular volume fraction- not the conventional sense of temperature. In terms of thermodynamics, the entropy of a gas of cannonballs will also depend on the volume fraction.

 

[256bits]

If the cannonballs are moving at an average velocity of 1 meter per second, then their temperature will be T=mv^2/3k = 2.4 x 10^22 Kelvin.
...

I guess I don't understand how the energy expended by the demon can compare to the energy of one "hot" cannonball being allowed to pass through the door. The demon can use photons to determine the position and energy of a cannonball, but the energy used is negligible compared to the cannonball. Same goes for the energy needed for the demon to record the information. Can we say the energy required to open and shut the door is negligible?

Using your result for temperature.
The demon has to measure the speed of the cannonball by absorption of photons as you suggest, these are now high energy photons coming from a source of 2.4 x 10^22 Kelvin, which surely will increase the demon's temperature greatly is he is tiny enough to not have an energy expenditure of appreciable value when opening or closing the door. So the question is then how many cannonballs can the demon measure and let through the door before his own temperature rises to that of the cannonballs.
I don't think size of "molecules" matters for Maxwell's demon.

 

[Rap]

Using your result for temperature.
The demon has to measure the speed of the cannonball by absorption of photons as you suggest, these are now high energy photons coming from a source of 2.4 x 10^22 Kelvin, which surely will increase the demon's temperature greatly is he is tiny enough to not have an energy expenditure of appreciable value when opening or closing the door. So the question is then how many cannonballs can the demon measure and let through the door before his own temperature rises to that of the cannonballs.
I don't think size of "molecules" matters for Maxwell's demon.

The demon does not need high energy photons. If the "molecules" are cannonballs, a flashlight will be overkill, and still will not appreciably affect the energy or momentum of the cannonballs. The demon is not tiny now, a human size demon will be fine. Note that the cannonballs are not hot to the touch of the human size demon. A basic assumption is that the cannonballs collide perfectly elastically which means it doesn't matter what their conventional temperature is. The thermodynamic temperature of the "cannonball gas" is such that each degree of freedom holds energy kT/2 and if they have mass 1 kg and average speed 1m/sec, that gives 2x10^22 Kelvin. Let's assume they are so far apart that the total volume of the cannonballs is much smaller than the volume that contains them. A volume of 10^33 cubic meters will give about a mole of cannonballs that will be about a kilometer apart, on average. That volume amounts to a cube a million kilometers on a side, about the distance from the Earth to the sun. This cannonball gas has enough cannonballs to be treated as a thermodynamic system. Now we have essentially an ideal gas of cannonballs. The density is about 2.4 x 10^-10 cannonballs per cubic meter. The pressure is about 8x10^-11 pascal. A person who knows only the pressure, temperature, and volume, for example, will assign an entropy to this system, the ideal gas entropy (Sackur-Tetrode, to within a constant, I guess).

Notice that the demon knows more than the temperature, pressure, and volume. The demon knows the position and momentum of the cannonballs in his vicinity. (They are big enough to be treated classically).

I don't think the size of the molecules matters either - which is why we should be able to prove that the second law is not violated for a human demon and a gas of cannonballs undergoing perfectly elastic collisions.

 

[AdrianMay]

Oh dear. I seem to have put the cat among the pigeons. Can we escalate this one? E.g. to Stephen Hawking or somebody like that ;-)

I think 256bits must be on to something with the temperature of the cannonballs thing. Real cannonballs would probably heat up as they bounced around, taking energy out of the translational modes. We'd better include rotation as well. If we said there were no other modes but translation, then I think we'd be talking about a quantum system which might be a totally different ball game (scuse the pun) and if we approximated the collisions to perfectly elastic I think we'd just be agreeing to wait a very long time but not talking about perpetual motion or equilibrium. So let's take the case where the energy spreads out over all modes that a real cannonball has and see if we still have a problem.

I think we do though because I can reduce the rate at which the demon gets sunbathed just by shrinking him and then I only need a very small fridge to keep him cool. I'm expecting to milk significant energy out of this contraption and I reckon I can spare that much for the fridge.

Using Rap's numbers as an example, can anybody figure out how much energy we can harvest if we naively ignore the bits in the demon's head? I'm so clueless about thermodynamics that it still looks like zero to me, which means I don't understand why breaking the 2nd law means breaking the 1st or what was so demonic about this trapdoor in the first place.

 

[Rap]

We should deal with the simplest situation possible that keeps the problem alive. Cannonballs colliding inelastically is a needless complication, it means the system is out of equilibrium - the cannonballs have an internal temperature and a "cannonball gas temperature" and they are vastly different. Equilibrium occurs when the two temperatures are the same - i.e. the cannonballs are practically motionless and have absorbed all the kinetic energy they used to have and that energy is now heat energy. When talking about entropy, you have to include the "internal entropies" of the cannonballs and the "cannonball gas entropy", etc. etc. Huge complication for the demon to keep track of. Do we really want to worry about Maxwell's demon in a non-equilibrium situation when the problem does not demand non-equilibrium? No.

We can forget about angular momentum too. Elastic collisions will impart no rotational energy to any cannonball during a collision, either from an off-center hit or from a rotating cannonball transferring rotational energy to another cannonball. Just as the internal temperature of a cannonball is irrelevant, so is its rotational energy.

Elastically colliding cannonballs are like a simple monatomic ideal gas, and Maxwells demon should give the same results in both cases. The problem is kept alive - what happens if you go to a "cannonball gas"? The bits that the demon will have to record will simply be the positions and momenta of the cannonballs, he won't have to worry about the internal entropy bits of the cannonballs, the rotation of the cannonballs, etc. Much simpler, and I don't think we will require quantum mechanics to resolve the issue.

I'm not very good at Maxwell's demon, there must be something missing. Is it that the demon is required to be at the same temperature as the gas he is manipulating? If that is the case, then if you multiply the information entropy of those recorded bits times kT where T=10^22 or whatever, you get a very big energy, on the order of a cannonball's kinetic energy, and that plus the photon energy he used to measure the cannonballs may be more than the energy "generated" by the demon. I don't know.

Also, violating the second law does not mean you violate the first. The second law is being violated all the time by small fluctuations, the first law, classically never. Its only in the "thermodynamic limit" of an infinite number of particles that the second law holds exactly.

 

[AdrianMay]

The elastic case is clearly simpler, but I'm worried that it might be too soft on me, i.e., make it too easy for me to make this case that the "bits-in-the-head" argument is dodgy. Perhaps if we get a full blown paradox in the elastic case, the solution is that perfect elasticity isn't achievable in practice.

Not in theory either if the balls have non-zero diameter. That follows from relativity - there has to be a wave in the ball to tell the back face to change direction, and that wave can't go at infinite speed, so it's going to bounce around and dissipate as heat.

Put it this way: it would be more satisfying to me to make my case fly in the inelastic case than just to get away with it in the idealised case.

As for rotation, you might be right that rotating modes won't get stimulated if the balls are *oily*, but if you're prepared to overlook the relativity argument, then we could also speak of perfect elasticity in the case that their surfaces have an enormous coefficient of static friction and therefore won't grind against each other. In that case, I think balls glancing off each other would end up rotating. I don't mind having oily balls though.

I'm not entirely convinced that the temperature of the demon is relevant. I reckon 1 bit of information is just that. Is it really applicable to take equations like dU = TdS ... which originally came from huge statistical systems like gases and apply them to single bits. I'd like to hear a more mechanical argument that a bit really really has to use TdS of energy when you erase it.

I also have my doubts about this 10^22 Kelvin. That would seem to imply that if I use 1kg balls of something inelastic like plasticine starting at absolute zero, then let all the KE turn to heat just by watching them all go splat, then they should reach this wild temperature. But in reality they only seem to heat up a few tenths of a degree or so. Perhaps the bug in the calculation is that pdV wasn't mentioned.

 

[Ken G]

Listen to Rap, he's making good sense. The first case to understand is the purely elastic case, because we should be able to account for that situation. Also, we do not want to make the Demon be the same T as the cannonballs, indeed it will be crucial that the Demon either not have a T associated with him at all, or else have a much lower T associated with him than the cannonballs have. Finally, the energy per "cannonball" is indeed very much like their temperature here, so if we have a counterintuitive case to understand, it is the case where the "Demon" is generating a huge T difference across the "door" by using what would seem to be a relatively small number of decisions.

First let me say there must not be anything "dodgy" about the "erasing bits" argument, even in the case of cannonballs and high T-- that argument must hit the nail on the head even in that situation, so this is what we need to understand. I would say the first thing we should do is distinguish the first and second laws-- the first merely being conservation of energy, the second counting entropy. What seems "paradoxical" here is that if the Demon can take a gas that is everywhere the same (very large) T, and make some decisions about a door, he can generate a huge T difference, which could then run a heat engine and do work, in a way that looks completely cyclic so sounds like it could be repeated over and over. This does not yield any problems with the first law, because energy is still being conserved-- if the heat engine does work, the cannonballs lose energy, and since nothing replaces that energy, there is no connection here to a perpetual motion machine. Perpetual motion machines are not power generators, they are simply machines that don't run down on their own, and that brings in the second law.

So the real issue here is with the second law-- clearly a huge T difference has lower entropy than the original state of a huge T that is the same on both sides. So how did the Demon achieve that with just a few decisions? If we have cannonballs, we have the same number of decisions, but a much larger T difference, so isn't that a contradiction or paradox?

No, it's fine. We just need to understand the connection between T and entropy. The first thing to do is get rid of the ridiculous units here-- we multiply T by k and divide S by k, so now S is S/k and means the natural log of the number of configurations N that are in the class associated with S, and T is kT and should be interepreted as the energy scale of changes that affect S/k. Specifically, an energy change of kT corresponds to augmenting S/k by one, which also corresponds to "one binary decision" about opening or closing the door. So you immediately see the key issue here-- if each cannonball has a huge energy, then one decision about the door automatically has a big effect on the energy difference, but not a huge effect on the entropy decrease! One decision by the Demon is always 1 unit of entropy lost in the gas, via the T difference it creates, regardless of the energy per cannonball. At any T, one decision corresponds to kT of energy added to the higher-T gas, which always lowers the entropy by 1, even if T is huge.

So in other words, we do not magnify the entropy consequences of the Demon's decisions simply by magnifying the energy of the cannonballs-- that is not the connection between entropy and energy. If there is already a huge energy per cannonball, than the entropy is "less impressed" by big energy shifts involving opening and closing the door. Either way, one bit in the head of the Demon is one kT of energy that is getting added to the high-T, high-S gas and removed from the low-T, low-S gas, and that always results in a net decrease in S/k of 1. But of course it also results in a net increase in the S/k in the Demon, so there is no issue, and you need to ignore (or "erase") that entropy increase in the Demon to get the "paradox."

So the bottom line is, the "paradox" stems from not understanding the fact that when the energy scale is magnified, the entropy consequences are not-- because at higher energy, each "decision" corresponds to more energy, but at higher T, more energy is needed to have the same impact on the entropy. The bottom line is, in thermodynamics, the Demon does not have "intention" or "purpose" that motivates his actions-- he is just another "dumb gas" that will only do what he does if it results in a net increase in entropy in the total Demon+gas system. He does what he does because it is a more likely configuration of the whole system, that's "intention" in themodynamics. And there is no issue with extracting work from the gas-- if you have access to an entropy-generating mechanism (like a brain), you can use it to drop the entropy somewhere else, and extract work in a way that conserves energy overall. This just means that brains are even more important to have in situations where there is access to high temperature, because each decision by the brain has a relatively larger energy consequence, which is also why you need good brains with their "fingers on the button" if you see what I mean!

 

[AdrianMay]

Thanks Ken, you seem to have got this pointing in the right direction, but I have one last misconception.

I already mentioned the question of how fussy the demon is. Where in the probability distribution of temperature per molecule does he set his threshold for opening the door? Can we really say that the entropy decrease of the gas per decision is independent of the threshold, and/or can we say that the entropy value of the bits in his head are independent of the threshold? I'd be tempted to say nO and yES respectively, which would resuscitate the problem.

I still haven't really understood this energy-entropy connection. Let's just banish k by an appropriate choice of units, as Einstein banished c and QM people banish h and so on.

Are you saying that one bit is just one bit of negentropy = ln(2), irrespective of any temperature? I think so. I think you'd only care about T if you cared about energy, which you already said was irrelevant because I can't make a power station out of this anyway.

But the threshold does affect the number of phonons being moved over the divide in each decision. That's the point right? We're talking about the number of ways we can distribute a known number of phonons over a known (but inconstant) number of modes. If we can sort a lot of phonons in one go then we've made a big difference to the number of ways the whole system can be arranged. So the earlier nO seems supported.

How about the yES?

 

[Rap]

...And there is no issue with extracting work from the gas-- if you have access to an entropy-generating mechanism (like a brain), you can use it to drop the entropy somewhere else, and extract work in a way that conserves energy overall...

So the bottom line is that yes, Maxwell's demon can take a high-entropy situation (like two containers of gas at the same temperature) and turn it into a low-entropy situation (the two containers have different temperatures). The first law is not violated - the total energy of the gas in the two containers is the same before and after. The second law is not violated - the lowered entropy of the two gases is more than offset by the increased entropy of the demon.

The lowered entropy (different temperatures) of the two gases can now be used to run an engine for a little while, until the temperatures are equal again (but lower than before). Then the demon can lower the entropy again, getting different temperatures again, which can then run the engine again for a little while (not delivering as much energy as before, because of lower temperatures). This could go on and on, but you could never get that engine to produce more energy than the amount of kinetic energy in the original gases.

The thing about the cannonball gas is only that there is a lot of energy there to be converted (the cannonball-gas temperature is so huge and there's a mole of cannonballs).

Is this right?

 

[AdrianMay]

Here's a completely different point: Bennet's take is that the Demon has to erase a bit when he forgets what he observed, which means setting a bit (that might have had any value) to a distinct value, which means lowering entropy and thus using energy. That's supposed to help. But now that Ken has unravelled energy and entropy it seems more like a hindrance.

The demon doesn't have to erase any bits.

He stores the bit by letting the molecule through!

Oops: wasn't there a change of sign somewhere? I think so. I say the demon picks up negentropy from the chamber he observes and dumps it in the one behind the door. He doesn't need much RAM and his entropy is steady. Simple. Bennet's take is considerably more convoluted and I think he sneaks in a change of sign by using the very law that's under attack - the 2nd.

Surely the demon was always maintaining a steady state - nobody ever asked him to store any internal state.

It's multiply circular to argue that the demon might be in danger of acquiring a long memory when he never needed to, then to say that he has to take precautions against that, and that the precautions would seem to reduce entropy (although the danger would have reduced entropy as well) and then to blithely assume that the second law dictates some energy requirement when the second law is the very thing under attack by the paradox we are trying to solve.

Whatever thermodynamic arguments are raised for claiming that the demon has to use energy, increase entropy or whatever, it should be possible to ground them in mechanistic, Newtonian style physics. Otherwise, there's a lot of danger of circularity.

 

[Ken G]

The thing about the cannonball gas is only that there is a lot of energy there to be converted (the cannonball-gas temperature is so huge and there's a mole of cannonballs).

Is this right?

Yes, that's how I see it as well.

 

[Ken G]

I already mentioned the question of how fussy the demon is. Where in the probability distribution of temperature per molecule does he set his threshold for opening the door? Can we really say that the entropy decrease of the gas per decision is independent of the threshold, and/or can we say that the entropy value of the bits in his head are independent of the threshold? I'd be tempted to say nO and yES respectively, which would resuscitate the problem.

I think your idea of looking at different thresholds is digging deeper into the details of the situation, but should not encounter any paradoxes. The Demon must increase entropy in the process of thinking and processing the information, and that allows the entropy in the gas to drop, and still have a net result that will actually occur spontaneously (the Demon has to function spontaneously here, it is ruled by thermodynamics also). But this doesn't mean that we know how much the net entropy will increase per decision-- that's what depends on the threshold and all the other details of the decisionmaking process. Thermodynamics doesn't tell us how human thought occurs, for example, we don't understand why my writing this, instead of something else, represents the largest possible increase in entropy. But if one takes the thermo perspective, that is the reason I am writing this. So even though we don't understand why the Demon sets the threshold where it does, we can calculate the decrease in entropy that occurs in the gas per decision. I'm not sure we really have a good model of what a "decision" actually is, but the main point is that once we get a model of that, the entropy consequences don't depend on the overall scale of the T of the gas (whether atom or cannonball), because what T effects is the conversion from the entropy scale to the energy scale, not the entropy scale itself (entropy scales like E/T, so cannonballs or atoms work the same for entropy). That's why decisions are more important when the energy scale is larger, but the entropy of a decision is more or less always the same (depending on how we are modeling what a "decision" actually is). Hence every decision you make has the potential to change your life, it all depends on how that decision interacts with other scales that don't relate to entropy.

 

[AdrianMay]

Yes, that's how I see it as well.

Me too, but what about:

is more than offset by the increased entropy of the demon

 

[Ken G]

Surely the demon was always maintaining a steady state - nobody ever asked him to store any internal state.

I don't think whether or not the Demon has a good memory is all that important for the entropy, unless you invoke more complex rules about "decisions" that involve memory. What is mostly important about the Demon is that it be an entropy generating machine, preferably at as low an effective temperature as possible (so you don't have to feed it so much to get it to make decisions). We can have the Demon be in a steady state, but we have to account for some kind of environment (with a low effective temperature) that it can be increasing the entropy of as it "thinks." If that environment is the gas itself, then its temperature is too high to get the Demon to function, and we won't be able to get the gas to decrease its entropy if we keep track of everything happening. That I think is the source of the "paradox"-- we are ignoring the need for the Demon to have access to its own environment, and the environment has to be something different from the gas for this to work. The one thing you can "take to the bank" is that the action of the Demon's decisions increase the entropy in total-- they are never "more than offset" by anything the gas is doing. Increasing the T of the gas does not create such an offset, it only increases E and T but not E/T, i.e., not the entropy decrease in the gas per action by the Demon.

 

[Rap]

I don't think whether or not the Demon has a good memory is all that important for the entropy, unless you invoke more complex rules about "decisions" that involve memory. What is mostly important about the Demon is that it be an entropy generating machine, preferably at as low an effective temperature as possible (so you don't have to feed it so much to get it to make decisions). We can have the Demon be in a steady state, but we have to account for some kind of environment (with a low effective temperature) that it can be increasing the entropy of as it "thinks." If that environment is the gas itself, then its temperature is too high to get the Demon to function, and we won't be able to get the gas to decrease its entropy if we keep track of everything happening. That I think is the source of the "paradox"-- we are ignoring the need for the Demon to have access to its own environment, and the environment has to be something different from the gas for this to work. The one thing you can "take to the bank" is that the action of the Demon's decisions increase the entropy in total-- they are never "more than offset" by anything the gas is doing. Increasing the T of the gas does not create such an offset, it only increases E and T but not E/T, i.e., not the entropy decrease in the gas per action by the Demon.

If the demon is moving low energy particles to one side of the partition between the two systems initially in equilibrium, then the demon is lowering the total entropy of the two systems. In order for the second law to hold, the combined entropy of the two systems and the demon must increase. What I meant was that the entropy decrease of the two systems is more than offset by the entropy increase of and by the demon. Now I am trying to figure out how the demon's entropy unavoidably increases. Part of it is that the measurement process increases entropy. In the case of the cannonball gas, shooting photons out to measure position and momenta of the cannonballs, I think. The other part is the modifications of the state of the demon as a result of these measurements. Thats the part that has me thinking right now.

 

[Ken G]

Yes, the Demon must increase entropy when it decides to open or close the door. Exactly how that happens I'm not that clear on, my comments were around trying to remove the idea that the energy of the particles mattered. I think the basic idea can be understood if we imagine a situation where you have a bunch of coins on a floor that can shake. First you shake the coins and get, let's say, 50 heads and 50 tails. You register which ones are which, and glue down all the heads, and shake the floor again. It's clear that eventually you will have all heads, so the entropy of the coins is decreasing. That has to be made up by the entropy associated with noticing which ones are heads and which are tails. I think that "noticing" part must mean that a bit in the brain in effect substitutes for the bit normally associated with the coin, such that the coin can be glued down, "removing" that bit from the entropy, without decreasing the total entropy because that bit is now in the head that decided to glue the coin down. In a sense a copy of the information has been made, in order to make the decision, so regardless of what decision is made, that copy retains any lost entropy.

 

[AdrianMay]

I think we're all in about the same place now. We agree that:

* The first law was never in danger, even if the second law was, because we can just look at all the balls on the level of kinetic energy.
* We only use temperature in dU=TdS, but if we're not attacking the 1st law we don't even need that equation, it's all about the entropy alone.
* Cannonballs and molecules are about the same thing because of the above
* We'll try to fix the paradox in the elastic case first. (I'd suggest considering inelastic complications if that fails.)
* The usual strategy for rescuing the 2nd law is to say that the demon's entropy increases, but none of us seems to know how or why.

Do you both agree that we all agree with the above?

I think that if we say that the demon is in a steady state, then by definition we'd have said that his entropy is constant and ruled out the standard rescue, so we'd better be careful about calling him a steady state, but frankly I don't see why I can't say that.

My fussiness argument from before didn't make sense though. Even if you only let the top percentile through (as opposed to the top 50% as usually imagined) then you are still making one decision per molecule. So on average, each decision sorts as many phonons as there are phonons on the average molecule, which is given by the temperature.

I think another possible strategy would be to show that he succeeds in decreasing the entropy of the whole system including himself, but only if he receives energy from outside with which to do work. That would be legal because the entropy increase occurs wherever he got the energy from. Then the temperature would be relevant because the work he has to do is TdS if he's planning on reducing entropy by dS. It would not be fair to put the cart before the horse and say that he must do work because the 2nd law says so, which I think is what Charles Bennet's argument sneaks into the soup somewhere around the bit erasure stage. I think the real purpose of Bennet's argument is to show that erasing a bit costs kT.ln(2) of energy assuming the 2nd law applies to Maxwell's demon. I don't think it was intended to rescue the 2nd law by throwing in a new axiom about erasure.

I emailed him BTW, but that might have been inappropriate. How would something like this typically escalate in a professional way? Letter to a journal or something?

 

[Rap]

I think the basic idea can be understood if we imagine a situation where you have a bunch of coins on a floor that can shake. First you shake the coins and get, let's say, 50 heads and 50 tails. You register which ones are which, and glue down all the heads, and shake the floor again. It's clear that eventually you will have all heads, so the entropy of the coins is decreasing. That has to be made up by the entropy associated with noticing which ones are heads and which are tails. I think that "noticing" part must mean that a bit in the brain in effect substitutes for the bit normally associated with the coin, such that the coin can be glued down, "removing" that bit from the entropy, without decreasing the total entropy because that bit is now in the head that decided to glue the coin down. In a sense a copy of the information has been made, in order to make the decision, so regardless of what decision is made, that copy retains any lost entropy.

I have no idea where I am going with this. I have been reading http://arxiv.org/pdf/physics/0210005v2 on Landauer's principle.

If a demon has 10 unglued coins, shakes the table, then before the demon looks at the coins, he describes the situation using entropy - 10 bits of entropy or 10 bits of missing information. Once he looks at them, he says zero entropy, and 10 bits of information in the demon's head. No uncertainty, no missing info, no entropy. The demon glues down all the heads (say 6 of them), shakes the table again. Before he looks, 10-6=4 bits of entropy, after he looks, none. If he remembers the original configuration, he now has 14 bits in his head. If he forgets the original configuration, he has 4 bits in his head, but by Landauer's principle, "forgetting" creates entropy, so those forgotten 10 bits are now demon entropy in the form of heat.

Suppose of those four, 2 were heads, 2 tails, so its now 8 heads, 2 tails. If he forgets the original configuration, its just 10 bits in his head. This goes on and on until there are 10 heads, zero coin entropy as far as the demon is concerned. If he is forgetful, he will have 10 bits in his head, all heads. If not, he will have something like 10+5+2.5+...=20 bits on average. So if he forgets, there will be 10 bits in his head and 10 bits of forgotten (heat) entropy.

To a scientist outside, who just measures the number of heads and the number of tails, not which are heads and which are tails (analogous to thermodynamic measurements, not knowing everything), he sees 6h,4t and then 8h,2t, etc. and he says coin entropy is decreasing. He is treating the coins as indistinguishable, while the demon is treating them as distinguishable, so their idea of entropy is different. To the scientist, 6h,4t has an entropy of H(6h,4t)=log2(B(10,6)), where B(a,b) is the binomial coefficient a!/b!/(a-b)!. That's about H(6h,4t)=7.7 bits. The scientist, looking at 8h,2t, says that's an entropy of H(8h,2t)=5.5 bits. He says coin entropy is decreasing.

As the scientist watches things, the entropy goes to zero - H(10h,0t)=0. He wants to know what happens to (his idea of) the entropy of the demon during this process.

When there are 6h,4t, the scientist will say there are 7.7 bits of coin entropy, and 7.7 bits of entropy the demon forgot and turned into heat for a total of 15.4 bits of entropy. When it goes to 8h,2t, the scientist will say there are 5.5 bits of coin entropy, and the demon forgot a total of 7.7+H(2,2)=10.3 bits of entropy for a total entropy of 15.7. Total entropy is increasing. This is kind of artificial, 6h,4t and 2h,2t are close to what you would expect - 50% heads, 50% tails. If the ratios were much different, I guess you could get a decrease in entropy, but for large numbers, no. On average, after two trials, you should get 10+5=15 bits of entropy, or something.

When there are finally 10 heads, the scientist will say there is zero coin entropy. If the demon is forgetful, the scientist will know he has 10 bits in his head, all heads, so zero demon entropy and something like 10+5+2.5+...=20 bits of forgotten entropy. It looks like the second law is saved.

Maybe that distinguishable/indistinguishable thing was a needless complication.

LOL - this reminds me of the discussion we had on Schroedinger's cat.

 

[AdrianMay]

In the absence of a challenge to the 1st law, the demon is going to have to do without his air conditioner and reach, say, the average temperature of the two chambers.

But I'd dispute that this temperature has anything at all to do with his trapdoor. I reckon the position of the trapdoor is the only entropy or internal state that he needs. Most of the time there are no molecules around and he just sleeps. When a molecule comes along, he takes a decision and stores it by opening or closing the door. No other RAM or state is involved. When the molecule is through, he just leaves the door where it was and goes back to sleep. When the next molecule arrives, he sets the door accordingly, thus erasing the memory of the previous decision. So every time he adjusts the door, which is once per molecule, he goes from any of 2 possible states to a precise state, which reduces the entropy of the demon-door by ln(2). Not (I claim) T.ln(2). (Banishing k by appropriate choice of units.)

That would be generous, but I might also claim that flipping the trapdoor goes from one known state to one known state, both of which had entropy of ln(1)=0.

I think this is basic to the definition of entropy. All that stuff about T came in when hot gases were considered. The reason you need a factor of T there is that a molecule is not the interesting unit here, rather, the phonon is, and a molecule is basically a bus load of phonons. T measures how many phonons are on the bus (kind of.)

No such argument can be applied to the trapdoor. Maybe the trapdoor consists of a big warm protein molecule with a kink in the middle that easily flips this way or that, but hates to be in the middle. The whole molecule is stuffed with phonons reflecting the ambient temperature, but they're all on a leg that's flipped left or right. Either way, there are just as many available states, and the entropy is the same. For a different entropy value, you'd have to consider some quantum superposition of left and right, or a population of such molecules, such that you could somehow speak of a left-and-right or left-or-right state. That would be more disordered. But we never needed to propose a moment in time when we didn't know whether the trapdoor was open or closed, and I think we can flip back and forth without any entropic consequences. Open and closed are equally disordered, irrespective of temperature.

I hope somebody can shoot that argument down, because my universe is turning distinctly pear shaped and I'm losing sleep.

 

[Rap]

In the absence of a challenge to the 1st law, the demon is going to have to do without his air conditioner and reach, say, the average temperature of the two chambers.

If that is the case, then the entropy acquired by the demon will yield an energy of the order of the energy removed from one side and deposited in the other. I think this might cause the engine not to work.

But hey, if the demon knows which molecule will hit where, why not just sidestep a molecule coming at him? If we look at a human demon and a cannonball gas, the human can just step out of the way when he sees a cannonball coming at him at 1 meter/sec. If you have a problem with the energy he will expend, then go to a gas of aircraft carriers.

 

[AdrianMay]

The paper Rap found confirms it: you can't use information theory to rescue the 2nd law from the demon, rather, that stuff assumes that even the demon obeys the 2nd law.

That would seem to mean that the demon is still as scary as when Maxwell invented him. Perhaps we could actually make one.

 

[Rap]

The paper Rap found confirms it: you can't use information theory to rescue the 2nd law from the demon, rather, that stuff assumes that even the demon obeys the 2nd law.

That would seem to mean that the demon is still as scary as when Maxwell invented him. Perhaps we could actually make one.

I interpreted it to mean that you can make one, and the second law will not be violated.

 

[AdrianMay]

I interpreted it to mean that you can make one, and the second law will not be violated.

Not me. I think he's saying that if you look at it Maxwell's way and consider the 2nd law disproved, then none of that information theory stuff even exists. They do the information theory by assuming that the demon is sociable enough not to break the 2nd law, then they figure out exactly what he can and can't do within those rules. But there's still no argument to say he will follow the 2nd law unless you just faithfully believe that.

People then make the mistake of taking the results of information theory as proven no matter what, and using them to protect the 2nd law from the demon. But that would be circular. Bennet himself is acknowledging this circularity in the paper you found.

 

[Ken G]

I believe the assumption that the Demon obeys the second law is a natural one to make, in lieu of a good theory of how "thought" works. According to thermodynamics, thought, like any other process of a large system, is "spontaneous", which simply means it is ruled by the principle that more likely things happen and less likely things don't happen. That's the only assumption behind the second law. So I think any difficulty we have with the Demon is simply traced to two problems:
1) we don't have a good model for what a decision is, thermodynamically, though we do see some connection between information entropy and the ability to answer questions that cull out possibilities (so the entropy of the gas can be seen as related to the number of questions needed to cull out its actual state from the set of states that physics deals with when it talks about the behavior of a gas), and the mind that is doing the culling. For the culling to be done physically, like separating hot and cold components of gas, the action of the mind must fill in for, or make a "copy" of, the entropy that is being lost from the gas.
2) We aren't accounting for the environment in which the Demon functions. This is what leads to ideas like the Demon being in a "steady state" if he can forget the decisions he makes, but I don't think it matters what the Demon remembers, what matters is that a brain is also a system, and must function in an environment. That environment must gain entropy for the brain to function spontaneously-- you can't have the brain "getting lucky" when it follows a rule, you must program it to follow that rule, and that program must play out as the spontaneous response of some entropy-generating environment. That's what makes the program reliable in the first place.

 

[Rap]

Not me. I think he's saying that if you look at it Maxwell's way and consider the 2nd law disproved, then none of that information theory stuff even exists. They do the information theory by assuming that the demon is sociable enough not to break the 2nd law, then they figure out exactly what he can and can't do within those rules. But there's still no argument to say he will follow the 2nd law unless you just faithfully believe that.

I have to disagree with this. The information theoretic idea of the demon is not that he is "sociable enough" not to break the second law, it is that by acquiring and possibly forgetting the information needed to decide when to open the door, he is unavoidably acquiring enough entropy to offset the decrease in entropy he produces by selectively passing fast particles from one side of the partition to the other.

We have to make clear two scenarios by which the second law is saved:

1) Maxwell's demon succeeds in reducing the total entropy of the two gases by selectively moving fast particles into one chamber. The second law is saved by the increase in entropy of the demon.

2) Maxwell's demon fails to reduce the total entropy of the two gases. The second law is saved, but the entropy balance sheet is unclear to me.

I have read Earman and Norton, and the Bennett paper, but not with anything near complete understanding. As I presently understand it, Earman and Norton reject the information entropy defense of the second law, while Bennett supports it. Earman and Norton present two principles - Szilard's and Landauer's, as information-theoretic explanations of how the second law is not violated, and disputes both.

Szilard says there is an entropy cost in acquiring information. Bennett disputes this. Me, I don't know. Looking at the cannonball gas, the demon needs photons to measure the position and momenta of the cannonballs, and I haven't thought about the entropy considerations here. Earman and Norton point out that if you really have equilibrium in this cannonball gas, there will be Planckian radiation at the huge temperature of the cannonball gas, which is why you need high energy photons to measure their position. Now I understand where the idea that high energy photons are needed came from. This is a good point, the cannonball gas without photons is not in equilibrium, and Maxwell's demon is not operating on equlibrated gases.

Landauer says there is an entropy cost in erasing information. Bennett agrees, and says this saves the second law in the case of a forgetful Maxwell's demon. I accepted this in my analysis of KenG's "coin entropy" scenario. I tend to think this is true. If you forget information, then there is missing information, which is information entropy, which is thermodynamic entropy.

People then make the mistake of taking the results of information theory as proven no matter what, and using them to protect the 2nd law from the demon. But that would be circular. Bennet himself is acknowledging this circularity in the paper you found.

I'm not sure what "results of information theory" you are referring to here.

 

[Ken G]

Now I understand where the idea that high energy photons are needed came from. This is a good point, the cannonball gas without photons is not in equilibrium, and Maxwell's demon is not operating on equlibrated gases.

I actually don't see that at as a good point, and I'm not sure those source aren't getting hung up on fairly irrelevant issues. The concept of equilibrium must be an effective, not literal, concept in physics-- we must be able to talk about equilibrium of what we care about, not necessarily equilibrium of everything. The second law is useless if we interpret it as a law that only applies in strict equilibrium, because no such situation exists anywhere. The patent office would have to reopen all the cases of perpetual motion machines that they refused to consider on the grounds that they violated the second law, if the inventor says "but my invention works in the corner of a room, where there is no strict thermodynamic equilibrium". The second law does not say that "entropy increases in strict thermodynamic equilibrium", because in strict equilibrium nothing changes, including entropy. Instead, the second law is about characterizing small deviations from equilibrium, and only in regard to whatever is the critical mode of behavior that is under study. For example, the universe is bathed in a very low-temperature neutrino bath, and no gas at normal temperature is in equilibrium with that. Saying that non-equilibrated photons in the cannonball gas invalidates the second law would be like saying that neutrinos always invalidate it. But actually this is only true in situations where the thermal neutrinos matter in some way, and it would only matter for the Demon and the cannonballs if thermal photons were somehow being used in the apparent entropy violation.

I don't know how those sources are arguing their points, but the summaries sound wholly unconvincing to me. I see the situation as much simpler than worrying about whether brains remember or forget, or whether they are or are not bathed in hot photons or hot neutrinos. It is simply that if you want to make a decision regarding some information, you have to be able to process that information, and information processing requires an environment. That environment will necessarily increase its entropy to at least "copy" the information being processed, and even then only if it is perfectly efficient. So the Demon will always generate more entropy than it destroys, and it will never matter whether the Demon is being bathed in photons or if it has a memory. Saying more probably requires having some specific model for how the Demon manages to "think", but much of the purpose of thermodynamics is being able to say things independently of the details of that kind of model, just as we don't have to talk about the mass of the cannonballs or how much they compress before they elastically rebound.

Landauer says there is an entropy cost in erasing information. Bennett agrees, and says this saves the second law in the case of a forgetful Maxwell's demon. I accepted this in my analysis of KenG's "coin entropy" scenario. I tend to think this is true. If you forget information, then there is missing information, which is information entropy, which is thermodynamic entropy.

Yes, I agree that one cannot get rid of entropy by forgetting. But what this says is that remembering or forgetting is irrelevant. One cannot argue the Demon is cyclic or in steady-state by letting it forget, because the entropy increase is in the Demon's environment, not its memory. One can treat the memory as part of the environment, but if forgetting occurs, then it's not the whole environment. All violations of the second law invariably boil down to not considering the whole environment, and I'd say that's just what is going on here.

 

[Rap]

I actually don't see that at as a good point, and I'm not sure those source aren't getting hung up on fairly irrelevant issues. The concept of equilibrium must be an effective, not literal, concept in physics-- we must be able to talk about equilibrium of what we care about, not necessarily equilibrium of everything.

Agreed - I didn't mean to imply that lack of equilibrium with the photons was deal-killing, only that it was something that needed to be thought about, and hopefully rejected. I tend to agree with you - it can be rejected.

It is simply that if you want to make a decision regarding some information, you have to be able to process that information, and information processing requires an environment. That environment will necessarily increase its entropy to at least "copy" the information being processed, and even then only if it is perfectly efficient. So the Demon will always generate more entropy than it destroys, and it will never matter whether the Demon is being bathed in photons or if it has a memory.

You are assuming that information entropy and thermodynamic entropy are equivalent via Boltzmann's constant, which I agree with, but apparently others do not.

Yes, I agree that one cannot get rid of entropy by forgetting. But what this says is that remembering or forgetting is irrelevant.

Yes.

But let me ask, just to be very clear - Do you think that a demon could successfully reduce the entropy of the two systems, with the idea that the second law is not violated due to the increase in entropy of the demon?

 

[Ken G]

You are assuming that information entropy and thermodynamic entropy are equivalent via Boltzmann's constant, which I agree with, but apparently others do not.

Yes, but to me that's the point of the Demon-- to show that they must be equivalent. Otherwise, we can substitute one for the other, and it's bye-bye second law. Perhaps those who don't see the equivalence don't think the second law is correct, but I'm not buying those wares. I hold that if one has a model for how the information is actually being processed, including the environment that is doing it, then the equivalence between information and thermodynamic entropy becomes clear-- it all boils down to the need for spontaneous results to be those which have a higher probability of occurring, and the sole reason they have a higher probability is that they represent larger entropy.

But let me ask, just to be very clear - Do you think that a demon could successfully reduce the entropy of the two systems, with the idea that the second law is not violated due to the increase in entropy of the demon?

Yes, the Demon can separate hot and cold gases, and it can do it by increasing the entropy even more in whatever environment the Demon uses for a brain, but the energy scale of the Demon can be vastly less than the energy scale of the gas. The latter would just mean that the Demon has a lower effective temperature. Hence the Demon cannot be born from the thermodynamic equilibrium it is disrupting, it needs some separate existence that must also be analyzed-- not treated like some magical process of intention, and not just some program and memory, but a whole thermodynamic environment of its own. But the Demon can be used to extract heat from a gas-- indeed there are many ways to do that, which violate neither the first nor second laws.

 

[AdrianMay]

OK let me quote from that paper.

First the top line: Bennet, 2011.

Then we have:

One of the main objections to Landauer’s principle,
and in my opinion the one of greatest merit, is that raised
by Earman and Norton [3], who argue that since it is not
independent of the Second Law, it is either unnecessary
or insufficient as an exorcism of Maxwell’s demon. I will
discuss this objection further in the third section.

So then we excitedly skip to the third section:

Earman and Norton have pointed out with some jus-
tice that Landauer’s principle appears both unnecessary
and insufficient as an exorcism Maxwell’s demon, be-
cause if the Demon is a thermodynamic system already
governed by the Second Law, no further supposition
about information and entropy is needed to save the
Second Law. On the other hand, if the Demon is not
assumed to obey the Second Law, no supposition about
the entropy cost of information processing can save the
Second Law from the Demon.
I would nevertheless argue that Landauer’s princi-
ple serves an important pedagogic purpose ...

That means he agrees. Pedagogic purposes are not logic.

You guys know me! Would I ever presume to question what a giant like Bennet says ?

 

[lugita15]

Yes, the Demon can separate hot and cold gases, and it can do it by increasing the entropy even more in whatever environment the Demon uses for a brain, but the energy scale of the Demon can be vastly less than the energy scale of the gas.

One of the formulations of the second law is the amount of usable energy in the universe can only decrease. Usable energy is essentially the kind of energy that a low-entropy system has, and entropy is essentially a measure of what percentage of the universe's energy isn't usable. So if Maxwell's demon isn't expending that much energy, is he increasing the amount of usable energy in the universe?

To put it another way, can Maxwell's demon extract more usable energy than a Carnot engine? If we take the fumes coming out of the (ideal) combustion engine of a car, can we feed it into Maxwell's demon and get more energy out? (Let the molecules of the car fumes be cannonballs... It's a giant car, OK?)

 

[AdrianMay]

I also agree that we don't care about equilibrium - that's for PM machines to worry about, not us.

I'd still like to quibble this enormous temperature business. If we don't care about equilibrium, I can use balls from the fridge. They don't have to move very fast so there's no relativistic doppler effect to worry about, so I think they will look to the demon like normal cool balls. He won't see them glowing like a quasar.

You guys keep saying "the entropy of the demon will increase." I'd like to see one piece of evidence for that which does not use the 2nd law as an assumption. This is E+N's point, and apparently Bennet accepts it. Without using the second law, there's no argument saying that there will be any increase of entropy anywhere. All that stuff (whether you think observing or forgetting is associated with +ve or -ve entropy or whatever) sits on top of the assumption that the demon does not break the 2nd law, so you can hardly turn it around and use it to defend the 2nd law.

I think Ken is saying there will be something else defending the 2nd law from the demon if we think hard enough about it, and then all that information theory will be valid. But my mind is blank. Yes we must examine the whole system including the demon's brain, and we'd be home and dry if we could prove (without invoking the 2nd law) that this brain must increase entropy somewhere else. But how? I'm at a loss as to how this should work. Presumably there's some light in the system so he can see what's coming, but I think I can let that light be totally disordered from start to finish, so no problem there. I think I can make his brain a steady state so no problem there either.

I think it might be helpful to get the whole temperature thing out of the way because it's just causing distraction. How about a gas of aromatic molecules in a mixture of left handed and right handed forms. We don't give a monkey's about the temperatures - we just want to sort the two forms. The chemical reflects circularly polarised light in the matching handedness. (A bit of detail is called for here but I think it can work.) So if the demon sees a lot of right-circular light in his vicinity he opens he door. We can use oodles of light because we no longer care about the temperature.

If there are no losses to the environment, the light energy won't get lost but it will tend to turn from a few high energy photons to a large number of low energy photons. But I think we can probably arrange for the interesting mode (the one where the chirality plays a role) to be the lowest frequency mode available. Failing that, we only need to show that we can sort a molecule without degrading a useful photon into two lower energy ones.

If this light stuff fails we can also make a chemical detector, like a nose.

I'm back to square one now - there may be losses, but I don't see how I can equate them with the entropy reduction without invoking the 2nd law.

 

[Rap]

This all seems to be making a mountain out of a molehill. The demon and the two gases are to be considered a single system. If an outside observer looks at the system, measuring only the thermodynamic states of the two systems and the state of the demon - a combination of logical state, mechanical state, and thermodynamic parameters, the outside observer has to ask, how many ways could such a set of measurements occur? Equivalently, how many microstates could yield this observation? The entropy will be k log(W) where W is that number of ways. The second law says that will increase (or stay the same, but probably not).

The only way the demon is going to violate the second law is if the number of ways the demon could give rise to our measured state of the demon is less than the number of ways the entropy of the two gases could have been reduced. I think the second law will hold, so the number of ways will be larger, but part of these ways is a description of the logical state of the Turing-machine demon. That means that information theory must be in part invoked in the calculation of the state of the demon. Information theory applied to the "logical state" of the demon is just a way of figuring the contribution to the calculation of the total number of ways the demon could exist in his final state. If the logical state of the demon is the same as before it made the decision, and the mechanical parts are all in the same place, then heat must have been generated.

This all turns on what we define as the "macrostate of the demon". The crux of the problem is here, I think.

OK let me quote from that paper.

First the top line: Bennet, 2011.

...

So then we excitedly skip to the third section:

Earman and Norton have pointed out with some jus-
tice that Landauer’s principle appears both unnecessary
and insufficient as an exorcism Maxwell’s demon, be-
cause if the Demon is a thermodynamic system already
governed by the Second Law, no further supposition
about information and entropy is needed to save the
Second Law. On the other hand, if the Demon is not
assumed to obey the Second Law, no supposition about
the entropy cost of information processing can save the
Second Law from the Demon.
I would nevertheless argue that Landauer’s princi-
ple serves an important pedagogic purpose ...

That means he agrees. Pedagogic purposes are not logic.

You guys know me! Would I ever presume to question what a giant like Bennet says ?

Ok, I'm still trying to understand these papers, and if I can summarize the above, I read that E&R point out that Landauer's principle is based on the second law and therefore applied to the demon, it will of course save the second law. Bennett agrees. But what does it mean to save the second law? What does it mean to "exorcise the demon"? Does it mean that the demon cannot reduce the entropy of the two gases (i.e. will fail in its purpose) or does it mean that it will (or can sometimes) succeed because the entropy of the demon increases as the entropy of the two gases decreases, yielding a net increase in entropy?

I think they mean the latter.

If the demon can succeed in reducing the entropy of the two gases, then what are the details of how its entropy increases? It seems to me that Landauer is giving an information-theoretic explanation of how this happens, consistent with the second law. Earman and Norton have three objections to Landauer's principle -

1) (pages 14-16) seems to me to be "who cares about the details, Landauer's principle upholds the second law, so everything is fine". Since I care about the details, I reject that.

2 and or 3) (pages 16 - 20) I could not find a delineation of two separate problems here, its all mushed together. I definitely do not understand these pages in detail. To begin with, E&N say Bennett requires that at the end of each decision by the demon, the demon (a Turing machine) must be reset to its original state, which involves erasure and the creation of entropy by Landauer's principle. (I don't think that's necessary, but ok, let's go with that.) They then give a case where erasure supposedly does not happen. They say that Zurek and Caves (Z&C) dealt with this problem by a more complicated definition of entropy involving algorithmic complexity of the Turing machine demon. etc. etc. etc. That's as far as I got.

One question I have is "are they assuming the temperature of the gases and the demon are the same or not?"

 

[Rap]

How about a gas of aromatic molecules in a mixture of left handed and right handed forms. We don't give a monkey's about the temperatures - we just want to sort the two forms.

This would avoid lugita15's problem which needs to be considered, if not just to dispose of it.

Note that the two actually can in principle be separated. There can exist in principle, a membrane that is permeable to one, but not the other. If you have a volume of mixture, you put the membrane at one end and start slowly pushing on it. It gets to the middle, and the pressure on the pushed side is lower than on the other side. You replace the membrane with an impermeable one, let the pressures equilibrate, put in the permeable membrane, and push again. This process will eventually "distill" one type from the other. When you calculate up the total work done by pushing, divide by temperature, you will get the new (lowered) entropy of the system. The entropy (and energy) that the system has lost has gone into the pushing machine. If its your muscles, they will heat up, etc.

One of the formulations of the second law is the amount of usable energy in the universe can only decrease. Usable energy is essentially the kind of energy that a low-entropy system has, and entropy is essentially a measure of what percentage of the universe's energy isn't usable. So if Maxwell's demon isn't expending that much energy, is he increasing the amount of usable energy in the universe?

To put it another way, can Maxwell's demon extract more usable energy than a Carnot engine? If we take the fumes coming out of the (ideal) combustion engine of a car, can we feed it into Maxwell's demon and get more energy out? (Let the molecules of the car fumes be cannonballs... It's a giant car, OK?)

Hmmmm

 

[AdrianMay]

We should be able to express all this in the terms lugita is asking for, but I'm not sure how. If we dodge it with my aromatic gas for now, we have to come back to it. He's quite right that usable energy is an important consequence of this whole entropy thing, but I lost track of that when it turned out that I couldn't make a power station out of my cannonball gas. I thought I would be able to because of the usable energy thing.

If we can clarify this, we might get a line of sight on the business of how much work the demon will have to do, i.e., how much energy needs to be pumped into keep him running, and I think that'll turn out to be the crux of it.

> what does it mean to save the second law?

I think it means to come up with a catch in the demon challenge, without resorting to the second law. The catch ought to be mechanistic or statistical, and it should show that this demon system will observe the 2nd law of its own accord. Then, all that Landauer stuff would rest on the 2nd law and be valid. But that's not happening right now, rather, they are calculating the performance limits of the demon such that it can do all things permitted under the 2nd law but nothing that's forbidden.

> the outside observer has to ask, how many ways could such a set of measurements occur? Equivalently, how many microstates could yield this observation?

Now we have a precise definition, and I think it's the correct one. But I see no reason why the demon needs more states in its entire life cycle than you can count on the fingers of one hand, so how could it contribute anything significant?

 

[Rap]

Now we have a precise definition, and I think it's the correct one. But I see no reason why the demon needs more states in its entire life cycle than you can count on the fingers of one hand, so how could it contribute anything significant?

No not the states in its life cycle, the number of microstates that could give its present macrostate. For a particle gas, classically, a microstate is the specification of the position and momenta of each particle in the gas. The macrostate is three thermodynamic parameters, like temperature, pressure, and number of particles.

For the demon, the microstate is the position and momenta of every molecule that makes up the demon. If the demon is a computer that measures nearby gas particle positions and momenta, makes some calculations using some mechanism, and then opens or shuts a trap door, and generates some heat, then what is its microstate? I think you can describe its microstate by the state of the registers in it and the position and momenta of every particle that would be considered "heat" but for the fact that we now know them. The "state of its registers" is where information theory steps in and contributes knowledge of the number of microstates the demon could be in. If we follow Bennett and require that the demon be in the same logical state after a measurement-trapdoor opening as it was before, then, if the second law holds, the whole calculation process had to generate some heat, and that heat represents the entropy generated by the demon. Landauer's principle is more specific, it says that erasure causes heat. So we want to look at the whole process by which the demon arrives at its conclusion and does what it does, and count up the erasures and add up the heat and hopefully say that it all fits together. I think it is the details of this process that Bennett and E&N are discussing.

This has, of course, ignored the contributions to entropy of the measurement process and the trapdoor opening process, which we are more or less ignoring, and maybe we should not be. Szilard's principle says the measurement process produces entropy, I think Bennett says no, not necessarily, me, I don't know, but if the inquiry into the demon's entropy production fixes things, then it doesn't matter. The trapdoor opening and closing - well, I'm willing to ignore that for the same reason.

The crux of the problem, and the thing that puzzles me, is the macrostate and microstate of the demon. It seems we can make different assumptions here, and get different results, but I expect all those results are in accord with the second law. Entropy is "missing information" and depending on how you define the demon's macrostate, you get different entropies. Its should be all ok nevertheless, though. Even with gases, you can suppose different levels of information knowledge, thus get different macrostates for the same situation, thus get different entropies, but no matter which description you use, the second law still holds.

 

[AdrianMay]

I think I can describe the microstates of my dog-leg protein. If the molecule is in a given macrostate (e.g. open, closed, thinking about it, etc) then the microstates are just the waqys of arranging thermal phonons over the molecule in that configuration and at that temperature. This is just a function of temperature if we're considering a specific configuration. Different configurations have different specific heats.

I think it's that simple. You'd only get inflation of the microstates per macrostate if the temperature was increasing. But we haven't shown that it will, at least, not without invoking the second law as Landauer, Bennet and friends do.

So I don't see a route in that line of thought. It still boils down to finding some mechanical reason why the demon has to get hotter. Then we'd be home and dry. Even if the demon then dissipated his heat into the chambers, we'd still be home and dry because that would still be the sought entropy increase.

But why can't he just be frictionless and flout the second law, leaving Landauer etc without a leg to stand on?

 

[Delta Kilo]

I think I can make his brain a steady state so no problem there either.

I'd like to see that :)

I'm pretty sure it's impossible and this is exactly the reason why the whole setup is not going to work.

If your computations return to back to exactly the same state, they are reversible and there is no particular reason for them to go forward, they are just as likeky to run backwards and release the molecules instead of capturing them.

 

[AdrianMay]

I'd like to see that :)

If your computations return to back to exactly the same state, they are reversible and there is no particular reason for them to go forward, they are just as likeky to run backwards and release the molecules instead of capturing them.

Hmm. Maybe there's something in that.

 

[Ken G]

One of the formulations of the second law is the amount of usable energy in the universe can only decrease. Usable energy is essentially the kind of energy that a low-entropy system has, and entropy is essentially a measure of what percentage of the universe's energy isn't usable. So if Maxwell's demon isn't expending that much energy, is he increasing the amount of usable energy in the universe?

The energy is already usable, because you have a container filled with hot gas (the temperature is associated with the average kinetic energy of the cannonballs, AdrianMay-- that doesn't mean the balls themselves are hot, indeed we can imagine they are at absolute zero internal temperature for all the difference it would make here), sitting in an environment of lower T. If you don't have that, say the whole universe is at that T, then the Demon's environment is also at that T, and then the Demon cannot work (it needs to be at a much lower effective T so it can increase entropy without using up all the energy being made extractable from the gas).

To put it another way, can Maxwell's demon extract more usable energy than a Carnot engine?

A Carnot engine can extract all the energy from a gas, it's all a question of the effective temperature of the brain making the decisions (or in the standard setup, the temperature of the exhaust reservoir). Like all thermodynamic engines, Carnot cycles participate with our intentions only so long as the total entropy increases in the cycle (or stays nearly the same). Frankly, I don't see what Bennett and Landauer etc. are on about, the situation all seems pretty clear. Unless their intention is to delve into the constraints on modeling how the Demon thinks, i.e., on just how it manages to increase the entropy of its environment, that's the only issue that seems subtle. I would say it is perfectly obvious that the Demon obeys the second law, and the only reason we might imagine giving it a "pass" on that law is that we are treating it as magic-- rather than as an actual physical system.

 

[Rap]

The energy is already usable, because you have a container filled with hot gas (the temperature is associated with the average kinetic energy of the cannonballs.

A Carnot engine can extract all the energy from a gas, it's all a question of the effective temperature of the brain making the decisions (or in the standard setup, the temperature of the exhaust reservoir).

I think what he means is that you cannot run an engine between two reservoirs at the same temperature. If the demon causes the two temperatures to differ, then and engine can extract "usable" energy. One of the gas systems is serving as the exhaust reservoir.

Frankly, I don't see what Bennett and Landauer etc. are on about, the situation all seems pretty clear. Unless their intention is to delve into the constraints on modeling how the Demon thinks, i.e., on just how it manages to increase the entropy of its environment, that's the only issue that seems subtle.

Yes, I think that's just what they are doing. I think its very interesting to try to understand how the demon increases its "entropy". I put that in quotes, because unless you can define a microstate and a macrostate, you cannot define entropy, and I am not always sure what constitutes a macrostate of the demon. If you demand that the demon return to its original state, then the macrostate is just a matter of temperatures, that's why they demand it, it simplifies things. But on the other extreme, if you say the demon never erases, just keeps using new memory, then what? Temperature does not change? In order not to violate the second law, the entropy has to be k times the information entropy of the demon's memory? That means the macrostate of the demon is one in which we cannot read its internal logical state and its entropy is the number of ways its memory could be configured given what we know about the two gases at that point. Does the second law therefore imply that there is no way we can communicate with the demon and read its logical state? Forgetful or not, the demon is not thermally connected to the system, yet it is acquiring or creating entropy to match the entropy it is removing, not by thermodynamic means, but by "logical" means.

 

[Ken G]

I think what he means is that you cannot run an engine between two reservoirs at the same temperature. If the demon causes the two temperatures to differ, then and engine can extract "usable" energy. One of the gas systems is serving as the exhaust reservoir.

What I'm saying is that you can run a heat engine between two temperatures, and you can run a Demon to create that temperature difference, and both stages increase entropy (the Carnot cycle can increase it very little). This isn't a problem with free energy, free energy depends on the context. A hot gas inside more of the same hot gas has no free energy, but a hot gas inside a cold or low pressure environment, or connected to a Demon that effectively has access to a cold environment, has lots of free energy. Free energy just means that there is some other class of states that are more numerous (possible just barely more numerous) in which that energy is somewhere else. That is just as true when the Demon is creating a T difference as when we are running an engine off that T difference. They both happen because entropy increases.

Does the second law therefore imply that there is no way we can communicate with the demon and read its logical state? Forgetful or not, the demon is not thermally connected to the system, yet it is acquiring or creating entropy to match the entropy it is removing, not by thermodynamic means, but by "logical" means.

The second law is way easier than that, I really don't know why they want to make it so complicated. It doesn't have anything to do with memory, unless memory is the way the entropy is increasing. If the memory isn't there, then it would just be some other way. The Demon's brain simply doesn't function unless it increases entropy, that's how it works. The entropy increase can be used to make decisions that drop the entropy somewhere else, but it can't operate to drop total entropy, or the behavior simply doesn't happen. More likely configurations do not evolve into less likely ones, that's all the second law says.

 

[lugita15]

The second law is way easier than that, I really don't know why they want to make it so complicated. It doesn't have anything to do with memory, unless memory is the way the entropy is increasing. If the memory isn't there, then it would just be some other way. The Demon's brain simply doesn't function unless it increases entropy, that's how it works. The entropy increase can be used to make decisions that drop the entropy somewhere else, but it can't operate to drop total entropy, or the behavior simply doesn't happen. More likely configurations do not evolve into less likely ones, that's all the second law says.

Ken G, I'm a bit uneasy about the philosophical attitude you seem to have concerning the 2nd law. It is not some iron-clad cosmic law of the universe, like the first law. It is, as you said, a statement that things tend to go from less likely configuration to more likely configurations. You make it sound as if machines cannot even in principle function if they decrease the total entropy of the system, as if the second law is what is driving them. But the second law is an effect, not a cause, of physical behavior, and it is a statistical rule, not a deterministic law; see the fluctuation theorem in statistical mechanics, which gives the probability that the entropy of a system increases or decreases.

For this reason, I think that instead of analyzing Maxwell's demon based on what the second law "permits" it to do, it's more useful to analyze it without assuming the second law, and thus finding out what makes the second law work in this case.

 

[Ken G]

You make it sound as if machines cannot even in principle function if they decrease the total entropy of the system, as if the second law is what is driving them.

Yes, that statement is correct-- no machine can function reliably unless its net effect is to increase entropy. It makes no difference if we imagine this constraint is "driving" them-- it's just a true constraint.

But the second law is an effect, not a cause, of physical behavior, and it is a statistical rule, not a deterministic law; see the fluctuation theorem in statistical mechanics, which gives the probability that the entropy of a system increases or decreases.

I did not say it is a "deterministic" law-- it is a constraint. You can't use it to determine what will happen because you can't always know what is most likely to happen without significant knowledge of the details. However, you can know what will not happen without any knowledge of the details-- the Demon will not work unless he increases entropy. Not having to know the details is the raison d'etre of thermodynamics.

For this reason, I think that instead of analyzing Maxwell's demon based on what the second law "permits" it to do, it's more useful to analyze it without assuming the second law, and thus finding out what makes the second law work in this case.

The reason that is not terribly useful is because you have a different analysis for every different Demon you can construct. The purpose of thermodynamics is to make general statements, not statements tied directly to some particular mechanism (which must be analyzed using mechanics, not thermodynamics). If you want to use mechanics to test thermodynamics, then you need a model of the Demon, but you cannot get "paradoxes" by imagining magical properties of the Demon-- you need an actual mechanism, and when you get that, you will see that the Demon either increases entropy, or doesn't work.

 

[lugita15]

The reason that is not terribly useful is because you have a different analysis for every different Demon you can construct. The purpose of thermodynamics is to make general statements, not statements tied directly to some particular mechanism (which must be analyzed using mechanics, not thermodynamics). If you want to use mechanics to test thermodynamics, then you need a model of the Demon, but you cannot get "paradoxes" by imagining magical properties of the Demon-- you need an actual mechanism, and when you get that, you will see that the Demon either increases entropy, or doesn't work.

But the second law of thermodynamics is a mere statistical consequence of the laws of mechanics. Thus, if the second law tells you that Maxwell's demon cannot reliably reduce total entropy, without actually knowing the inner workings of the demon, it should also be possible to draw the same statistical inference directly from the laws of mechanics. In other words, you are in essence redoing the proof of the second law from Newton's laws in the particular case where a device has the properties Maxwell's demon claims to. It is this kind of analysis that has allowed people to figure out some of the underlying reasons why the second law is an emergent property of mechanical systems, for instance the connection between information and entropy.

 

[Ken G]

But the second law of thermodynamics is a mere statistical consequence of the laws of mechanics.

I would say it must be consistent with any laws of mechanics, but it is independent of those laws, and it doesn't even really require any laws of mechanics be in place. Mechanics are the details, thermodynamics is what you can do without even knowing the details.

Thus, if the second law tells you that Maxwell's demon cannot reliably reduce total entropy, without actually knowing the inner workings of the demon, it should also be possible to draw the same statistical inference directly from the laws of mechanics.

Yes, if you have a model for the Demon. Do we have mechanical models of brains?

In other words, you are in essence redoing the proof of the second law from Newton's laws in the particular case where a device has the properties Maxwell's demon claims to. It is this kind of analysis that has allowed people to figure out some of the underlying reasons why the second law is an emergent property of mechanical systems, for instance the connection between information and entropy.

The "why" of the second law is independent of mechanics, it is thermodynamics. It all boils down to, entropy is our way of counting which collections of configurations contain more equally likely states, and hence are what will happen.