# Maxwell's Demon revisited

1. Feb 24, 2012

Hi Folks,

(Skip to next paragraph if you already know what M's D is.) Maxwell's demon was a counterargument to the second law of thermodynamics (and hence the first) involving two chambers of air connected by a trap door which this demon would open and close to let fast molecules go one way and slow ones the other. One chamber gets hotter and can perpetually drive a heat engine.

The catch apparently lies in the assumption that the demon himself can be neglected but in reality one has to consider the entropy associated with his decision. But this argument makes no reference to the size of the air molecules.

What if they were whopping great cannonballs? Can one still claim that the energetic equivalent of whatever bits he has to erase in his head is comparable with the power station you could drive that way? How do we even make the comparison between energy and entropy here?

Now that I think about it, I don't really understand why it would have been a problem in the first place. If I milk kinetic energy out of either chamber, all the cannonballs are gonna slow down eventually. But they did teach me in school that breaking the second law was tantamount to breaking the first.

Confused,

2. Feb 24, 2012

Another problem with the "erasing bits in his head" argument is that it doesn't mention the temperatures involved, or how fussy he is. The number of bits he has to erase is independent of whether he sets his threshold at 100^C or 1000^C. But surely the entropy reduction is different in the two cases.

3. Feb 24, 2012

### Andy Resnick

You have to erase the bits to ensure the process is cyclic- your end state is the same as the initial state.

4. Feb 24, 2012

I know I have to erase the bits, but I don't see how that problem can compare with an entropy reduction that seems to be in totally different units.

5. Feb 24, 2012

### Rap

I think this is a good question, and I don't know the answer (yet). A few points though - if you have a gas of cannonballs, say they have a mass of 1 kilogram each, and lets assume they collide elastically with each other, so that their internal thermodynamics doesn't matter, then their kinetic energy will be 3kT/2=mv^2/2 where v is their velocity, m is their mass, T is their temperature. If the cannonballs are moving at an average velocity of 1 meter per second, then their temperature will be T=mv^2/3k = 2.4 x 10^22 Kelvin.

Also, entropy is "missing information" and may be different for different people. If person A knows only the temperature, pressure, volume, etc of a gas, then A will calculate some entropy for that gas, but if person B knows the position and velocity (speaking classically, now) of every particle in that gas, then they will say there is no entropy. Person A will write the second law of thermodynamics as dU=TdS-PdV+µdN (µ is chemical potential, you have to include that when possibly changing the number of particles). Person B will write the second law as $dU=\sum m_iv_i^2/2$ where the index i runs over all particles. It won't be the second law, really, it will be a mechanical equation.

I guess I don't understand how the energy expended by the demon can compare to the energy of one "hot" cannonball being allowed to pass through the door. The demon can use photons to determine the position and energy of a cannonball, but the energy used is negligible compared to the cannonball. Same goes for the energy needed for the demon to record the information. Can we say the energy required to open and shut the door is negligible?

So this means that energy is flowing from one side to the other. The entropy of the system may be increasing, but the temperature of demon's process is way lower than the temperature of the cannonballs.

6. Feb 24, 2012

### Andy Resnick

There are two ways to make the comparison. The first is to associate entropy with the transfer of information: in this case, the information associated with measuring the velocity of a cannonball and deciding to open a door (or not). Shannon was the first (AFAIK) to work out that the transfer of 1 bit of information is equivalent to kT ln(2) units of energy, or a change in entropy of k ln(2).

Alternatively, there is the information required to specify the microstate itself- this has been worked out in terms of algorithms, and the most straightforward definition is the Kolmogorov complexity. The two measures are somewhat related (for example, the amount of information required to transmit instructions for constructing a power plant is related to the information required to actually construct the power plant), but Shannon's result is better understood and is the basis for several data compression algorithms.

Your example of a gas of cannonballs a very interesting one (especially if gravity is not important)- defining the temperature of such a gas is not so obvious: see, for example, the hard-sphere phase transition. The transition between fluid and crystal occurs at a particular volume fraction- not the conventional sense of temperature. In terms of thermodynamics, the entropy of a gas of cannonballs will also depend on the volume fraction.

7. Feb 24, 2012

### 256bits

The demon has to measure the speed of the cannonball by absorption of photons as you suggest, these are now high energy photons coming from a source of 2.4 x 10^22 Kelvin, which surely will increase the demon's temperature greatly is he is tiny enough to not have an energy expenditure of appreciable value when opening or closing the door. So the question is then how many cannonballs can the demon measure and let through the door before his own temperature rises to that of the cannonballs.
I don't think size of "molecules" matters for Maxwell's demon.

8. Feb 24, 2012

### Rap

The demon does not need high energy photons. If the "molecules" are cannonballs, a flashlight will be overkill, and still will not appreciably affect the energy or momentum of the cannonballs. The demon is not tiny now, a human size demon will be fine. Note that the cannonballs are not hot to the touch of the human size demon. A basic assumption is that the cannonballs collide perfectly elastically which means it doesn't matter what their conventional temperature is. The thermodynamic temperature of the "cannonball gas" is such that each degree of freedom holds energy kT/2 and if they have mass 1 kg and average speed 1m/sec, that gives 2x10^22 Kelvin. Lets assume they are so far apart that the total volume of the cannonballs is much smaller than the volume that contains them. A volume of 10^33 cubic meters will give about a mole of cannonballs that will be about a kilometer apart, on average. That volume amounts to a cube a million kilometers on a side, about the distance from the earth to the sun. This cannonball gas has enough cannonballs to be treated as a thermodynamic system. Now we have essentially an ideal gas of cannonballs. The density is about 2.4 x 10^-10 cannonballs per cubic meter. The pressure is about 8x10^-11 pascal. A person who knows only the pressure, temperature, and volume, for example, will assign an entropy to this system, the ideal gas entropy (Sackur-Tetrode, to within a constant, I guess).

Notice that the demon knows more than the temperature, pressure, and volume. The demon knows the position and momentum of the cannonballs in his vicinity. (They are big enough to be treated classically).

I don't think the size of the molecules matters either - which is why we should be able to prove that the second law is not violated for a human demon and a gas of cannonballs undergoing perfectly elastic collisions.

Last edited: Feb 24, 2012
9. Feb 24, 2012

Oh dear. I seem to have put the cat among the pigeons. Can we escalate this one? E.g. to Stephen Hawking or somebody like that ;-)

I think 256bits must be on to something with the temperature of the cannonballs thing. Real cannonballs would probably heat up as they bounced around, taking energy out of the translational modes. We'd better include rotation as well. If we said there were no other modes but translation, then I think we'd be talking about a quantum system which might be a totally different ball game (scuse the pun) and if we approximated the collisions to perfectly elastic I think we'd just be agreeing to wait a very long time but not talking about perpetual motion or equilibrium. So let's take the case where the energy spreads out over all modes that a real cannonball has and see if we still have a problem.

I think we do though because I can reduce the rate at which the demon gets sunbathed just by shrinking him and then I only need a very small fridge to keep him cool. I'm expecting to milk significant energy out of this contraption and I reckon I can spare that much for the fridge.

Using Rap's numbers as an example, can anybody figure out how much energy we can harvest if we naively ignore the bits in the demon's head? I'm so clueless about thermodynamics that it still looks like zero to me, which means I don't understand why breaking the 2nd law means breaking the 1st or what was so demonic about this trapdoor in the first place.

10. Feb 24, 2012

### Rap

We should deal with the simplest situation possible that keeps the problem alive. Cannonballs colliding inelastically is a needless complication, it means the system is out of equilibrium - the cannonballs have an internal temperature and a "cannonball gas temperature" and they are vastly different. Equilibrium occurs when the two temperatures are the same - i.e. the cannonballs are practically motionless and have absorbed all the kinetic energy they used to have and that energy is now heat energy. When talking about entropy, you have to include the "internal entropies" of the cannonballs and the "cannonball gas entropy", etc. etc. Huge complication for the demon to keep track of. Do we really want to worry about Maxwell's demon in a non-equilibrium situation when the problem does not demand non-equilibrium? No.

We can forget about angular momentum too. Elastic collisions will impart no rotational energy to any cannonball during a collision, either from an off-center hit or from a rotating cannonball transferring rotational energy to another cannonball. Just as the internal temperature of a cannonball is irrelevant, so is its rotational energy.

Elastically colliding cannonballs are like a simple monatomic ideal gas, and Maxwells demon should give the same results in both cases. The problem is kept alive - what happens if you go to a "cannonball gas"? The bits that the demon will have to record will simply be the positions and momenta of the cannonballs, he won't have to worry about the internal entropy bits of the cannonballs, the rotation of the cannonballs, etc. Much simpler, and I don't think we will require quantum mechanics to resolve the issue.

I'm not very good at Maxwell's demon, there must be something missing. Is it that the demon is required to be at the same temperature as the gas he is manipulating? If that is the case, then if you multiply the information entropy of those recorded bits times kT where T=10^22 or whatever, you get a very big energy, on the order of a cannonball's kinetic energy, and that plus the photon energy he used to measure the cannonballs may be more than the energy "generated" by the demon. I don't know.

Also, violating the second law does not mean you violate the first. The second law is being violated all the time by small fluctuations, the first law, classically never. Its only in the "thermodynamic limit" of an infinite number of particles that the second law holds exactly.

Last edited: Feb 24, 2012
11. Feb 25, 2012

The elastic case is clearly simpler, but I'm worried that it might be too soft on me, i.e., make it too easy for me to make this case that the "bits-in-the-head" argument is dodgy. Perhaps if we get a full blown paradox in the elastic case, the solution is that perfect elasticity isn't achievable in practice.

Not in theory either if the balls have non-zero diameter. That follows from relativity - there has to be a wave in the ball to tell the back face to change direction, and that wave can't go at infinite speed, so it's gonna bounce around and dissipate as heat.

Put it this way: it would be more satisfying to me to make my case fly in the inelastic case than just to get away with it in the idealised case.

As for rotation, you might be right that rotating modes won't get stimulated if the balls are *oily*, but if you're prepared to overlook the relativity argument, then we could also speak of perfect elasticity in the case that their surfaces have an enormous coefficient of static friction and therefore won't grind against each other. In that case, I think balls glancing off each other would end up rotating. I don't mind having oily balls though.

I'm not entirely convinced that the temperature of the demon is relevant. I reckon 1 bit of information is just that. Is it really applicable to take equations like dU = TdS ... which originally came from huge statistical systems like gases and apply them to single bits. I'd like to hear a more mechanical argument that a bit really really has to use TdS of energy when you erase it.

I also have my doubts about this 10^22 Kelvin. That would seem to imply that if I use 1kg balls of something inelastic like plasticine starting at absolute zero, then let all the KE turn to heat just by watching them all go splat, then they should reach this wild temperature. But in reality they only seem to heat up a few tenths of a degree or so. Perhaps the bug in the calculation is that pdV wasn't mentioned.

12. Feb 25, 2012

### Ken G

Listen to Rap, he's making good sense. The first case to understand is the purely elastic case, because we should be able to account for that situation. Also, we do not want to make the Demon be the same T as the cannonballs, indeed it will be crucial that the Demon either not have a T associated with him at all, or else have a much lower T associated with him than the cannonballs have. Finally, the energy per "cannonball" is indeed very much like their temperature here, so if we have a counterintuitive case to understand, it is the case where the "Demon" is generating a huge T difference across the "door" by using what would seem to be a relatively small number of decisions.

First let me say there must not be anything "dodgy" about the "erasing bits" argument, even in the case of cannonballs and high T-- that argument must hit the nail on the head even in that situation, so this is what we need to understand. I would say the first thing we should do is distinguish the first and second laws-- the first merely being conservation of energy, the second counting entropy. What seems "paradoxical" here is that if the Demon can take a gas that is everywhere the same (very large) T, and make some decisions about a door, he can generate a huge T difference, which could then run a heat engine and do work, in a way that looks completely cyclic so sounds like it could be repeated over and over. This does not yield any problems with the first law, because energy is still being conserved-- if the heat engine does work, the cannonballs lose energy, and since nothing replaces that energy, there is no connection here to a perpetual motion machine. Perpetual motion machines are not power generators, they are simply machines that don't run down on their own, and that brings in the second law.

So the real issue here is with the second law-- clearly a huge T difference has lower entropy than the original state of a huge T that is the same on both sides. So how did the Demon achieve that with just a few decisions? If we have cannonballs, we have the same number of decisions, but a much larger T difference, so isn't that a contradiction or paradox?

No, it's fine. We just need to understand the connection between T and entropy. The first thing to do is get rid of the ridiculous units here-- we multiply T by k and divide S by k, so now S is S/k and means the natural log of the number of configurations N that are in the class associated with S, and T is kT and should be interepreted as the energy scale of changes that affect S/k. Specifically, an energy change of kT corresponds to augmenting S/k by one, which also corresponds to "one binary decision" about opening or closing the door. So you immediately see the key issue here-- if each cannonball has a huge energy, then one decision about the door automatically has a big effect on the energy difference, but not a huge effect on the entropy decrease! One decision by the Demon is always 1 unit of entropy lost in the gas, via the T difference it creates, regardless of the energy per cannonball. At any T, one decision corresponds to kT of energy added to the higher-T gas, which always lowers the entropy by 1, even if T is huge.

So in other words, we do not magnify the entropy consequences of the Demon's decisions simply by magnifying the energy of the cannonballs-- that is not the connection between entropy and energy. If there is already a huge energy per cannonball, than the entropy is "less impressed" by big energy shifts involving opening and closing the door. Either way, one bit in the head of the Demon is one kT of energy that is getting added to the high-T, high-S gas and removed from the low-T, low-S gas, and that always results in a net decrease in S/k of 1. But of course it also results in a net increase in the S/k in the Demon, so there is no issue, and you need to ignore (or "erase") that entropy increase in the Demon to get the "paradox."

So the bottom line is, the "paradox" stems from not understanding the fact that when the energy scale is magnified, the entropy consequences are not-- because at higher energy, each "decision" corresponds to more energy, but at higher T, more energy is needed to have the same impact on the entropy. The bottom line is, in thermodynamics, the Demon does not have "intention" or "purpose" that motivates his actions-- he is just another "dumb gas" that will only do what he does if it results in a net increase in entropy in the total Demon+gas system. He does what he does because it is a more likely configuration of the whole system, that's "intention" in themodynamics. And there is no issue with extracting work from the gas-- if you have access to an entropy-generating mechanism (like a brain), you can use it to drop the entropy somewhere else, and extract work in a way that conserves energy overall. This just means that brains are even more important to have in situations where there is access to high temperature, because each decision by the brain has a relatively larger energy consequence, which is also why you need good brains with their "fingers on the button" if you see what I mean!

Last edited: Feb 25, 2012
13. Feb 25, 2012

Thanks Ken, you seem to have got this pointing in the right direction, but I have one last misconception.

I already mentioned the question of how fussy the demon is. Where in the probability distribution of temperature per molecule does he set his threshold for opening the door? Can we really say that the entropy decrease of the gas per decision is independent of the threshold, and/or can we say that the entropy value of the bits in his head are independent of the threshold? I'd be tempted to say nO and yES respectively, which would resuscitate the problem.

I still haven't really understood this energy-entropy connection. Let's just banish k by an appropriate choice of units, as Einstein banished c and QM people banish h and so on.

Are you saying that one bit is just one bit of negentropy = ln(2), irrespective of any temperature? I think so. I think you'd only care about T if you cared about energy, which you already said was irrelevant because I can't make a power station out of this anyway.

But the threshold does affect the number of phonons being moved over the divide in each decision. That's the point right? We're talking about the number of ways we can distribute a known number of phonons over a known (but inconstant) number of modes. If we can sort a lot of phonons in one go then we've made a big difference to the number of ways the whole system can be arranged. So the earlier nO seems supported.

14. Feb 25, 2012

### Rap

So the bottom line is that yes, Maxwell's demon can take a high-entropy situation (like two containers of gas at the same temperature) and turn it into a low-entropy situation (the two containers have different temperatures). The first law is not violated - the total energy of the gas in the two containers is the same before and after. The second law is not violated - the lowered entropy of the two gases is more than offset by the increased entropy of the demon.

The lowered entropy (different temperatures) of the two gases can now be used to run an engine for a little while, until the temperatures are equal again (but lower than before). Then the demon can lower the entropy again, getting different temperatures again, which can then run the engine again for a little while (not delivering as much energy as before, because of lower temperatures). This could go on and on, but you could never get that engine to produce more energy than the amount of kinetic energy in the original gases.

The thing about the cannonball gas is only that there is a lot of energy there to be converted (the cannonball-gas temperature is so huge and there's a mole of cannonballs).

Is this right?

15. Feb 25, 2012

Here's a completely different point: Bennet's take is that the Demon has to erase a bit when he forgets what he observed, which means setting a bit (that might have had any value) to a distinct value, which means lowering entropy and thus using energy. That's supposed to help. But now that Ken has unravelled energy and entropy it seems more like a hindrance.

The demon doesn't have to erase any bits.

He stores the bit by letting the molecule through!

Oops: wasn't there a change of sign somewhere? I think so. I say the demon picks up negentropy from the chamber he observes and dumps it in the one behind the door. He doesn't need much RAM and his entropy is steady. Simple. Bennet's take is considerably more convoluted and I think he sneaks in a change of sign by using the very law that's under attack - the 2nd.

Surely the demon was always maintaining a steady state - nobody ever asked him to store any internal state.

It's multiply circular to argue that the demon might be in danger of acquiring a long memory when he never needed to, then to say that he has to take precautions against that, and that the precautions would seem to reduce entropy (although the danger would have reduced entropy as well) and then to blithely assume that the second law dictates some energy requirement when the second law is the very thing under attack by the paradox we are trying to solve.

Whatever thermodynamic arguments are raised for claiming that the demon has to use energy, increase entropy or whatever, it should be possible to ground them in mechanistic, Newtonian style physics. Otherwise, there's a lot of danger of circularity.

16. Feb 25, 2012

### Ken G

Yes, that's how I see it as well.

17. Feb 25, 2012

### Ken G

I think your idea of looking at different thresholds is digging deeper into the details of the situation, but should not encounter any paradoxes. The Demon must increase entropy in the process of thinking and processing the information, and that allows the entropy in the gas to drop, and still have a net result that will actually occur spontaneously (the Demon has to function spontaneously here, it is ruled by thermodynamics also). But this doesn't mean that we know how much the net entropy will increase per decision-- that's what depends on the threshhold and all the other details of the decisionmaking process. Thermodynamics doesn't tell us how human thought occurs, for example, we don't understand why my writing this, instead of something else, represents the largest possible increase in entropy. But if one takes the thermo perspective, that is the reason I am writing this. So even though we don't understand why the Demon sets the threshhold where it does, we can calculate the decrease in entropy that occurs in the gas per decision. I'm not sure we really have a good model of what a "decision" actually is, but the main point is that once we get a model of that, the entropy consequences don't depend on the overall scale of the T of the gas (whether atom or cannonball), because what T effects is the conversion from the entropy scale to the energy scale, not the entropy scale itself (entropy scales like E/T, so cannonballs or atoms work the same for entropy). That's why decisions are more important when the energy scale is larger, but the entropy of a decision is more or less always the same (depending on how we are modeling what a "decision" actually is). Hence every decision you make has the potential to change your life, it all depends on how that decision interacts with other scales that don't relate to entropy.

18. Feb 25, 2012

19. Feb 25, 2012

### Ken G

I don't think whether or not the Demon has a good memory is all that important for the entropy, unless you invoke more complex rules about "decisions" that involve memory. What is mostly important about the Demon is that it be an entropy generating machine, preferably at as low an effective temperature as possible (so you don't have to feed it so much to get it to make decisions). We can have the Demon be in a steady state, but we have to account for some kind of environment (with a low effective temperature) that it can be increasing the entropy of as it "thinks." If that environment is the gas itself, then its temperature is too high to get the Demon to function, and we won't be able to get the gas to decrease its entropy if we keep track of everything happening. That I think is the source of the "paradox"-- we are ignoring the need for the Demon to have access to its own environment, and the environment has to be something different from the gas for this to work. The one thing you can "take to the bank" is that the action of the Demon's decisions increase the entropy in total-- they are never "more than offset" by anything the gas is doing. Increasing the T of the gas does not create such an offset, it only increases E and T but not E/T, i.e., not the entropy decrease in the gas per action by the Demon.

Last edited: Feb 25, 2012
20. Feb 25, 2012

### Rap

If the demon is moving low energy particles to one side of the partition between the two systems initially in equilibrium, then the demon is lowering the total entropy of the two systems. In order for the second law to hold, the combined entropy of the two systems and the demon must increase. What I meant was that the entropy decrease of the two systems is more than offset by the entropy increase of and by the demon. Now I am trying to figure out how the demon's entropy unavoidably increases. Part of it is that the measurement process increases entropy. In the case of the cannonball gas, shooting photons out to measure position and momenta of the cannonballs, I think. The other part is the modifications of the state of the demon as a result of these measurements. Thats the part that has me thinking right now.