Maxwell's Demon revisited

AdrianMay

Me too, but what about:

Ken G

I don't think whether or not the Demon has a good memory is all that important for the entropy, unless you invoke more complex rules about "decisions" that involve memory. What is mostly important about the Demon is that it be an entropy generating machine, preferably at as low an effective temperature as possible (so you don't have to feed it so much to get it to make decisions). We can have the Demon be in a steady state, but we have to account for some kind of environment (with a low effective temperature) that it can be increasing the entropy of as it "thinks." If that environment is the gas itself, then its temperature is too high to get the Demon to function, and we won't be able to get the gas to decrease its entropy if we keep track of everything happening. That I think is the source of the "paradox"-- we are ignoring the need for the Demon to have access to its own environment, and the environment has to be something different from the gas for this to work. The one thing you can "take to the bank" is that the action of the Demon's decisions increase the entropy in total-- they are never "more than offset" by anything the gas is doing. Increasing the T of the gas does not create such an offset, it only increases E and T but not E/T, i.e., not the entropy decrease in the gas per action by the Demon.

Rap

If the demon is moving low energy particles to one side of the partition between the two systems initially in equilibrium, then the demon is lowering the total entropy of the two systems. In order for the second law to hold, the combined entropy of the two systems and the demon must increase. What I meant was that the entropy decrease of the two systems is more than offset by the entropy increase of and by the demon. Now I am trying to figure out how the demon's entropy unavoidably increases. Part of it is that the measurement process increases entropy. In the case of the cannonball gas, shooting photons out to measure position and momenta of the cannonballs, I think. The other part is the modifications of the state of the demon as a result of these measurements. Thats the part that has me thinking right now.

Ken G

Yes, the Demon must increase entropy when it decides to open or close the door. Exactly how that happens I'm not that clear on, my comments were around trying to remove the idea that the energy of the particles mattered. I think the basic idea can be understood if we imagine a situation where you have a bunch of coins on a floor that can shake. First you shake the coins and get, let's say, 50 heads and 50 tails. You register which ones are which, and glue down all the heads, and shake the floor again. It's clear that eventually you will have all heads, so the entropy of the coins is decreasing. That has to be made up by the entropy associated with noticing which ones are heads and which are tails. I think that "noticing" part must mean that a bit in the brain in effect substitutes for the bit normally associated with the coin, such that the coin can be glued down, "removing" that bit from the entropy, without decreasing the total entropy because that bit is now in the head that decided to glue the coin down. In a sense a copy of the information has been made, in order to make the decision, so regardless of what decision is made, that copy retains any lost entropy.

AdrianMay

I think we're all in about the same place now. We agree that:

* The first law was never in danger, even if the second law was, because we can just look at all the balls on the level of kinetic energy.
* We only use temperature in dU=TdS, but if we're not attacking the 1st law we don't even need that equation, it's all about the entropy alone.
* Cannonballs and molecules are about the same thing because of the above
* We'll try to fix the paradox in the elastic case first. (I'd suggest considering inelastic complications if that fails.)
* The usual strategy for rescuing the 2nd law is to say that the demon's entropy increases, but none of us seems to know how or why.

Do you both agree that we all agree with the above?

I think that if we say that the demon is in a steady state, then by definition we'd have said that his entropy is constant and ruled out the standard rescue, so we'd better be careful about calling him a steady state, but frankly I don't see why I can't say that.

My fussiness argument from before didn't make sense though. Even if you only let the top percentile through (as opposed to the top 50% as usually imagined) then you are still making one decision per molecule. So on average, each decision sorts as many phonons as there are phonons on the average molecule, which is given by the temperature.

I think another possible strategy would be to show that he succeeds in decreasing the entropy of the whole system including himself, but only if he receives energy from outside with which to do work. That would be legal because the entropy increase occurs wherever he got the energy from. Then the temperature would be relevant because the work he has to do is TdS if he's planning on reducing entropy by dS. It would not be fair to put the cart before the horse and say that he must do work because the 2nd law says so, which I think is what Charles Bennet's argument sneaks into the soup somewhere around the bit erasure stage. I think the real purpose of Bennet's argument is to show that erasing a bit costs kT.ln(2) of energy assuming the 2nd law applies to Maxwell's demon. I don't think it was intended to rescue the 2nd law by throwing in a new axiom about erasure.

I emailed him BTW, but that might have been inappropriate. How would something like this typically escalate in a professional way? Letter to a journal or something?

Rap

I have no idea where I am going with this. I have been reading http://arxiv.org/pdf/physics/0210005v2 on Landauer's principle.

If a demon has 10 unglued coins, shakes the table, then before the demon looks at the coins, he describes the situation using entropy - 10 bits of entropy or 10 bits of missing information. Once he looks at them, he says zero entropy, and 10 bits of information in the demon's head. No uncertainty, no missing info, no entropy. The demon glues down all the heads (say 6 of them), shakes the table again. Before he looks, 10-6=4 bits of entropy, after he looks, none. If he remembers the original configuration, he now has 14 bits in his head. If he forgets the original configuration, he has 4 bits in his head, but by Landauer's principle, "forgetting" creates entropy, so those forgotten 10 bits are now demon entropy in the form of heat.

Suppose of those four, 2 were heads, 2 tails, so its now 8 heads, 2 tails. If he forgets the original configuration, its just 10 bits in his head. This goes on and on until there are 10 heads, zero coin entropy as far as the demon is concerned. If he is forgetful, he will have 10 bits in his head, all heads. If not, he will have something like 10+5+2.5+...=20 bits on average. So if he forgets, there will be 10 bits in his head and 10 bits of forgotten (heat) entropy.

To a scientist outside, who just measures the number of heads and the number of tails, not which are heads and which are tails (analogous to thermodynamic measurements, not knowing everything), he sees 6h,4t and then 8h,2t, etc. and he says coin entropy is decreasing. He is treating the coins as indistinguishable, while the demon is treating them as distinguishable, so their idea of entropy is different. To the scientist, 6h,4t has an entropy of H(6h,4t)=log2(B(10,6)), where B(a,b) is the binomial coefficient a!/b!/(a-b)!. That's about H(6h,4t)=7.7 bits. The scientist, looking at 8h,2t, says thats an entropy of H(8h,2t)=5.5 bits. He says coin entropy is decreasing.

As the scientist watches things, the entropy goes to zero - H(10h,0t)=0. He wants to know what happens to (his idea of) the entropy of the demon during this process.

When there are 6h,4t, the scientist will say there are 7.7 bits of coin entropy, and 7.7 bits of entropy the demon forgot and turned into heat for a total of 15.4 bits of entropy. When it goes to 8h,2t, the scientist will say there are 5.5 bits of coin entropy, and the demon forgot a total of 7.7+H(2,2)=10.3 bits of entropy for a total entropy of 15.7. Total entropy is increasing. This is kind of artificial, 6h,4t and 2h,2t are close to what you would expect - 50% heads, 50% tails. If the ratios were much different, I guess you could get a decrease in entropy, but for large numbers, no. On average, after two trials, you should get 10+5=15 bits of entropy, or something.

When there are finally 10 heads, the scientist will say there is zero coin entropy. If the demon is forgetful, the scientist will know he has 10 bits in his head, all heads, so zero demon entropy and something like 10+5+2.5+...=20 bits of forgotten entropy. It looks like the second law is saved.

Maybe that distinguishable/indistinguishable thing was a needless complication.

LOL - this reminds me of the discussion we had on Schroedinger's cat.

AdrianMay

In the absence of a challenge to the 1st law, the demon is gonna have to do without his air conditioner and reach, say, the average temperature of the two chambers.

But I'd dispute that this temperature has anything at all to do with his trapdoor. I reckon the position of the trapdoor is the only entropy or internal state that he needs. Most of the time there are no molecules around and he just sleeps. When a molecule comes along, he takes a decision and stores it by opening or closing the door. No other RAM or state is involved. When the molecule is through, he just leaves the door where it was and goes back to sleep. When the next molecule arrives, he sets the door accordingly, thus erasing the memory of the previous decision. So every time he adjusts the door, which is once per molecule, he goes from any of 2 possible states to a precise state, which reduces the entropy of the demon-door by ln(2). Not (I claim) T.ln(2). (Banishing k by appropriate choice of units.)

That would be generous, but I might also claim that flipping the trapdoor goes from one known state to one known state, both of which had entropy of ln(1)=0.

I think this is basic to the definition of entropy. All that stuff about T came in when hot gases were considered. The reason you need a factor of T there is that a molecule is not the interesting unit here, rather, the phonon is, and a molecule is basically a bus load of phonons. T measures how many phonons are on the bus (kind of.)

No such argument can be applied to the trapdoor. Maybe the trapdoor consists of a big warm protein molecule with a kink in the middle that easily flips this way or that, but hates to be in the middle. The whole molecule is stuffed with phonons reflecting the ambient temperature, but they're all on a leg that's flipped left or right. Either way, there are just as many available states, and the entropy is the same. For a different entropy value, you'd have to consider some quantum superposition of left and right, or a population of such molecules, such that you could somehow speak of a left-and-right or left-or-right state. That would be more disordered. But we never needed to propose a moment in time when we didn't know whether the trapdoor was open or closed, and I think we can flip back and forth without any entropic consequences. Open and closed are equally disordered, irrespective of temperature.

I hope somebody can shoot that argument down, because my universe is turning distinctly pear shaped and I'm losing sleep.

Rap

If that is the case, then the entropy acquired by the demon will yield an energy of the order of the energy removed from one side and deposited in the other. I think this might cause the engine not to work.

But hey, if the demon knows which molecule will hit where, why not just sidestep a molecule coming at him? If we look at a human demon and a cannonball gas, the human can just step out of the way when he sees a cannonball coming at him at 1 meter/sec. If you have a problem with the energy he will expend, then go to a gas of aircraft carriers.

AdrianMay

The paper Rap found confirms it: you can't use information theory to rescue the 2nd law from the demon, rather, that stuff assumes that even the demon obeys the 2nd law.

That would seem to mean that the demon is still as scary as when Maxwell invented him. Perhaps we could actually make one.

Rap

I interpreted it to mean that you can make one, and the second law will not be violated.

AdrianMay

Not me. I think he's saying that if you look at it Maxwell's way and consider the 2nd law disproved, then none of that information theory stuff even exists. They do the information theory by assuming that the demon is sociable enough not to break the 2nd law, then they figure out exactly what he can and can't do within those rules. But there's still no argument to say he will follow the 2nd law unless you just faithfully believe that.

People then make the mistake of taking the results of information theory as proven no matter what, and using them to protect the 2nd law from the demon. But that would be circular. Bennet himself is acknowledging this circularity in the paper you found.

Ken G

I believe the assumption that the Demon obeys the second law is a natural one to make, in lieu of a good theory of how "thought" works. According to thermodynamics, thought, like any other process of a large system, is "spontaneous", which simply means it is ruled by the principle that more likely things happen and less likely things don't happen. That's the only assumption behind the second law. So I think any difficulty we have with the Demon is simply traced to two problems:
1) we don't have a good model for what a decision is, thermodynamically, though we do see some connection between information entropy and the ability to answer questions that cull out possibilities (so the entropy of the gas can be seen as related to the number of questions needed to cull out its actual state from the set of states that physics deals with when it talks about the behavior of a gas), and the mind that is doing the culling. For the culling to be done physically, like separating hot and cold components of gas, the action of the mind must fill in for, or make a "copy" of, the entropy that is being lost from the gas.
2) We aren't accounting for the environment in which the Demon functions. This is what leads to ideas like the Demon being in a "steady state" if he can forget the decisions he makes, but I don't think it matters what the Demon remembers, what matters is that a brain is also a system, and must function in an environment. That environment must gain entropy for the brain to function spontaneously-- you can't have the brain "getting lucky" when it follows a rule, you must program it to follow that rule, and that program must play out as the spontaneous response of some entropy-generating environment. That's what makes the program reliable in the first place.

Rap

I have to disagree with this. The information theoretic idea of the demon is not that he is "sociable enough" not to break the second law, it is that by acquiring and possibly forgetting the information needed to decide when to open the door, he is unavoidably acquiring enough entropy to offset the decrease in entropy he produces by selectively passing fast particles from one side of the partition to the other.

We have to make clear two scenarios by which the second law is saved:

1) Maxwell's demon succeeds in reducing the total entropy of the two gases by selectively moving fast particles into one chamber. The second law is saved by the increase in entropy of the demon.

2) Maxwell's demon fails to reduce the total entropy of the two gases. The second law is saved, but the entropy balance sheet is unclear to me.

I have read Earman and Norton, and the Bennett paper, but not with anything near complete understanding. As I presently understand it, Earman and Norton reject the information entropy defense of the second law, while Bennett supports it. Earman and Norton present two principles - Szilard's and Landauer's, as information-theoretic explanations of how the second law is not violated, and disputes both.

Szilard says there is an entropy cost in acquiring information. Bennett disputes this. Me, I don't know. Looking at the cannonball gas, the demon needs photons to measure the position and momenta of the cannonballs, and I haven't thought about the entropy considerations here. Earman and Norton point out that if you really have equilbirium in this cannonball gas, there will be Planckian radiation at the huge temperature of the cannonball gas, which is why you need high energy photons to measure their position. Now I understand where the idea that high energy photons are needed came from. This is a good point, the cannonball gas without photons is not in equilibrium, and Maxwell's demon is not operating on equlibrated gases.

Landauer says there is an entropy cost in erasing information. Bennett agrees, and says this saves the second law in the case of a forgetful Maxwell's demon. I accepted this in my analysis of KenG's "coin entropy" scenario. I tend to think this is true. If you forget information, then there is missing information, which is information entropy, which is thermodynamic entropy.

I'm not sure what "results of information theory" you are referring to here.

Ken G

I actually don't see that at as a good point, and I'm not sure those source aren't getting hung up on fairly irrelevant issues. The concept of equilibrium must be an effective, not literal, concept in physics-- we must be able to talk about equilibrium of what we care about, not necessarily equilibrium of everything. The second law is useless if we interpret it as a law that only applies in strict equilibrium, because no such situation exists anywhere. The patent office would have to reopen all the cases of perpetual motion machines that they refused to consider on the grounds that they violated the second law, if the inventor says "but my invention works in the corner of a room, where there is no strict thermodynamic equilibrium". The second law does not say that "entropy increases in strict thermodynamic equilibrium", because in strict equilibrium nothing changes, including entropy. Instead, the second law is about characterizing small deviations from equilibrium, and only in regard to whatever is the critical mode of behavior that is under study. For example, the universe is bathed in a very low-temperature neutrino bath, and no gas at normal temperature is in equilibrium with that. Saying that non-equilibrated photons in the cannonball gas invalidates the second law would be like saying that neutrinos always invalidate it. But actually this is only true in situations where the thermal neutrinos matter in some way, and it would only matter for the Demon and the cannonballs if thermal photons were somehow being used in the apparent entropy violation.

I don't know how those sources are arguing their points, but the summaries sound wholly unconvincing to me. I see the situation as much simpler than worrying about whether brains remember or forget, or whether they are or are not bathed in hot photons or hot neutrinos. It is simply that if you want to make a decision regarding some information, you have to be able to process that information, and information processing requires an environment. That environment will necessarily increase its entropy to at least "copy" the information being processed, and even then only if it is perfectly efficient. So the Demon will always generate more entropy than it destroys, and it will never matter whether the Demon is being bathed in photons or if it has a memory. Saying more probably requires having some specific model for how the Demon manages to "think", but much of the purpose of thermodynamics is being able to say things independently of the details of that kind of model, just as we don't have to talk about the mass of the cannonballs or how much they compress before they elastically rebound.
Yes, I agree that one cannot get rid of entropy by forgetting. But what this says is that remembering or forgetting is irrelevant. One cannot argue the Demon is cyclic or in steady-state by letting it forget, because the entropy increase is in the Demon's environment, not its memory. One can treat the memory as part of the environment, but if forgetting occurs, then it's not the whole environment. All violations of the second law invariably boil down to not considering the whole environment, and I'd say that's just what is going on here.

Rap

Agreed - I didn't mean to imply that lack of equilibrium with the photons was deal-killing, only that it was something that needed to be thought about, and hopefully rejected. I tend to agree with you - it can be rejected.

You are assuming that information entropy and thermodynamic entropy are equivalent via Boltzmann's constant, which I agree with, but apparently others do not.

Yes.

But let me ask, just to be very clear - Do you think that a demon could successfully reduce the entropy of the two systems, with the idea that the second law is not violated due to the increase in entropy of the demon?

Ken G

Yes, but to me that's the point of the Demon-- to show that they must be equivalent. Otherwise, we can substitute one for the other, and it's bye-bye second law. Perhaps those who don't see the equivalence don't think the second law is correct, but I'm not buying those wares. I hold that if one has a model for how the information is actually being processed, including the environment that is doing it, then the equivalence between information and thermodynamic entropy becomes clear-- it all boils down to the need for spontaneous results to be those which have a higher probability of occuring, and the sole reason they have a higher probability is that they represent larger entropy.Yes, the Demon can separate hot and cold gases, and it can do it by increasing the entropy even more in whatever environment the Demon uses for a brain, but the energy scale of the Demon can be vastly less than the energy scale of the gas. The latter would just mean that the Demon has a lower effective temperature. Hence the Demon cannot be born from the thermodynamic equilibrium it is disrupting, it needs some separate existence that must also be analyzed-- not treated like some magical process of intention, and not just some program and memory, but a whole thermodynamic environment of its own. But the Demon can be used to extract heat from a gas-- indeed there are many ways to do that, which violate neither the first nor second laws.

AdrianMay

OK let me quote from that paper.

First the top line: Bennet, 2011.

Then we have:

One of the main objections to Landauer’s principle,
and in my opinion the one of greatest merit, is that raised
by Earman and Norton [3], who argue that since it is not
independent of the Second Law, it is either unnecessary
or insufficient as an exorcism of Maxwell’s demon. I will
discuss this objection further in the third section.

So then we excitedly skip to the third section:

Earman and Norton have pointed out with some jus-
tice that Landauer’s principle appears both unnecessary
and insufficient as an exorcism Maxwell’s demon, be-
cause if the Demon is a thermodynamic system already
governed by the Second Law, no further supposition
about information and entropy is needed to save the
Second Law. On the other hand, if the Demon is not
assumed to obey the Second Law, no supposition about
the entropy cost of information processing can save the
Second Law from the Demon.
I would nevertheless argue that Landauer’s princi-
ple serves an important pedagogic purpose ....

That means he agrees. Pedagogic purposes are not logic.

You guys know me! Would I ever presume to question what a giant like Bennet says ???