Question about information change as gas expands and contracts

[absolutenoob]

seeing the information is a form of entropy, what happens when gas confined to one corner of a cubic volume expands to fill the whole volume? Is information created? Would the reverse destroy information? I don't understand how a group of air molecules could change the amount of information they contain just by expanding/contracting. Does it have more to do with the system of the cubic volume? What information is there that could be transferred from the expanded volume of gas that could not be transferred from the confined gas? What is wrong with my thinking of this?

 

[mfb]

If it expands (perfectly) adiabatically, entropy stays the same.
If it does not, entropy can change.

I don't understand how a group of air molecules could change the amount of information they contain just by expanding/contracting.

Chaos has an important role here. If you suddenly open a shutter and let the gas expand, for example, you won't be able to reconstruct the initial state (whole gas in some smaller volume) based on the final state (gas fills some larger volume). Entropy is not information, even if the concepts are related. Entropy is a measurement how much information about the system you do not have - and that value can increase*.

*that is true for many systems - even for our everday life. Don't watch/read news and the amount of information you don't have will increase with time.

 

[TumblingDice]

seeing the information is a form of entropy, what happens when gas confined to one corner of a cubic volume expands to fill the whole volume? Is information created? Would the reverse destroy information?

The relationship between entropy and information is that higher entropy levels require more information to completely define all states in the system. So moving from lower to higher entropy 'creates' information in the sense that it represents more information.

Moving a system to a lower entropy would require less information to completely describe the system.

 

[absolutenoob]

Entropy is not information, even if the concepts are related. Entropy is a measurement how much information about the system you do not have - and that value can increase*.

I have read this in other places and the overwhelming response is that this is not true. Most sources i have read have stated that higher entropy= more information. I agree that your statement does make sense, but from what i have read it isn't true.

 

[absolutenoob]

I guess it does make more sense though that entropy is how much information you don't know about a system when you think of the probability that all the gas in a cubic volume is contained in a small corner of it. The probability of that state is low, so the information is high correct? Is it just that the majority of people misunderstand information theory? Arg this is so frustrating. I hope more experts weigh in

 

[mfb]

Most sources i have read have stated that higher entropy= more information.

More information you do not have, indeed.

The probability of that state is low, so the information is high correct?

The probability of all microscopic states is low (there are many of those states). The probability of the macroscopic description (all atoms in one corner) is small, corresponding to a small amount of microscopic states. It is easier to fully describe this system (if you know the macroscopic description), therefore it has a lower entropy.

 

[TumblingDice]

I guess it does make more sense though that entropy is how much information you don't know about a system.

It makes no difference whether you know the information or not. If you could measure every last state of a system so you knew everything about it, that wouldn't change the entropy.

You can look at this either way: Higher entropy system requires more information to describe it completely, or represents more information. Same thing...

[Edit: I spoke too soon. It seems physicists count knowing everything about a system as that it has no information to offer them. My opportunity to learn, too]

 

[craigi]

The problem we have here, is that information is a seemingly well defined term in everyday language. When you use the term for a physical system, or in a pure mathematical sense, it can take on a number of quite contradictory meanings. If entropy is the quantitiy that you're talking about, then it's best to use the word entropy, to avoid ambiguity.

Both answers that you have been given are correct. Entropy is a measure of uncertainty, ie. information that you do not have. Also, to fully describe a higher entropy system requires a higher minimal amount of information.

 

[absolutenoob]

More information you do not have, indeed.

The probability of all microscopic states is low (there are many of those states). The probability of the macroscopic description (all atoms in one corner) is small, corresponding to a small amount of microscopic states. It is easier to fully describe this system (if you know the macroscopic description), therefore it has a lower entropy.

Could you give me an example of what a microstate would be in a volume of gas? Also your first sentence, does that mean when you have the low entropy case, you KNOW more information than you do in the high entropy case?

 

[craigi]

Could you give me an example of what a microstate would be in a volume of gas? Also your first sentence, does that mean when you have the low entropy case, you KNOW more information than you do in the high entropy case?

A microstate describes the positions of the molecules in the gas.


Try Susskind's lectures on Statistical Mechanics:

 

[TumblingDice]

Important to recognize that my posts #3 and #7 were misleading. Mfb explained well and craigi advised correctly not to mix terminologies. As I noted in my edit on post #7. I think the reason that knowing all of the microstates (e.g., information) reduces to zero entropy is because that could only be done by influencing the system whether it be measuring or reordering, and that means entropy will be raised elsewhere, like in the instruments used to gain the knowledge.

Sorry to interrupt the good info from mfb and craigi. :(

 

[craigi]

Important to recognize that my posts #3 and #7 were misleading. Mfb explained well and craigi advised correctly not to mix terminologies. As I noted in my edit on post #7. I think the reason that knowing all of the microstates (e.g., information) reduces to zero entropy is because that could only be done by influencing the system whether it be measuring or reordering, and that means entropy will be raised elsewhere, like in the instruments used to gain the knowledge.

Sorry to interrupt the good info from mfb and craigi. :(

It's ok to talk of the entropy of a subsystem and we can non-destructively obtain knowledge of a subsystem ie. without changing it's entropy. The issue of increasing entropy elsewhere, applies to acting on an open subsystem within a closed system and is pertinent the second law of thermodynamics.

 

[absolutenoob]

So let me get this straight- gas confined to one corner= higher information you know, lower total information. Gas expanded= lower information you know, higher total information. Is this correct? Or does the gas confined to one corner mean you know a greater proportion of the total information there is?

 

[TumblingDice]

I'm thinking:
Gas confined to one corner means less uncertainty (position microstate) than in larger volume. It is lower total unknown information, and lower entropy.
Expanded gas means more uncertainty - more total unknown information and higher entropy.

 

[TumblingDice]

So let me get this straight- gas confined to one corner= higher information you know, lower total information. Gas expanded= lower information you know, higher total information. Is this correct?

Those sound to be true statements. I wouldn't say that quantity of info you know or don't know is correlated to quantity of total information.

 

[kith]

Information entropy can be viewed as the number of yes-or-no questions you need to get from a state of limited knowledge to a state of maximum knowledge. In thermodynamics, we have states of limited knowledge because we know only macro variables like temperature, volume or the chemical potential. We don't know the true microscopic state of the system but in thermodynamic equilibrium, we know the probability distribution of the possible microstates. Their number is constrained by the values of macro variables.

In physics, a yes-or-no question corresponds to a measurement which has only two outcomes. So entropy is a measure of how many such measurements you would have to perform in principle, to determine the true microscopic state of the system. In equilibrium, a gas in a small volume has a lower entropy than a gas in a big volume. You know more about the gas in the small volume, because you have to ask less questions to get to its true microstate.

If entropy is high, the information content of the system is high in the sense that you would have to perform lots of measurements to determine the microstate. Up to a factor, it is the maximum number of bits you can "learn" about the system by performing measurements.