Entropy: why measure disorder?

[Robert Webb]

Something I've always wondered: why do we measure the amount of disorder (entropy) rather than the amount of order?

We don't measure brightness by the amount of "dark". Surely order is the thing of interest, so why don't we measure that rather than measuring the absence of it?

And in conversation it always feels wrong! I end up just talking about the amount of order rather than the amount of entropy which is always counter-intuitive.

Thanks,
Rob.

 

[anorlunda]

I often wondered the same thing. It sounds illogical. But then I thought of this case.

Consider gas particles in a room. The more knowledge we have about their positions and velocities, the less the entropy. The more the entropy (disorder) the less our knowledge for the same number of particles. But now introduce one more particle with unknown position. The entropy increases a defined amount, but our knowledge is unchanged. That defined amount is useful in calculations.

 

[Vanadium 50]

You asked for an I-level answer, so I am going to give you one, even though I suspect you are looking for a B-level answer.

Surely order is the thing of interest

Why would you think that? Entropy is, up to a logarithm and a constant, the number of different microscopic states that give you the same macroscopic state. Why is that unimportant? And how would you even mathematically define "order"?

 

[Nugatory]

Something I've always wondered: why do we measure the amount of disorder (entropy) rather than the amount of order?

Entropy is defined to be $k\ln\Omega$. Without a similar definition of "order", we have nothing to measure.

 

[Jamison Lahman]

Something I've always wondered: why do we measure the amount of disorder (entropy) rather than the amount of order?

Here's a couple of questions for you: how do we measure entropy? How would you measure order?

 

[jtbell]

Entropy is defined to be $k\ln\Omega$.

For a simple example, see here:

http://hyperphysics.phy-astr.gsu.edu/hbase/Therm/entrop2.html

Note: Older books (and Boltzmann, the creator of that definition) used $W$ for multiplicity. Nowadays we usually use $\Omega$.

 

[Simon Phoenix]

Something I've always wondered: why do we measure the amount of disorder (entropy) rather than the amount of order?

That's a good question.

When Shannon was searching for a name for his new uncertainty function, von Neumann is reported to have said "You should call it entropy; for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage"

I think 'order' and 'disorder' are actually really quite difficult to define. Take the sequence of digits 35897932384 and 1234512345. We'd want to say that the second sequence is more 'ordered' because we recognize a pattern - yet the second sequence is just 10 consecutive digits of $\pi$ so there's some structure there too. Take the 4 Cartesian points (1,1), (1,-1), (-1,-1) and (-1,1) and plot them. We'd want to say that there's 'order' there because they're the vertices of a square - our eye is drawn to the pattern - but is there really more 'order' there than in the points (1,0), (17, -1/2), (-2,3) and (1/3, 7/5) ? I don't properly know how to answer that question. Is it just because we've evolved to be very good at seeing patterns and drawn to symmetries or is there really more 'order' in the first collection of points?

That's why I think the approach in terms of uncertainty is more satisfactory. If there are a large number of microstates that yield the same macrostate - then, given that macrostate, there is a lot of uncertainty about which particular microstate the system is actually in. The entropy is really then a measure of the number of available states subject to some constraints on the 'overall' properties (like energy, etc).

Consider flipping a fair coin 10 times - there are lots of ways of getting 5 heads and 5 tails - HHHHHTTTTT, HTHTHTHTHT, THTTTTHHHH, and so on - it would take a while to list them all. But there's only 1 way to get all heads. So in any 10 flips it's not likely we're going to get all heads - possible, but it occurs with a low probability compared to getting exactly 5 heads and 5 tails in 10 flips. So the entropy, subject to the constraint of getting 5 heads, is higher than the entropy subject to the constraint of getting 10 heads - there are many ways to get 5 heads from 10 flips, but only one way to get 10 heads from 10 flips.

Another way to say exactly the same thing would be to attempt to answer the question "given that I got $n$ heads in 10 flips, what was the actual sequence of heads and tails I got?" If $n = 10$ then we can answer that question precisely and there's no uncertainty at all. If, however, $n = 5$, then we can't specify the actual sequence - we can rule out a lot of them, but we're still left with a fair degree of uncertainty about which particular sequence was obtained. Entropy is a measure of that uncertainty.