Question about Entropy/Shannon Information

[celal777]

Hello,

I would like second (and more) opinions on comments i received from a physicist in response to my observations on a certain experimental setup designed to make a particular point about information theory in the context of entropy.

The experimental narrative is found on pages 9 & 10 of the following document :
http://www.colorado.edu/philosophy/vstenger/Found/04Message.pdf

A diagram of what is discussed in given at : http://www.colorado.edu/philosophy/vstenger/Found/042Barmag.pdf

What follows is the discussion that ensued between "me" and "my respondent":

ME :
1) You started out with a more highly ordered state (magnets in contact N-S & S-N) .
2) You opened the window
3) Magnets scattered - so magnets in a more disordered state
i.e. (entropy is increased =information is decreased)
4) Result : yet you say "information in the system has increased by
one bit"(p.10) i.e. (entropy is decreased= information is increased)

How is this possible ? Why are the conclusions in #3 & #4
different ?

The bits at the end of the experiment tell me there is more
information yet the system is also more disordered hence there is more
entropy which by definition means there is less information. The
contradiction in #3 & #4 below remains, doesn't it ? What am i missing
here ?

MY RESPONDENT:
You are using an intuitive definition of entropy rather than a
quantitative one, assuming the system is less orderly. Since entropy is
negative information, the entropy has decreased and so the system is
more orderly.
==========

My comments and questions for the Physics Help Forum :
-------------------------------------------------------------------------
Why is my respondant calling my interpretation "intuitive" ? Does he mean "wrong" ?
Is the system's final state really "more orderly" as he says ?
Is it possible that his experiment is really not demonstrating what he thinks it demonstrates either in terms of entropy or information theory ?

Many Thanks in advance for all comments received,

Celal Berker
London, England

 

[ramollari]

Celal,

I think your respondent is wrong when saying that the entropy has decreased. It has increased because it's the result of random action.

But I doubt that the information has increased with the scattering. Wouldn't there be some way to retrieve the orientation of both magnets when they are on top of each other?

And in general, it is true that entropy in general does decrease information. It may increase the quantity of data, but not necessarily information. Noise on the transmission cables, for example, usually will corrupt the bits. They may still remain as data, but the correct information has decreased as Shannon's law predicts. In your experiment, suppose the magnets fall on the opposite orientation. Isn't that a corruption of data, and hence a decrease in information?

Regards

 

[celal777]

Hello Ramollari,

I was thinking the same way as you but take a look at this :

http://groups.yahoo.com/group/undernetphysics/message/1384

Celal

 

[ramollari]

As I already said, I don't know if 4 bits of information result in the final system. They are random and don't make any sense. The original bits are lost by the event, so there is also a resulting loss from 2 bits to 0 bits of useful infromation.
I'm not an expert in Information Theory and it's formulas, but if they relate entropy with any pattern of bits, I begin to doubt them.