What is Entropy? Understanding the Measurement

[imjustcurious]

Entropy has been described to me many different ways. Overall, I understand it as the measurement of randomness or disorder in a system. This doesn't make sense to me. It seems that it is a term made for humans. For example, if we knew all the information about every particle in the universe, wouldn't the universe be deterministic? We can't do that because we are humans. If you look at it simply, every particle has information and it isn't random. The only reason it seems random is due to the fact that we can't observe it all at once.

I'm sure this is a stupid question and I don't have something right. It just isn't sitting well in my head.

 

[Fooality]

Entropy is a probabilistic measurement the reflects our uncertainty about a system. I'm a computer guy way more familiar with information entropy, but I understand physics entropy is essentially the same. Suppose you have a box, divided into two sides. If all the particles are on one side of the box, the entropy is minimal, but as they spread out, the entropy increases. Now if we just know there's some particle we'll call Joe but don't know which it is, we have total information on which side of the box its on if the entropy is minimal, but way near 50/50 uncertainty as they all go out and rattle around and entropy increases. The reason the idea of entropy is powerful is that we don't HAVE to have total information on where Joe is to know something about Joe. Even not knowing which particle Joe is, we still have total certainty Joe is on the left side of the box if all the particles are, and 80% certainty he's on the right if 80% of the particles are. As the entropy gets maximum, we have max uncertainty about the location of Joe, its basically 50/50.

The beauty and power of the idea of it is that it let's us know something useful when there are too many details to actually know what's going on at the small level.

 

[jtbell]

http://en.wikipedia.org/wiki/Boltzmann's_entropy_formula

 

[imjustcurious]

Thank you for the explanation, it is a bit more clear; however, the main question that is boggling my mind is still unanswered. You said...

"The beauty and power of the idea of it is that it let's us know something useful when there are too many details to actually know what's going on at the small level."

"-it let's US know"
Us as in humans because we can't understand all the particles interactions at the same time. So it is a term coined by humans for practical use. But realistically there isn't entropy because it is some how plausible to know all the information and make the system deterministic. As humans we can't though.
Isn't this true?

 

[Fooality]

http://en.wikipedia.org/wiki/Boltzmann's_entropy_formula

Thanks, this is what I'm more familiar with:
http://en.wikipedia.org/wiki/Entropy_(information_theory)#Definition

 

[Dale]

So it is a term coined by humans for practical use.

All physics terms are terms coined by humans. Some are more practical than others.

The rest of what you said isn't true anyway. Classically, you could consider entropy to be related to ignorance of many of the fine degrees of freedom, with some underlying deterministic "truth". Unfortunately, at the level of detail that you would require, the world simply isn't classical.

 

[Fooality]

Thank you for the explanation, it is a bit more clear; however, the main question that is boggling my mind is still unanswered. You said...

"The beauty and power of the idea of it is that it let's us know something useful when there are too many details to actually know what's going on at the small level."

"-it let's US know"
Us as in humans because we can't understand all the particles interactions at the same time. So it is a term coined by humans for practical use. But realistically there isn't entropy because it is some how plausible to know all the information and make the system deterministic. As humans we can't though.
Isn't this true?

Yeah, this puts me into physics, which I'm starting learning, but its not my forte. What I do remember reading though was that Niels Bohr said that Quantum Mechanics makes sense if we accept the idea that probability is fundamentally real. If the basic building blocks of our universe require probability to be real, than it is real, and entropy is physically real too. But I don't see why it couldn't be used to give descriptions of systems where details could be known but aren't.

 

[imjustcurious]

All physics terms are terms coined by humans. Some are more practical than others.

Yes, this is true. So in the universe there isn't entropy. Nothing is random and it is just a idea to account for OUR uncertainty of a system. So if there isn't entropy, then the universe is deterministic. The laws of physics actually exist and govern our universe. Entropy is just used to say that for this volume we are uncertain this much, but realistically there is no uncertainty?
(Sorry if I'm just not understanding something that seems to be obvious)

 

[Fooality]

Yes, this is true. So in the universe there isn't entropy. Nothing is random and it is just a idea to account for OUR uncertainty of a system. So if there isn't entropy, then the universe is deterministic. The laws of physics actually exist and govern our universe. Entropy is just used to say that for this volume we are uncertain this much, but realistically there is no uncertainty?
(Sorry if I'm just not understanding something that seems to be obvious)

The question you have to ask is deterministic for who? Not us. For an all-knowing being? We we're talking about something related in the computer science forum, my post here mentions a situation we can create where a computer program always comes to the right answer in finite time, but never can KNOW that it came to the right answer, or a logical paradox results that brings down everything. That same paradox would exist for an all knowing observer of the system, but not for an observer with limited knowledge. So there's problems with the idea of an all knowing observer.

 

[atyy]

Yes, this is true. So in the universe there isn't entropy. Nothing is random and it is just a idea to account for OUR uncertainty of a system. So if there isn't entropy, then the universe is deterministic. The laws of physics actually exist and govern our universe. Entropy is just used to say that for this volume we are uncertain this much, but realistically there is no uncertainty?
(Sorry if I'm just not understanding something that seems to be obvious)

We don't know if the universe is deterministic or random. However, this view that the universe is deterministic and that the entropy is due to our ignorance is an acceptable one.

http://necsi.edu/projects/baranger/cce.pdf
Chaos, Complexity, and Entropy
A physics talk for non-physicists

 

[Dale]

So in the universe there isn't entropy. Nothing is random

I didn't say any of that. I was just pointing out that your objection is entirely uninformative since it can be applied to all physics terms.

 

[imjustcurious]

The question you have to ask is deterministic for who? Not us. For an all-knowing being?

I'm saying isn' t it possible that the universe, for any being, is deterministic. But what can be determined from the information depends on the being observing it. For example, we could know that the universe is deterministic, but we just can't determine what will happen because we aren't able to.

 

[imjustcurious]

I didn't say any of that. I was just pointing out that your objection is entirely uninformative since it can be applied to all physics terms.

Sorry, I phrased it in a bad way which made it sound like you said that. I was just stating what I thought. That since entropy is due to our "ignorance", there is no randomness in the universe and that it is deterministic.

 

[imjustcurious]

We don't know if the universe is deterministic or random. However, this view that the universe is deterministic and that the entropy is due to our ignorance is an acceptable one.

Do you know what might be a different view? How might entropy not be do to our ignorance?

 

[Dale]

That since entropy is due to our "ignorance", there is no randomness in the universe and that it is deterministic.

That we are ignorant is factual. That there is underlying determinism is pure conjecture which does not follow from the fact of our ignorance nor does it follow from the things that we do know

 

[atyy]

Do you know what might be a different view? How might entropy not be do to our ignorance?

I agree with what DaleSpam said above. We don't know whether the universe is deterministic or not (how could we know the "ultimate laws", whatever that may mean?). However, within the framework of classical physics, the laws are certainly deterministic, and there one understands entropy as springing from our ignorance. The quantum case is trickier, and in fact still being researched, but I believe that essentially the same idea holds (eg. http://arxiv.org/abs/1007.3957).

So I don't know a different view. I'm just saying that I can't rule out that someone will come up with a different view that makes sense.

 

[Fooality]

I'm saying isn' t it possible that the universe, for any being, is deterministic. But what can be determined from the information depends on the being observing it. For example, we could know that the universe is deterministic, but we just can't determine what will happen because we aren't able to.

(Going off the Wikipedia definition: Determinism is the philosophical position that for every event there exist conditions that could cause no other event)
The tricky part is certainty about the conditions. For us humans, its limited by our observation. There will always be situations - most in fact - where our observations are limited, and therefore our observations are inadequate for predicting the outcome of events in a deterministic universe. What I'm racking my brain over is if we could ever know that the universe was deterministic even if it was, or if any observer could ever experience the universe as deterministic, always knowing outcomes from observations. The latter I doubt, as I said. I can't see how we'll ever experience a deterministic universe even if we did live in one, or honestly, even know that we were living in one. The biggest problem is I can't see how either idea is falsifiable. If someone claimed the universe was deterministic and you walked in with a by-definition black box of non-determinism, they could always claim ignorance as to the deterministic processes that guide it, that must be there. Or alternatively if someone claims a non-deterministic universe, you could never show there isn't some situation which is beyond your capability to predict outcomes which might be non-deterministic.

 

[imjustcurious]

Thanks, this helped a lot. I was very confused.