Entropy: Thermodynamics & Information Theory Explained

[Jam Smith]

I was reading some articles related to entropy and I come to know that,
The term “Entropy” shows up both in thermodynamics and information theory.

Now my question is :
What’s the relationship between entropy in the information-theory sense and the thermodynamics sense?

I need some clear and short guidance.

 

[DrClaude]

Start with Wikipedia: https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory

 

[Jam Smith]

Start with Wikipedia: https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory

I can’t think of a best way to signify intuitively that entropy and information theory are essentially the same. Imperatively I found they both are hard to describe.

The amount of information it takes to describe something is proportional to its entropy. Once you have the equations (“I = log2(N)” and “E = k log(N)”) this is pretty obvious. However, the way the word “entropy” is used in common speech is a little misleading.

Right?

 

[ZapperZ]

owever, the way the word “entropy” is used in common speech is a little misleading.
Right?

The way the word [insert any physics terminology here] is used in common speech is a little misleading. This should not be surprising and it is very common.

However, you did ask this in a physics forum, and we seldom answer such questions using such misleading/mistaken concepts. What you will get is the standard physics answer. Isn't this what you were looking for?

Zz.

 

[Jam Smith]

The way the word [insert any physics terminology here] is used in common speech is a little misleading. This should not be surprising and it is very common.

However, you did ask this in a physics forum, and we seldom answer such questions using such misleading/mistaken concepts. What you will get is the standard physics answer. Isn't this what you were looking for?

Zz.

You misunderstood me. I was talking about,
In thermodynamics every state is as likely to come up as any other. In information theory,

Why in thermodynamics every state is as likely to come up as any other? Can this be proved?
I am still not clear.

 

[DrClaude]

Why in thermodynamics every state is as likely to come up as any other? Can this be proved?

It's a fundamental assumption of statistical physics. It can be proven, but it appears to be valid (no experimental evidence has shown otherwise).

We had a discussion about his in a recent thread: https://www.physicsforums.com/threads/boltzmann-with-degenerate-levels.902321/

 

[Jam Smith]

It's a fundamental assumption of statistical physics. It can be proven, but it appears to be valid (no experimental evidence has shown otherwise).

We had a discussion about his in a recent thread: https://www.phyhttps://www.physicsforums.com/threads/the-term-entropy.904205/sicsforums.com/threads/boltzmann-with-degenerate-levels.902321/

Thanks for providing such fundamental information. I have go through both links you provided and it helps me to clear my doubts.