Information Theory and Entropy

In summary: CPT seems to imply it. You can reverse the system evolution by applying charge, parity and time conjugation, so the information about the past must be contained in the present state. That implies conservation of information by the evolution.
  • #1
Arman777
Insights Author
Gold Member
2,168
193
1-Whats the relationship between entropy and İnformation ?
2-Can Entrophy always increases statement imply information lost ?
3-If it implies how its lost ?
 
Physics news on Phys.org
  • #2
1. Entropy and information are basically opposites. Entropy tells you how many quantum states are consistent with the information you have. If you have more information about the state, then there are fewer states that match the description, so the entropy is less.
2. Yes.
3. When two particles interact, they trade some energy and momentum. The amount of energy/momentum transfer is in superposition. Somehow, the wavefunction collapses, and the amount of transfer takes on some random value. So, if we initially knew the energy/momentum of each particle before the interaction, afterwards we only know the total. So some information was lost. It's not clear how this wavefunction collapse occurs.
 
Last edited by a moderator:
  • Like
Likes Arman777
  • #3
Khashishi said:
1. Entropy and information are basically opposites. Entropy tells you how many quantum states are consistent with the information you have. If you have more information about the state, then there are fewer states that match the description, so the entropy is less.
2. Yes.
3. When two particles interact, they trade some energy and momentum. The amount of energy/momentum transfer is in superposition. Somehow, the wavefunction collapses, and the amount of transfer takes on some random value. So, if we initially knew the energy/momentum of each particle before the interaction, afterwards we only know the total. So some information was lost. It's not clear how this wavefunction collapse occurs.

I understand thanks, "So some information was lost" , I remembered now that I heard İnformation never lost, never vanishes. Is this true ?
Like Conservation of Information ?
 
Last edited by a moderator:
  • #4
I searched a bit and says there's no such conservation but From unitarity or cpt there could be
"CPT seems to imply it. You can reverse the system evolution by applying charge, parity and time conjugation, so the information about the past must be contained in the present state. That implies conservation of information by the evolution.

This may not be the answer you wanted, because it does not imply unitarity, but it is the only relationship between symmetry and information conservation that I can think of. Unitarity seems to be a very fundamental assumption though, and there is not much more fundamental mathematical structure you could use to argue about its necessity." (A.O.Tell,Physics Stack Exchange,https://physics.stackexchange.com/q...associated-to-the-conservation-of-information)
 
  • #5
But, wavefunction collapse is not a unitary transformation.
 
  • #6
Khashishi said:
But, wavefunction collapse is not a unitary transformation.
I don't know much things about wavefunction actually I am freshman at physics :)
So it doesn't conserved.
Like energy,momentum etc is even not conserved in macroscopic states,like galaxy clusters scale etc.So It make sense to me that its not conserved.Maybe we can say the information about the system is lost during the time.It never disappeares but it just turns something that we can't describe the full system after a period of time using previous information
 
  • #8
Arman777 said:
I searched a bit and says there's no such conservation but From unitarity or cpt there could be
"CPT seems to imply it. You can reverse the system evolution by applying charge, parity and time conjugation, so the information about the past must be contained in the present state. That implies conservation of information by the evolution.
/QUOTE]

Sure, that's fine if it doesn't interact with anything.
 
  • #10
Makes sense...
 
  • #11
Well I read the article and says information is conserved.It can not be copied or lost.Which Complementarity came from that.
 

FAQ: Information Theory and Entropy

1. What is Information Theory and how is it relevant to science?

Information Theory is a branch of mathematics and computer science that deals with the quantification, storage, and communication of information. It is relevant to science because it provides a framework for understanding how information is processed and transmitted in various systems, including biological, social, and technological systems.

2. What is the relationship between Information Theory and Entropy?

Entropy is a measure of the uncertainty or randomness in a system. In Information Theory, entropy is used to quantify the amount of information contained in a message or signal. The higher the entropy, the more uncertain or random the information is. Therefore, Information Theory and Entropy are closely related and often used interchangeably.

3. How is Shannon's Entropy used to measure information?

Shannon's Entropy is a mathematical formula developed by Claude Shannon in 1948, which measures the amount of uncertainty or randomness in a message or signal. It is calculated by taking the negative sum of the probability of each possible outcome multiplied by the logarithm of that probability. This formula allows us to quantify the amount of information contained in a message or signal.

4. Can Information Theory be applied to fields other than communication and computer science?

Yes, Information Theory has been applied to a wide range of fields, including biology, neuroscience, economics, and linguistics. It can be used to study how information is processed and transmitted in various systems, including biological networks, neural networks, and social networks. It has also been used to analyze language and communication patterns in different cultures.

5. How has Information Theory impacted modern technology?

Information Theory has played a crucial role in the development of modern technology, particularly in the field of communication and information technology. It has led to the creation of more efficient coding and compression techniques, which have greatly improved data storage and transmission. It has also influenced the design of communication systems, such as wireless networks and the internet, making them more reliable and secure.

Similar threads

Replies
4
Views
1K
Replies
3
Views
2K
Replies
10
Views
1K
Back
Top