Information Theory - Shannon's Self-Information units

AI Thread Summary
Shannon's self-information units are defined in terms of logarithmic functions, specifically using -log_2(P(A)) to express information content in "bits." While "bits" are the standard unit for binary information, alternative units like "nats" can be used for natural logarithms. The discussion raises the question of whether the designation of "bits" as the unit of measurement can be rigorously proven or if it should simply be accepted as a definition supported by coding examples. The flexibility in using different logarithmic bases allows for various interpretations of information measurement. Ultimately, the conversation emphasizes the foundational role of logarithmic functions in defining information units.
Mihel
Messages
1
Reaction score
0
Information Theory - Shannon's "Self-Information" units

Hi,

I'm familiar with information and coding theory, and do know that the units of Shannon information content (-log_2(P(A))) are "bits". Where "bit" is a "binary digit", or a "storage device that has two stable states".

But, can someone rigorously prove that the units are actually "bits"? Or we should only accept it as a definition and then justify it with coding examples.

Thanks!
 
Engineering news on Phys.org


No, you can use what ever logarithmic base. For natural information you could use unit "nat" with base e and for binary information unit "bit" with base 2.
 

Thread 'Questions about the Tesla turbine'

Hi all, i have some questions about the tesla turbine: is a tesla turbine more efficient than a steam engine or a stirling engine ? about the discs of the tesla turbine warping because of the high speed rotations; does running the engine on a lower speed solve that or will the discs warp anyway after time ? what is the difference in efficiency between the tesla turbine running at high speed and running it at a lower speed ( as fast as possible but low enough to not warp de discs) and: i...
View full post »
Thread 'Where is my curb stop?'

Thread 'Where is my curb stop?'

My water meter is submerged under water for about 95% of the year. Today I took a photograph of the inside of my water meter box because today is one of the rare days that my water meter is not submerged in water. Here is the photograph that I took of my water meter with the cover on: Here is a photograph I took of my water meter with the cover off: I edited the photograph to draw a red circle around a knob on my water meter. Is that knob that I drew a red circle around my meter...
View full post »

Similar threads

Is the Theory of Conservation of Information Faulty?
Replies
14
Views
1K
B Is information the 5th state of matter?
Replies
18
Views
3K
Help understanding Shannon entropy
Replies
9
Views
4K
Proof that all information can be coded in binary?
Replies
3
Views
2K
B Transferring information faster than the speed of light
Replies
24
Views
2K
A Algorithmic information theory's QM view of reality
Replies
2
Views
2K
I A Bold New Take on Quantum Theory
Replies
25
Views
5K
How is information sent to and interpreted by computer monitors?
Replies
8
Views
2K
What is the minimum observer energy?
Replies
3
Views
2K
How Does Unequal Probability Affect Entropy Calculation in Information Theory?
Replies
3
Views
2K

Hot Threads

Recent Insights

Back
Top