Information Theory - Shannon's Self-Information units

  • Thread starter Mihel
  • Start date
  • #1
1
0
Information Theory - Shannon's "Self-Information" units

Hi,

I'm familiar with information and coding theory, and do know that the units of Shannon information content (-log_2(P(A))) are "bits". Where "bit" is a "binary digit", or a "storage device that has two stable states".

But, can someone rigorously prove that the units are actually "bits"? Or we should only accept it as a definition and then justify it with coding examples.

Thanks!
 

Answers and Replies

  • #2
3
0


No, you can use what ever logarithmic base. For natural information you could use unit "nat" with base e and for binary information unit "bit" with base 2.
 

Related Threads on Information Theory - Shannon's Self-Information units

  • Last Post
Replies
3
Views
4K
  • Last Post
Replies
1
Views
1K
  • Last Post
Replies
2
Views
2K
  • Last Post
Replies
1
Views
3K
  • Last Post
Replies
1
Views
3K
L
  • Last Post
Replies
4
Views
6K
  • Last Post
Replies
6
Views
30K
  • Last Post
Replies
3
Views
2K
  • Sticky
  • Last Post
2
Replies
43
Views
107K
Replies
6
Views
1K
Top