Converting between bits, nats and dits

  • Thread starter Thread starter Rasalhague
  • Start date Start date
  • Tags Tags
    Bits
Click For Summary
Converting between bits, nats, and dits involves understanding their relationship through logarithmic functions. While bits are commonly used, nats are based on natural logarithms, leading to confusion about their compatibility. The discussion highlights that although both units measure entropy, they do not have a direct affine relationship, similar to Celsius and Fahrenheit. A conversion factor exists, as demonstrated by the logarithmic identity linking natural and base-2 logarithms. Ultimately, the method of expressing entropy as a sum or integral does not affect the conversion between these units.
Rasalhague
Messages
1,383
Reaction score
2
Given a number representing information entropy in some base, is there a well-defined way to convert this to the number which would represent the same entropy according to another base? Most of the definitions I've read so far use bits, but Mathematic uses nats, and Wolfram Alpha says "nats and bits are not compatible" in response to the input "convert nats to bits". But don't the two units measure the same quality? Perhaps, by not compatible, it means only there isn't an affine relationship between them, as between, say Celcius and Fahrenheit.

It seems like there would usually be more than one way to express given number as a sum or integral. Suppose I found one way of expressing an entropy in nats as a sum or integral, then replaced the natural logarithms with base 2 logarithms, would the result be the same whichever way I found of expressing the original number of bits as a sum or integral?
 
Physics news on Phys.org
Just a simple conversion factor:

http://en.wikipedia.org/wiki/Bit#Other_information_units

Because

\log_e(x)=\frac{\log_2(x)}{\log_2(e)},

so

- \sum_{x \in A_X} \log_2(P_X(\left \{ x \right \}))\cdot P_X(\left \{ x \right \})=-\log_2(e)\sum_{x \in A_X}\log_e(P_X(\left \{ x \right \}))\cdot P_X(\left \{ x \right \}).
 
Last edited:
The standard _A " operator" maps a Null Hypothesis Ho into a decision set { Do not reject:=1 and reject :=0}. In this sense ( HA)_A , makes no sense. Since H0, HA aren't exhaustive, can we find an alternative operator, _A' , so that ( H_A)_A' makes sense? Isn't Pearson Neyman related to this? Hope I'm making sense. Edit: I was motivated by a superficial similarity of the idea with double transposition of matrices M, with ## (M^{T})^{T}=M##, and just wanted to see if it made sense to talk...

Similar threads

  • · Replies 11 ·
Replies
11
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 85 ·
3
Replies
85
Views
8K
  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 11 ·
Replies
11
Views
2K
  • · Replies 11 ·
Replies
11
Views
3K
Replies
3
Views
2K
  • · Replies 9 ·
Replies
9
Views
4K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K