Why is the minus sign removed in information entropy calculations?

  • Thread starter Thread starter dervast
  • Start date Start date
  • Tags Tags
    Entropy
dervast
Messages
132
Reaction score
1
Hey to the nice community. I am reading about information entropy a size that can be calculated by the H(x)=-Sum[p(x)log(2)p(x)]
then i want to calculate the entropy for the following problem. Consider a random variable that has a uniform distribution over 32 outcomes and i want to find the entroyp
H(x)=-Sum(1/32*log(1/32) and i ll sum 32 times the quantity before . /the problem is that the result is -5 and the book says that the entropy is 5 bits. Why do we remoive the minus sign fromentropy calculations?
 
Physics news on Phys.org
The result is 5. log(1/32)=(-5).
 
My excel calculates the =LOG(1/32;0.5)
for 5 and not for -5 as u mention
 
Thx a lot for your reply
now i need something more
Why in a binary symetric channel the channel is calculated for
C=1+plogp+(1-p)log(1-p)

I only know that the channel is denoted as C=maxI(X;Y)
btw what ; means in X;Y?
Unfortunately my book doesn't mention these things so if u can reply me or provide me with some good links that will be rezlly nice
 
LOG(1/32;0.5)? Isn't that the log base 1/2? Thought you wanted log base 2? Why not LOG(1/32;2)? As for your followup question - I don't know!? You might want to start a new thread and see if you can find someone who does.
 
Thx a lot.. u are right i have used the function log incorrectly. As for my 2nd question i ll wait to see if someone will reply.
Thx a lot for help though
 
Hi, I had an exam and I completely messed up a problem. Especially one part which was necessary for the rest of the problem. Basically, I have a wormhole metric: $$(ds)^2 = -(dt)^2 + (dr)^2 + (r^2 + b^2)( (d\theta)^2 + sin^2 \theta (d\phi)^2 )$$ Where ##b=1## with an orbit only in the equatorial plane. We also know from the question that the orbit must satisfy this relationship: $$\varepsilon = \frac{1}{2} (\frac{dr}{d\tau})^2 + V_{eff}(r)$$ Ultimately, I was tasked to find the initial...
The value of H equals ## 10^{3}## in natural units, According to : https://en.wikipedia.org/wiki/Natural_units, ## t \sim 10^{-21} sec = 10^{21} Hz ##, and since ## \text{GeV} \sim 10^{24} \text{Hz } ##, ## GeV \sim 10^{24} \times 10^{-21} = 10^3 ## in natural units. So is this conversion correct? Also in the above formula, can I convert H to that natural units , since it’s a constant, while keeping k in Hz ?
Back
Top