1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Information Theory and Mathematics

  1. Jan 26, 2013 #1
    In (Shannon) information theory, information is said to be the log of the inverse of the probability - what is the information content of 1+1=2? Or for that matter any fundamental axiom.
  2. jcsd
  3. Jan 27, 2013 #2


    User Avatar

    the information content (measured in the sense of Shannon) of the message: "1+1=2" or any other proven theorem is 0.

    you have to define what all of the possible messages are first, then figure out the probability of each message, and then you can start asking what the measure of information content is.
  4. Jan 27, 2013 #3


    User Avatar
    Science Advisor

    That follows very simply from what you give. The probability that "1+ 1= 2" (or any axiom) is true is 1.0 and its logarithm is 0.
  5. Jan 27, 2013 #4
    Doesn't this seem to use a relative sense of probability? For example could we have said at one time this was in fact a very improbable event or do we rely on mathematical realism to tell us this was never improbable?

    To give another example can we say structure in mathematics is what gives this a probability of 1?

    I really appreciate your responses.
  6. Jan 27, 2013 #5

    Stephen Tashi

    User Avatar
    Science Advisor

    You could say that, but it doesn't have a specific meaning! - and all the vague philosophical threads get locked or moved to the PF Lounge section.

    If you are looking for some universal interpretation of "information" in the physical world, you'll have to go to the physics sections. In mathematics, the concept of "information" is no more universal than the concept of "rate of change". If you define a specific variable (such as the position of an object, or the price of a stock) then you can talk about a specific "rate of change". If you define a specific probability distribution, then you can talk about it's information. If you don't get specific then "information" isn't a specific thing.
  7. Jan 27, 2013 #6
    I understand thank you.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook