Information Theory and Mathematics

wheelersbit

In (Shannon) information theory, information is said to be the log of the inverse of the probability - what is the information content of 1+1=2? Or for that matter any fundamental axiom.

rbj

the information content (measured in the sense of Shannon) of the message: "1+1=2" or any other proven theorem is 0.

you have to define what all of the possible messages are first, then figure out the probability of each message, and then you can start asking what the measure of information content is.

HallsofIvy

That follows very simply from what you give. The probability that "1+ 1= 2" (or any axiom) is true is 1.0 and its logarithm is 0.

wheelersbit

Doesn't this seem to use a relative sense of probability? For example could we have said at one time this was in fact a very improbable event or do we rely on mathematical realism to tell us this was never improbable?

To give another example can we say structure in mathematics is what gives this a probability of 1?

I really appreciate your responses.

Stephen Tashi

You could say that, but it doesn't have a specific meaning! - and all the vague philosophical threads get locked or moved to the PF Lounge section.

If you are looking for some universal interpretation of "information" in the physical world, you'll have to go to the physics sections. In mathematics, the concept of "information" is no more universal than the concept of "rate of change". If you define a specific variable (such as the position of an object, or the price of a stock) then you can talk about a specific "rate of change". If you define a specific probability distribution, then you can talk about it's information. If you don't get specific then "information" isn't a specific thing.

wheelersbit

I understand thank you.