Is there any background independent theory of information?

[divadwg]

Is there any theory of information that considers a 'bit' to be something that only exists relative to a pair of systems exchanging information? I.E there is no such thing as an isolated 'bit'.

 

[jim mcnamara]

Do you mean something like the Shannon-Wiener H? That seems to match what you are asking.

 

[ravenprp]

There is ongoing debate in the field of information theory about the concept of background independence. Some theories, such as classical information theory, are based on a fixed background, such as a physical medium or a set of predetermined symbols. In these theories, a bit is defined as a fundamental unit of information and is considered to be independent of any specific system or context.

However, there are also theories that propose a more background independent approach, such as quantum information theory and algorithmic information theory. In these theories, a bit is seen as a more abstract concept that is dependent on the specific system or context in which it is being used. This means that a bit may have different properties or meanings in different systems or contexts.

One example of a theory that considers a bit to be relative to a pair of systems exchanging information is the relational theory of information. This theory suggests that a bit only exists in the context of a relationship between two systems, and it is not a meaningful concept in isolation. In this view, information is not a property of a single system, but rather emerges from the interaction between multiple systems.

Overall, the concept of background independence in information theory is still a topic of discussion and there is no consensus on a single theory that fully embraces it. However, there are certainly theories that consider a bit to be a relative and context-dependent concept, rather than an isolated and independent entity.