What is information?or how is information connected with change

[nouveau_riche]

what is information?
or
how is information connected with change and dependency ?
take an example ,
y=f(x)
if y is independent of x(say y=3),will there be information?

how does the change df(x)/dx effect the information content?

 

[xts]

Classical Shannon's definition of information carried by some measurement giving the result x is that I(x)=-log₂p(x); where p(x) was the a priori probability of that outcome. Especially, if the measurement may give only two equiprobable results (like tossed coin) the information carried by every measurement is 1 bit.

It makes no sense to speak about information carried by real functions (except of very limited subset of them), as such functions are never realized. Formally - they carry infinite information. But in reality your measurements have always limited accuracy, and outcomes on consecutive measurements are usually correlated.
The amount of information carried by signals of limited frequency bandwith, measured with limited accuracy (or blurred by noise) is finite and described by Shannon-Hartley theorem - which is one of foundations of telecommunications

See: http://en.wikipedia.org/wiki/Information_theory

 

[nouveau_riche]

Classical Shannon's definition of information carried by some measurement giving the result x is that I(x)=-log₂p(x); where p(x) was the a priori probability of that outcome. Especially, if the measurement may give only two equiprobable results (like tossed coin) the information carried by every measurement is 1 bit.

It makes no sense to speak about information carried by real functions (except of very limited subset of them), as such functions are never realized. Formally - they carry infinite information. But in reality your measurements have always limited accuracy, and outcomes on consecutive measurements are usually correlated.
The amount of information carried by signals of limited frequency bandwith, measured with limited accuracy (or blurred by noise) is finite and described by Shannon-Hartley theorem - which is one of foundations of telecommunications

See: http://en.wikipedia.org/wiki/Information_theory

it would be better if you could clarify things with my example

 

[Studiot]

it would be better if you could clarify things with my example

XTS did give you an answer -

If df(x)/dx exists the function: f(x) is continuous and thus carries infinite information.

 

[xts]

it would be better if you could clarify things with my example

I either can't (as maybe I didn't get your point in not quite clear example) or I did it already (telling you, that for real functions, being only idealisations used in modelling of real processes, the information they carry is infinite, so it makes no sense to discuss them in context of information theory)

 

[nouveau_riche]

XTS did give you an answer -

If df(x)/dx exists the function: f(x) is continuous and thus carries infinite information.

as to my knowledge ,according to shannon's theorem anything has information if nothing if there isn't uncertainty at receiver's end
in the above case,knowing the function in advance reveals every information in advance

 

[Studiot]

as to my knowledge ,according to shannon's theorem anything has information if nothing if there isn't uncertainty at receiver's end
in the above case,knowing the function in advance reveals every information in advance

Pardon?

 

[nouveau_riche]

Pardon?

A key measure of information is known as entropy, which is usually expressed by the average number of bits needed for storage or communication. Entropy quantifies the uncertainty involved in predicting the value of a random variable. For example, specifying the outcome of a fair coin flip (two equally likely outcomes) provides less information (lower entropy) than specifying the outcome from a roll of a die (six equally likely outcomes).

those are lines from wiki link of information theory

if u still don't get it then this one is interesting

Suppose one transmits 1000 bits (0s and 1s). If these bits are known ahead of transmission (to be a certain value with absolute probability), logic dictates that no information has been transmitted. If, however, each is equally and independently likely to be 0 or 1, 1000 bits (in the information theoretic sense) have been transmitted

 

[xts]

Suppose one transmits 1000 bits (0s and 1s). If these bits are known ahead of transmission (to be a certain value with absolute probability), logic dictates that no information has been transmitted. If, however, each is equally and independently likely to be 0 or 1, 1000 bits (in the information theoretic sense) have been transmitted

That's exactly what I gave you as explanation in my first response: information is equal to -log₂ of a priori probability of the outcome. If the outcome is known in advance, then its probability is 1 , thus information is 0.

But, how is it related to your original question about continuous functions and their derivatives in context of information theory? I still can't get what were you asking about.

 

[nouveau_riche]

That's exactly what I gave you as explanation in my first response: information is equal to -log₂ of a priori probability of the outcome. If the outcome is known in advance, then its probability is 1 , thus information is 0.

But, how is it related to your original question about continuous functions and their derivatives in context of information theory? I still can't get what were you asking about.

okay let's begin from start
"suppose i gave you an equation describing an event"
now you know in advance about the behavior of event,so was there any information in the event you were observing?

 

[xts]

"suppose i gave you an equation describing an event"
now you know in advance about the behavior of event,so was there any information in the event you were observing?

If you give me not only equation, but also starting parameters (not only pendulum equation, but also its initial angle and the time you released it), then I can get no information watching the experiment. I've already got all possible knowledge about it - and there is nothing more to learn from it.

 

[nouveau_riche]

If you give me not only equation, but also starting parameters (not only pendulum equation, but also its initial angle and the time you released it), then I can get no information watching the experiment. I've already got all possible knowledge about it - and there is nothing more to learn from it.

so as per your lines there a difference in information between the following cases
case 1: forming a graph of any event from the equation
case 2:forming an equation from the observation

 

[nouveau_riche]

the thread remains unresponsive
and i cannot absorb that there is not much left in this in thread to discuss

 

[xts]

I didn't respond, as I just didn't understand your question. Would you ask it more clearly?

 

[nouveau_riche]

I didn't respond, as I just didn't understand your question. Would you ask it more clearly?

is there a difference between observing events after having an equation with that of having an observation first followed by checking it's validation?

 

[xts]

There is no difference in observing.
But there is a difference in information you obtain by this observation.

Take an example of looking out the window to see the Sun on the sky.
If you just woke up with terrible hangover - it brings you some (don't ask how many bits) information, that it is already about 11 AM.
If you just got waken by an alarm clock programmed for 7:15AM and you see the Sun on the sky - it brings you no information (about time, however it may still bring some about the weather) - you already knew what time was, so - as you are fresh and sober - you could easily predict where the Sun should be seen. Confirmation of something certain brings no information.

 

[nouveau_riche]

There is no difference in observing.
But there is a difference in information you obtain by this observation.

Take an example of looking out the window to see the Sun on the sky.
If you just woke up with terrible hangover - it brings you some (don't ask how many bits) information, that it is already about 11 AM.
If you just got waken by an alarm clock programmed for 7:15AM and you see the Sun on the sky - it brings you no information (about time, however it may still bring some about the weather) - you already knew what time was, so - as you are fresh and sober - you could easily predict where the Sun should be seen. Confirmation of something certain brings no information.

if there is a difference in information
then the more laws we discover in this universe,the more we loose the net information in the universe?

 

[xts]

if there is a difference in information
then the more laws we discover in this universe,the more we loose the net information in the universe?

We don't lose information. We just sometimes receive the same information twice. Information is not an addititive property. If you know the laws ruling the experiment, you may better predict its results, so its actual outcome gives you less (or no at all) information.