1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

What is information?or how is information connected with change

  1. Sep 19, 2011 #1
    what is information?
    or
    how is information connected with change and dependency ?
    take an example ,
    y=f(x)
    if y is independent of x(say y=3),will there be information?

    how does the change df(x)/dx effect the information content?
     
  2. jcsd
  3. Sep 19, 2011 #2

    xts

    User Avatar

    Re: information

    Classical Shannon's definition of information carried by some measurement giving the result x is that I(x)=-log2p(x); where p(x) was the a priori probability of that outcome. Especially, if the measurement may give only two equiprobable results (like tossed coin) the information carried by every measurement is 1 bit.

    It makes no sense to speak about information carried by real functions (except of very limited subset of them), as such functions are never realised. Formally - they carry infinite information. But in reality your measurements have always limited accuracy, and outcomes on consecutive measurements are usually correlated.
    The amount of information carried by signals of limited frequency bandwith, measured with limited accuracy (or blurred by noise) is finite and described by Shannon-Hartley theorem - which is one of foundations of telecommunications

    See: http://en.wikipedia.org/wiki/Information_theory
     
  4. Sep 19, 2011 #3
    Re: information

    it would be better if you could clarify things with my example
     
  5. Sep 19, 2011 #4
    Re: information

    XTS did give you an answer -

    If df(x)/dx exists the function: f(x) is continuous and thus carries infinite information.
     
  6. Sep 19, 2011 #5

    xts

    User Avatar

    Re: information

    I either can't (as maybe I didn't get your point in not quite clear example) or I did it already (telling you, that for real functions, being only idealisations used in modelling of real processes, the information they carry is infinite, so it makes no sense to discuss them in context of information theory)
     
    Last edited: Sep 19, 2011
  7. Sep 20, 2011 #6
    Re: information

    as to my knowledge ,according to shannon's theorem anything has information if nothing if there isn't uncertainty at receiver's end
    in the above case,knowing the function in advance reveals every information in advance
     
  8. Sep 20, 2011 #7
    Re: information

    Pardon?
     
  9. Sep 20, 2011 #8
    Re: information

    A key measure of information is known as entropy, which is usually expressed by the average number of bits needed for storage or communication. Entropy quantifies the uncertainty involved in predicting the value of a random variable. For example, specifying the outcome of a fair coin flip (two equally likely outcomes) provides less information (lower entropy) than specifying the outcome from a roll of a die (six equally likely outcomes).

    those are lines from wiki link of information theory

    if u still don't get it then this one is interesting

    Suppose one transmits 1000 bits (0s and 1s). If these bits are known ahead of transmission (to be a certain value with absolute probability), logic dictates that no information has been transmitted. If, however, each is equally and independently likely to be 0 or 1, 1000 bits (in the information theoretic sense) have been transmitted
     
  10. Sep 20, 2011 #9

    xts

    User Avatar

    Re: information

    That's exactly what I gave you as explanation in my first response: information is equal to -log2 of a priori probability of the outcome. If the outcome is known in advance, then its probability is 1 , thus information is 0.

    But, how is it related to your original question about continuous functions and their derivatives in context of information theory? I still can't get what were you asking about.
     
  11. Sep 23, 2011 #10
    Re: information

    okay lets begin from start
    "suppose i gave you an equation describing an event"
    now you know in advance about the behavior of event,so was there any information in the event you were observing?
     
  12. Sep 23, 2011 #11

    xts

    User Avatar

    Re: information

    If you give me not only equation, but also starting parameters (not only pendulum equation, but also its initial angle and the time you released it), then I can get no information watching the experiment. I've already got all possible knowledge about it - and there is nothing more to learn from it.
     
  13. Sep 25, 2011 #12
    Re: information

    so as per your lines there a difference in information between the following cases
    case 1: forming a graph of any event from the equation
    case 2:forming an equation from the observation
     
  14. Sep 29, 2011 #13
    Re: information

    the thread remains unresponsive
    and i cannot absorb that there is not much left in this in thread to discuss
     
  15. Sep 29, 2011 #14

    xts

    User Avatar

    Re: information

    I didn't respond, as I just didn't understand your question. Would you ask it more clearly?
     
  16. Sep 30, 2011 #15
    Re: information

    is there a difference between observing events after having an equation with that of having an observation first followed by checking it's validation?
     
  17. Sep 30, 2011 #16

    xts

    User Avatar

    Re: information

    There is no difference in observing.
    But there is a difference in information you obtain by this observation.

    Take an example of looking out the window to see the Sun on the sky.
    If you just woke up with terrible hangover - it brings you some (don't ask how many bits) information, that it is already about 11 AM.
    If you just got waken by an alarm clock programmed for 7:15AM and you see the Sun on the sky - it brings you no information (about time, however it may still bring some about the weather) - you already knew what time was, so - as you are fresh and sober - you could easily predict where the Sun should be seen. Confirmation of something certain brings no information.
     
  18. Oct 3, 2011 #17
    Re: information

    if there is a difference in information
    then the more laws we discover in this universe,the more we loose the net information in the universe?
     
  19. Oct 3, 2011 #18

    xts

    User Avatar

    Re: information

    We don't lose information. We just sometimes receive the same information twice. Information is not an addititive property. If you know the laws ruling the experiment, you may better predict its results, so its actual outcome gives you less (or no at all) information.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: What is information?or how is information connected with change
  1. What is INFORMATION? (Replies: 20)

  2. What is Information (Replies: 6)

Loading...