Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Information Theory: Data Processing Inequality, violation?

  1. Jun 7, 2009 #1
    Let's suppose I have a speech signal with frequency content >300 Hz. I then add noise to this signal, that happens to be somewhere below 300 Hz. I then high-pass filter the signal (300+ Hz) and I have increased the mutual information and seemingly violated the data processing inequality.

    Can anyone explain why this can occur? I assume it's because the data processing inequality does not apply since the input signal is not stationary and it's probably not really a Markov chain...but I'm not sure.

  2. jcsd
  3. Jun 8, 2009 #2
    By "data processing inequality" I guess you mean this: http://www.vis.caltech.edu/~zoltan/szeged4/img9.htm

    By "increased the mutual information" I assume you mean, increased the mutual information between the signal and the output of the high-pass filter, by adding the noise. But the data processing inequality doesn't say the inclusion of R1 can't increase I(S, R2), it only says I(S,R1) > I(S,R2). There's no contradiction.
    Last edited by a moderator: Apr 24, 2017
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Similar Discussions: Information Theory: Data Processing Inequality, violation?