Information Theory: Data Processing Inequality, violation?

  • Thread starter dezeegt
  • Start date
  • #1
1
0

Main Question or Discussion Point

Let's suppose I have a speech signal with frequency content >300 Hz. I then add noise to this signal, that happens to be somewhere below 300 Hz. I then high-pass filter the signal (300+ Hz) and I have increased the mutual information and seemingly violated the data processing inequality.

Can anyone explain why this can occur? I assume it's because the data processing inequality does not apply since the input signal is not stationary and it's probably not really a Markov chain...but I'm not sure.

Thanks!
 

Answers and Replies

  • #2
315
1
By "data processing inequality" I guess you mean this: http://www.vis.caltech.edu/~zoltan/szeged4/img9.htm

By "increased the mutual information" I assume you mean, increased the mutual information between the signal and the output of the high-pass filter, by adding the noise. But the data processing inequality doesn't say the inclusion of R1 can't increase I(S, R2), it only says I(S,R1) > I(S,R2). There's no contradiction.
 
Last edited by a moderator:

Related Threads for: Information Theory: Data Processing Inequality, violation?

Replies
2
Views
112
  • Last Post
Replies
7
Views
2K
Replies
1
Views
3K
  • Last Post
Replies
9
Views
3K
  • Last Post
Replies
0
Views
1K
Replies
0
Views
2K
Replies
2
Views
2K
Top