PetitPrince
Nov20-11, 09:17 AM
hi everybody, :smile:
while looking on the mutual information of two variables, one find that it is concave of p(x) given p(x|y) and convex of p(x|y) given p(x).
the first statement is okey, but when it comes to proving the second, i get stuck, even when i find proofs already done i didn't get how they can conclude the convexity of I(x,y) as a function of p(x|y) from the convexity of the relative entropy D(p||q).
here is a piece of the proof i didnt understand
http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture3.pdf
if you have any idea, i'd very much appreciate it.
thank you in advance.
while looking on the mutual information of two variables, one find that it is concave of p(x) given p(x|y) and convex of p(x|y) given p(x).
the first statement is okey, but when it comes to proving the second, i get stuck, even when i find proofs already done i didn't get how they can conclude the convexity of I(x,y) as a function of p(x|y) from the convexity of the relative entropy D(p||q).
here is a piece of the proof i didnt understand
http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture3.pdf
if you have any idea, i'd very much appreciate it.
thank you in advance.