Entropy question for 2 input 1 output system

perplexabot · May 26, 2015

Hey all. I have a question regarding the solution to a question, both shown below (only part of the solution is shown). Specifically the line that states: H(Y|X) = H(Z|X). Why does this equality hold? Expanding and using the definition of entropy, I can see that for the above equality to hold, p(Y,X) must equal p(Z,X), right? I am not able to show this. Please some help will be greatly appreciated.

Here is the question with part of the solution:

Printed textbook page showing Channel capacity problem 4 with Y = X + Z (mod 11), Z taking values 1, 2, 3 with probabilities 1/3 each, and the solution's equations 7.14 to 7.16 including H(Y|X) = H(Z|X) = H(Z) = log 3.

perplexabot · May 26, 2015

Hey again! I found the answer to my question, here it is:

Handwritten-style solution to an entropy problem: Z = X + Y gives p(Z = z|X = x) = p(Y = z - x|X = x), followed by a chain of equalities showing H(Z|X) equals H(Y|X) via sums of p(x) times conditional probabilities and logarithms.

PS: Maybe a better name for this thread would have been "Entropy of sum question." I would have renamed it if I could.
Thanks for reading.

Entropy question for 2 input 1 output system

Discussion

Similar threads