Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Formula for entropy conditional on >1 variables?

  1. Oct 6, 2011 #1

    I want to compute the conditional entropy H(A | B,C), given probability distributions for each of the variables.

    It would be nice to have a right-hand side not involving conditionals. H(A|B) = H(A,B) - H(B) but how does it work out if there are more than one conditional variable?

    I tried, as a dumb guess, H(A,B,C) - H(B,C), but from using it further down the line, I doubt that's correct.

  2. jcsd
  3. Oct 7, 2011 #2
    I worked through it again and discovered an error or two in my calculations. In the end it looks like

    H(A|B,C) = H(A,B,C) - H(B,C)

    is correct after all.

    This is quite clear when illustrated with a Venn diagram similar to the one https://secure.wikimedia.org/wikipedia/en/wiki/Conditional_entropy" [Broken].
    Last edited by a moderator: May 5, 2017
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook