Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Formula for entropy conditional on >1 variables?

  1. Oct 6, 2011 #1
    Hello,

    I want to compute the conditional entropy H(A | B,C), given probability distributions for each of the variables.

    It would be nice to have a right-hand side not involving conditionals. H(A|B) = H(A,B) - H(B) but how does it work out if there are more than one conditional variable?

    I tried, as a dumb guess, H(A,B,C) - H(B,C), but from using it further down the line, I doubt that's correct.

    Thanks!
     
  2. jcsd
  3. Oct 7, 2011 #2
    I worked through it again and discovered an error or two in my calculations. In the end it looks like

    H(A|B,C) = H(A,B,C) - H(B,C)

    is correct after all.

    This is quite clear when illustrated with a Venn diagram similar to the one https://secure.wikimedia.org/wikipedia/en/wiki/Conditional_entropy" [Broken].
     
    Last edited by a moderator: May 5, 2017
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Formula for entropy conditional on >1 variables?
Loading...