Formula for entropy conditional on >1 variables?


by Inquirer
Tags: conditional entropy, entropy, multiple variables
Inquirer
Inquirer is offline
#1
Oct6-11, 06:34 PM
P: 4
Hello,

I want to compute the conditional entropy H(A | B,C), given probability distributions for each of the variables.

It would be nice to have a right-hand side not involving conditionals. H(A|B) = H(A,B) - H(B) but how does it work out if there are more than one conditional variable?

I tried, as a dumb guess, H(A,B,C) - H(B,C), but from using it further down the line, I doubt that's correct.

Thanks!
Phys.Org News Partner Science news on Phys.org
Better thermal-imaging lens from waste sulfur
Hackathon team's GoogolPlex gives Siri extra powers
Bright points in Sun's atmosphere mark patterns deep in its interior
Inquirer
Inquirer is offline
#2
Oct7-11, 11:25 AM
P: 4
I worked through it again and discovered an error or two in my calculations. In the end it looks like

H(A|B,C) = H(A,B,C) - H(B,C)

is correct after all.

This is quite clear when illustrated with a Venn diagram similar to the one here.


Register to reply

Related Discussions
conditional PDF with multiple random variables Calculus & Beyond Homework 4
Conditional expectation on multiple variables Set Theory, Logic, Probability, Statistics 2
Conditional Entropy, H(g(X)|X) Set Theory, Logic, Probability, Statistics 2
Expectation conditional on the sum of two random variables Calculus & Beyond Homework 0