New Reply

Formula for entropy conditional on >1 variables?

 
Share Thread Thread Tools
Oct6-11, 06:34 PM   #1
 

Formula for entropy conditional on >1 variables?


Hello,

I want to compute the conditional entropy H(A | B,C), given probability distributions for each of the variables.

It would be nice to have a right-hand side not involving conditionals. H(A|B) = H(A,B) - H(B) but how does it work out if there are more than one conditional variable?

I tried, as a dumb guess, H(A,B,C) - H(B,C), but from using it further down the line, I doubt that's correct.

Thanks!
 
PhysOrg.com
PhysOrg
science news on PhysOrg.com

>> Hong Kong launches first electric taxis
>> Morocco to harness the wind in energy hunt
>> Galaxy's Ring of Fire
Oct7-11, 11:25 AM   #2
 
I worked through it again and discovered an error or two in my calculations. In the end it looks like

H(A|B,C) = H(A,B,C) - H(B,C)

is correct after all.

This is quite clear when illustrated with a Venn diagram similar to the one here.
 
New Reply

Tags
conditional entropy, entropy, multiple variables
Thread Tools


Similar Threads for: Formula for entropy conditional on >1 variables?
Thread Forum Replies
conditional PDF with multiple random variables Calculus & Beyond Homework 4
Conditional expectation on multiple variables Set Theory, Logic, Probability, Statistics 2
Conditional Entropy, H(g(X)|X) Set Theory, Logic, Probability, Statistics 2
Expectation conditional on the sum of two random variables Calculus & Beyond Homework 0