• Support PF! Buy your school textbooks, materials and every day products Here!

Conditional Entropy and Kullback–Leibler divergence

  • Thread starter avkr
  • Start date
  • #1
12
0

Homework Statement


To find relation between conditional (Shanon) entropy and KL divergence.

Homework Equations


Conditional Entropy: H[X | Y] = H[X,Y] - H[Y]
KL Divergenece: H[X || Y] = -H[X] - Σx ln(y)

The Attempt at a Solution


H[p(x,y) || p(x)p(y)] = -H[p(x,y)] + H[p(x)] + H[p(y)]
 

Answers and Replies

Related Threads on Conditional Entropy and Kullback–Leibler divergence

  • Last Post
Replies
10
Views
1K
  • Last Post
Replies
7
Views
3K
  • Last Post
Replies
2
Views
3K
  • Last Post
Replies
5
Views
2K
  • Last Post
Replies
5
Views
7K
  • Last Post
Replies
2
Views
889
  • Last Post
Replies
2
Views
3K
  • Last Post
Replies
1
Views
2K
  • Last Post
Replies
3
Views
2K
  • Last Post
Replies
2
Views
2K
Top