- #1

- 12

- 0

## Homework Statement

To find relation between conditional (Shanon) entropy and KL divergence.

## Homework Equations

Conditional Entropy: H[X | Y] = H[X,Y] - H[Y]

KL Divergenece: H[X || Y] = -H[X] - Σx ln(y)

## The Attempt at a Solution

H[p(x,y) || p(x)p(y)] = -H[p(x,y)] + H[p(x)] + H[p(y)]