# Maximum Entropy Distribution Given Marginals

1. Jul 5, 2011

### Legendre

Hi all,

I'm a pure mathematician (Graph Theory) who has to go through a Physics paper, and I am having trouble getting through a part of it. Maybe you guys can point me in the right direction:

Let P(x,y) be a joint distribution function.

Let H = - $\Sigma$x,y P(x,y) log P(x,y), which is the Entropy.

We are given the marginal probability density functions q1(x) and q2(y).

To get the maximum entropy distribution consistent with the marginals, we solve this problem:

max H

subject to constraints,

$\Sigma$y P(x,y) = q1(x)
$\Sigma$x P(x,y) = q2(y)

---------------------

The Lagrangian is,

L = - $\Sigma$ x,y (P(x,y) log P(x,y)) + $$\lambda$$1 $\Sigma$y (P(x,y) - q1(x)) + $$\lambda$$2 $\Sigma$x (P(x,y) - q2(y))

To find the stationary point of L, we differentiate it with respect to a particular P(x',y') while keeping the other variables constant.

I know how to do this for the first sum, which gives the solution -(1 + log P(x',y')).

---------------------

How do we differentiate the second and third sum with respect to P(x',y')?

The paper gave the solution to differentiating L with respect to P(x',y') as:

-(1 + log P(x',y')) + $$\lambda$$1 + $$\lambda$$2

Thanks!