# Minimize entropy of P = maximize entropy of ?

1. Dec 12, 2008

### mnb96

minimize entropy of P == maximize entropy of ???

Hello,
I am facing the following problem:

- I have distribution (or function) which depends on some parameter.
- I want to find the parameter which minimizes the entropy of the distribution.

In the particular situation I am facing I really need to reformulate this problem in a entropy-maximization problem.

In other words, is it possible to find a P' for which maximizing its entropy, is equivalent to minimize the entropy of P?

Thanks in advance!

2. Dec 12, 2008

### winterfors

Re: minimize entropy of P == maximize entropy of ???

If I understand your question right, you have a conditional probability distribution, e.g. p(x|y) conditional on y. You want to find the y that minimizes the entropy of p(x|y).

Then you hope finding some transformation x -> z so that the y that maximizes the entropy of p(z|y) minimizes that of p(x|y).

In that case, the short answer is: no, it is not possible.

3. Dec 12, 2008

### quadraphonics

Re: minimize entropy of P == maximize entropy of ???

Perhaps you could be a bit more specific about what you're trying to do? Why are you interested in the minimum-entropy parameter, and why would you want to recast such a problem as a maximum-entropy problem? I can think of some examples where an entropy minimization could be re-written as an entropy maximization of a related distribution, but they're all kind of trivial, and I can't think of a case where doing so wouldn't just be a needless complication.

Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?