Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Minimize entropy of P = maximize entropy of ?

  1. Dec 12, 2008 #1
    minimize entropy of P == maximize entropy of ???

    I am facing the following problem:

    - I have distribution (or function) which depends on some parameter.
    - I want to find the parameter which minimizes the entropy of the distribution.

    In the particular situation I am facing I really need to reformulate this problem in a entropy-maximization problem.

    In other words, is it possible to find a P' for which maximizing its entropy, is equivalent to minimize the entropy of P?

    Thanks in advance!
  2. jcsd
  3. Dec 12, 2008 #2
    Re: minimize entropy of P == maximize entropy of ???

    If I understand your question right, you have a conditional probability distribution, e.g. p(x|y) conditional on y. You want to find the y that minimizes the entropy of p(x|y).

    Then you hope finding some transformation x -> z so that the y that maximizes the entropy of p(z|y) minimizes that of p(x|y).

    In that case, the short answer is: no, it is not possible.
  4. Dec 12, 2008 #3
    Re: minimize entropy of P == maximize entropy of ???

    Perhaps you could be a bit more specific about what you're trying to do? Why are you interested in the minimum-entropy parameter, and why would you want to recast such a problem as a maximum-entropy problem? I can think of some examples where an entropy minimization could be re-written as an entropy maximization of a related distribution, but they're all kind of trivial, and I can't think of a case where doing so wouldn't just be a needless complication.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook