1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Minimize entropy of P = maximize entropy of ?

  1. Dec 12, 2008 #1
    minimize entropy of P == maximize entropy of ???

    I am facing the following problem:

    - I have distribution (or function) which depends on some parameter.
    - I want to find the parameter which minimizes the entropy of the distribution.

    In the particular situation I am facing I really need to reformulate this problem in a entropy-maximization problem.

    In other words, is it possible to find a P' for which maximizing its entropy, is equivalent to minimize the entropy of P?

    Thanks in advance!
  2. jcsd
  3. Dec 12, 2008 #2
    Re: minimize entropy of P == maximize entropy of ???

    If I understand your question right, you have a conditional probability distribution, e.g. p(x|y) conditional on y. You want to find the y that minimizes the entropy of p(x|y).

    Then you hope finding some transformation x -> z so that the y that maximizes the entropy of p(z|y) minimizes that of p(x|y).

    In that case, the short answer is: no, it is not possible.
  4. Dec 12, 2008 #3
    Re: minimize entropy of P == maximize entropy of ???

    Perhaps you could be a bit more specific about what you're trying to do? Why are you interested in the minimum-entropy parameter, and why would you want to recast such a problem as a maximum-entropy problem? I can think of some examples where an entropy minimization could be re-written as an entropy maximization of a related distribution, but they're all kind of trivial, and I can't think of a case where doing so wouldn't just be a needless complication.
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?

Similar Discussions: Minimize entropy of P = maximize entropy of ?
  1. Shannon entropy of QXZ (Replies: 2)

  2. Rank Vector Entropy (Replies: 0)