Does entropy increase with improbability?

[MeneMestre]

Well, maybe that's a simple question, but it has been pissing me off for some time ... Does entropy increase with improbability?

 

[Nugatory]

What is the definition of entropy (the real one that has an equation in it, not the vague English-language bit about "disorder")? The answer to that question will suggest an answer to your question.

 

[NFuller]

Does entropy increase with improbability?

Entropy is defined for all systems (whether or not the are at equilibrium) by the Gibbs entropy equation
$$S=-k\sum_{i}^{l}p_{i}\ln p_{i}$$
where $p_{i}$ is the probability for the system to enter the $i$th state. Notice that the entropy has a minimum value of zero for the delta function distribution $p_{i}=\delta_{ij}$ and a maximum value for the uniform distribution $p_{i}=1/l$. Thus the entropy is how spread out the probability distribution $p_{i}$ is. A system which has many equal probability states is at higher entropy than a system with only a few probable states.