Mathematica Mathematical detail regarding Boltzman's H thm

[quasar987]

In the "proof" of the theorem, my course notes defines P_r(t) as the probability to find the system is state r at time t, and it defined H as the mean value of \ln P_r over all acesible states:

H \equiv \sum_r P_r\ln P_r

Is is right to call the above sum the "mean value of ln P_r" ?! Cause given a quantity u, the mean value of f(u) is defined as

\sum_i P(u_i)f(u_i)

So the mean value of \ln P_r should be

\sum_r P(P_r)\ln P_r

But P(P_r) does not make sense.I confessed my confusion to the professor in more vague terms (at the time, I only tought the equation looked suspicious), but he said there was nothing wrong with it. I say, H could be called at best "some kind" of mean value of ln(Pr).

 

[marcusl]

The formula you are asking about
H=SUM(P_r * ln(P_r))
is Shannon's definition of information entropy; it is related back to statistical mechanics by multiplying by the Boltzmann factor -k to give Gibbs' entropy
S=-k*SUM(P_r * ln(P_r)).
H is correctly called the mean of ln(P_r). Don't get confused by your notation--you must sum over states, not variables. P(u_i) is the probability of finding u in the ith state, so your first and second equations are actually the same.

You may prefer a less confusing way of writing the mean or expectation
E[f(x)] = SUM(P_r * f(x_r)),
as used in Jaynes, Phys. Rev. 106:620-630 (1957) (who discusses the connection between information H and stat mech S)
or Chandler, Intro to Modern Stat Mech, ch. 3 (1987).

Having defended the correctness of the definition, I have to add that it isn't a very useful way of thinking of H. Take a very simple case, that of an ideal gas, as an example. One sums r over W equally probable microstates (p=1/W) so the entropy of a macrostate of the gas system reduces to
H = -lnW;
multiplying by the constant -k gives exactly Boltzmann's entropy H=k*lnW. But how is thinking of H as the mean value of log of probability helpful or insightful? Instead, entropy reflects the number of possible ways W that a macrostate can be realized, and the second law ensures that the macrostate adopted in equilibrium is that which can be realized in the most number of ways. To tie it to information theory, this is the maximum entropy state.

Hope this helps.