Mathematical detail regarding Boltzman's H thm

[quasar987]

In the "proof" of the theorem, my course notes defines $P_r(t)$ as the probability to find the system is state r at time t, and it defined H as the mean value of $\ln P_r$ over all acesible states:

$$H \equiv \sum_r P_r\ln P_r$$

Is is right to call the above sum the "mean value of ln P_r" ?! Cause given a quantity u, the mean value of f(u) is defined as

$$\sum_i P(u_i)f(u_i)$$

So the mean value of $\ln P_r$ should be

$$\sum_r P(P_r)\ln P_r$$

But P(P_r) does not make sense.I confessed my confusion to the professor in more vague terms (at the time, I only tought the equation looked suspicious), but he said there was nothing wrong with it. I say, H could be called at best "some kind" of mean value of ln(Pr).

 

[marcusl]

The formula you are asking about
H=SUM(P_r * ln(P_r))
is Shannon's definition of information entropy; it is related back to statistical mechanics by multiplying by the Boltzmann factor -k to give Gibbs' entropy
S=-k*SUM(P_r * ln(P_r)).
H is correctly called the mean of ln(P_r). Don't get confused by your notation--you must sum over states, not variables. P(u_i) is the probability of finding u in the ith state, so your first and second equations are actually the same.

You may prefer a less confusing way of writing the mean or expectation
E[f(x)] = SUM(P_r * f(x_r)),
as used in Jaynes, Phys. Rev. 106:620-630 (1957) (who discusses the connection between information H and stat mech S)
or Chandler, Intro to Modern Stat Mech, ch. 3 (1987).

Having defended the correctness of the definition, I have to add that it isn't a very useful way of thinking of H. Take a very simple case, that of an ideal gas, as an example. One sums r over W equally probable microstates (p=1/W) so the entropy of a macrostate of the gas system reduces to
H = -lnW;
multiplying by the constant -k gives exactly Boltzmann's entropy H=k*lnW. But how is thinking of H as the mean value of log of probability helpful or insightful? Instead, entropy reflects the number of possible ways W that a macrostate can be realized, and the second law ensures that the macrostate adopted in equilibrium is that which can be realized in the most number of ways. To tie it to information theory, this is the maximum entropy state.

Hope this helps.

 

[tacman]

It is correct to call the sum H as the mean value of ln P_r over all accessible states. The notation \sum_r P_r\ln P_r is a shorthand way of writing the summation \sum_{r} P(r)\ln P(r), where P(r) represents the probability of the system being in state r. This notation is commonly used in statistical mechanics and does not cause any confusion.

The mean value of a function f(u) is defined as \sum_i P(u_i)f(u_i), where P(u_i) represents the probability of the system having the value u_i. In this case, the function f(u) is ln P_r, and the values u_i are the probabilities P_r. Therefore, the mean value of ln P_r is correctly written as \sum_r P_r\ln P_r.

As for the notation P(P_r), it is not used in this context and does not make sense. P_r represents the probability of the system being in state r, and it cannot be used as an argument for another probability function. Therefore, it is correct to say that H is the mean value of ln P_r over all accessible states.