Natural logarithms in stat mech

[HJ Farnsworth]

Greetings,

This question makes reference to the stat mech book, “Fundamentals of Statistical and Thermal Physics”, by Reif, so people who have that book will probably understand where I am coming from most easily. However, the main points/questions of this post are independent of the book, so I would very much appreciate contributions from anyone.

In section 3.3, pages 98-99, Reif derives the energy at which P(E) reaches a maximum value, where E is the energy of one system which is thermally interacting with another. He states,

“To locate the position of the maximum of P(E), or equivalently, of the maximum of its logarithm,…”

He then continues by setting the derivative of ln(P) with respect to E equal to 0, rather than simply setting the derivative of P with respect to E equal to 0.

He also says, in a footnote on page 99,

“…The reason that it is somewhat more convenient to work with ln P instead of P itself is that the logarithm is a much more slowly varying function of the energy E, and that it involves the numbers Ω and Ω' as a simple sum rather than as a product.”

I think Reif’s textbook is pretty good overall – however, I dislike it when derivations in textbooks involve steps that seem pointless, and the switch to natural logarithms, as presented in this derivation, seems pointless. Additionally, the reason given in the footnote for switching to natural logarithms seems weak to me. It wouldn’t bother me except for the fact that it seems to needlessly add an extra level of abstraction to the quantities being dealt with.

In other words, probability P and number of states Ω are simple and un-abstract quantities, and I’m not satisfied with the explanations that I have heard for switching to logarithms of those quantities. It’s not that I find the math hard – it’s that it is not clear to me why it is necessary, or why it is worth the cost adding more levels of abstraction.

The best justification that I can think of is that natural logarithms relate more easily to other statistical quantities, like temperature, as well as various state variables. However, this doesn't fully satisfy me either, so I was wondering – what do other people think, and does anyone have any concise reasons/ways of thinking about it that justify the use of natural logarithms more convincingly than the ones that I have?

Thanks very much for any thoughts.

-HJ Farnsworth

 

[mfb]

The logarithm simplifies the equations significantly - maybe not the equation you have there right now, but several equations you will see in statistical physics and thermodynamics.
In addition, those logs will get a physical meaning later on.

 

[Philip Wood]

You could argue that the logarithms enter naturally, for we seek the number of states of systems 1 and 2 in contact with each other (consistent with total energy E₀), so we need to maximise $\Omega_1 \Omega_2$ wrt the energy of one of the systems, E₁ (say).

So $\frac{d(\Omega_1 \Omega_2)}{dE_1} = 0$,

So $\Omega_2 \frac{d\Omega_1}{dE_1} + \Omega_1 \frac{d\Omega_2}{dE_1}= 0$

But $E_1 + E_2 = E_0$ so $\frac{d}{dE_1} = - \frac{d}{dE_2}$.

So $\Omega_2 \frac{d\Omega_1}{dE_1} = \Omega_1 \frac{d\Omega_2}{dE_2}$

Dividing through by $\Omega_1 \Omega_2$,

$\frac{\frac{d\Omega_2}{dE_2}}{\Omega_2} = \frac{\frac{d\Omega_1}{dE_1}}{\Omega_1}$

So $\frac{d(\ln\Omega_2)}{dE_2} = \frac{d(\ln\Omega_1)}{dE_1}$

Having said all this, I do think that in some places in his book, Reif makes unjustified claims about what can legitimately be deduced from the slowly varying nature of the log function. But he wears one into submission with the lengthiness of his explanations. The alternative derivation of the canonical distribution which he give on pp229-231, describing it rather patronisingly as 'more cumbersome [...] but [with] some instructive features' is in fact rather more rigorous, and logs enter naturally - but you do have to get to grips with the Lagrange Multiplier method...

 

[Philip Wood]

HJF: I'd be interested to know whether you've made any progress on the point you raised.

 

[BruceW]

I'd say pretty much the same as mfb. the logarithm shows up a lot in statistical physics. For your particular example, using the logarithm just simplifies calculation. It is not really necessary. So I can see where you're coming from, by saying 'why does Reif introduce the logarithm here?' The reason I think is because this Reif person wants to get the reader familiar with logarithm of probability.

 

[vanhees71]

Sometimes it's simpler to get a derivative done using
$$\frac{\mathrm{d}}{\mathrm{d} x} \ln[f(x)]=\frac{f'(x)}{f(x)}.$$
That's all in this example.

I've had always the impression that Reif explains a bit too much rather than too little. Usually, if I don't understand something in statistical physics, I look at Reif for lengthier calculations. Often that helps to get the question answered, sometimes you get confused from too many words ;-).

 

[jtbell]

In other words, probability P and number of states Ω are simple and un-abstract quantities, and I’m not satisfied with the explanations that I have heard for switching to logarithms of those quantities. It’s not that I find the math hard – it’s that it is not clear to me why it is necessary, or why it is worth the cost adding more levels of abstraction.

Have you tried doing the derivation without using logarithms? If so, how does the amount of effort involved compare between the two methods?

I don't have Reif myself, and haven't done that particular derivation, so I can't speak to this particular situation from personal experience. However, I do know from experience that in many situations there is a "natural" or obvious method for doing something which turns out to be rather laborious, and a "clever" method that is less time-consuming but involves a trick that is not obvious at first glance. After doing many derivations, you eventually learn to recognize tricks like this in advance.

 

[BruceW]

It's a great (and terrible) feeling when you are working on something for ages, and then you look on wiki, and there is a simple trick that makes it all so much easier.

 

[Philip Wood]

Agree that, with experience, one learns when taking logs early on will simplify subsequent maths. But I thought HJF wanted logs to enter naturally. Hence my first post (hash 3).

 

[AlephZero]

“…The reason that it is somewhat more convenient to work with ln P instead of P itself is that the logarithm is a much more slowly varying function of the energy E, and that it involves the numbers Ω and Ω' as a simple sum rather than as a product.”

I don't have the book, but reading across from other applications of statistics I think he is saying two different things here.

(1) Logarithmic differentiation is a good mathematical "trick" when dealing with functions that are a product of many terms, since it eliminates a lot of "clutter" in the math.

(2) working with the log of probabilities is very useful in large scale numerical work, since it keeps the magnitude of the numbers within the range of standard computer arithmetic. For example if individual events have a probability of the order of 0.1, the probability of some combination of a million events will typically have a probability of the order if $10^{-1000000}$, but computer floating point arithmetic can only represent numbers down to about $10^{-300}$. But the log of the probability is of the order of $10^{-6}$ which is no problem to deal with numerically.

That certainly applies to numerical methods to find maximum likelihood estimates of statistical parameters, and I expect there would be similar numerical applications in statistical mechanics.

 

[Dale]

the logarithm shows up a lot in statistical physics

To amplify this, the log shows up a lot just in statistics alone. It is probably the most common statistical transformation used.

 

[BruceW]

that's interesting. I guess I have not learned much statistics that was not 'for' physics.