Given equilibrium, why does one only consider a mixed state where the pure states are eigenfunctions of the hamiltonian, i.e. states with an energy eigenvalue?

And for the second question, I quote David Tong's ``Lectures on Statistical Physics'' ( freely and legally accessible on http://www.damtp.cam.ac.uk/user/tong/statphys.html more concretely part 1 page 5):

Why would we conceptually expect the delta E? Say my system is isolated and initially in a certain energy eigenstate, then it will never hop to another energy eigenstate, no matter how close it is to the previous one, right? So what's the physics of this claim?

Thank you! They're simple questions, but every introducing book on this subject seems to ignore these points completely, making me wonder if my questions are somehow unjust. PF to the rescue.

It's a matter of convenience-- why do you choose an axis to be "up" when you do a problem involving gravity? You can use any basis for the density matrix, but energy eigenstates are the most important because they are easy to evolve with time.

The point of statistical physics is that you are trying to count all the states that would be consistent with your measured energy bin, and you are going to assume all consistent states are equally likely, so the more states possible, the higher the probability of the system being in that energy bin. So it's not a claim about what the system is "really doing", the first step in any type of statistical physics is to admit that you don't know what the system is really doing, and you only want to know how many different things it might be doing that are consistent with the information you have. It's a lot like playing poker-- you know there is a "real deal" that you are playing, but none of the strategies of poker are based on the real deal because you don't know what the real deal is. The strategies are based on all the possible deals that are consistent with what you know. So within the energy resolution of your instrument, there are many dE bins of many states, and you are going to assume that dE intervals with more states in them are more likely, given the information you have.

How so? If your density matrix is just a mixed state where the pure states are energy eigenstates, then the probability of it being in a state like [itex]\frac{1}{\sqrt{2}} \left( \psi_{E_n} + \psi_{E_m} \right)[/itex] is equal to zero. In other words, the system can then onlybe in energy eigenstates. This is a physical assumption, not a mathematical one.

Oh so it's just a technical convention due to the fact that you often don't know exactly what energy it has? In other words, suppose I happen to know exactly what energy it has, then I can take [itex]\delta E=0[/itex]?

Because it is the (unique) state that corresponds to equilibrium. In equilibrium statistical mechanics this state is postulated, in non-equilibrium it is derived from the equations of motion.

Split your system into two subsystems, then each one can interchange energy with the other and in fact they do.

That's not true. The probability for beeing in a state |ϕ> is <P_{ϕ}>=tr{ϱ|ϕ><ϕ|} which in general does not vanish for the given state.

Let's consider the simpler example of a completely incoherent mixture of states of a spin-1/2 particle. Starting with the density matrix in the z-basis ϱ~|up_{z}><up_{z}|+|down_{z}><down_{z}| it is instructive to make a basis transformation into the x- and y-bases, which consist of states similar to the one you just mentioned. The populations there are the same as in the z-basis (for the completely incoherent case). If they were equal to zero, the trace of your new density matrix would be zero too.

Thanks for the answers, and apologies for my late reply:

Regarding my first question: I was indeed wrong in my statement about density matrices (my 2nd post); I didn't mean to talk about density matrices, Ken G sidetracked me on it: he started talking about density matrices while my question wasn't about them. juanrga seems to try to answer my original question, but I'm not sure I understand. Why are only the eigenfunctions in "equilibrium"? What definition of equilibrium is used? The definition of equilibrium in classical mechanics never excluded any points in phase space.
Maybe I can make my confusion more concrete: we saw [itex]Z = \sum \exp{- \beta E_n}[/itex] for the canonical ensemble. The canonical ensemble is defined as the system which was brought in contact with a reservoir at temperature T. The quoted formula for Z suggests that the probability of the such a system being in state [itex]\psi_1 + \psi_2[/itex] (not normalized) is zero. Why?

Regarding my second question: what if my system is isolated (which it can be in between measurements), then [itex]\delta E = 0[/itex]. What am I missing?

The density matrix is not a "sidetrack", it is the whole heart of your question. You can choose any basis you like to represent the matrix, including energy eigenstates, but the matrix will have the exact same physical meaning. A system in equilibrium is not in a bunch of energy eigenstates, but we can picture it that way without making an error-- this is the point of a mixed state, you can neglect the coherences that are present in a pure state. Each pure state is unique, each mixed state is not-- its representation depends on the basis you choose but is physically equivalent to a mixed state in another basis, much like the linear transformations represented by matrices are equivalent to other matrices in other bases.

Let me give you concrete example. Let's take a two-level atom in thermodynamic equilibrium at T. If the energy difference is E, then the mixed state in the energy eigenbasis corresponds to a probability ratio of e^{-E/kT} between the two eigenstates. This does not mean the equilibrium system is really in a bunch of energy eigenstates with that ratio, it just means it may as well be for all we care.

I also asked a concrete question in my previous, and it might be you've tried to answer it but that I didn't find the answer because I don't understand it well enough. But I'll restate it and I kindly ask you to try to follow my reasoning and try to pinpoint which step is wrong:
the canonical ensemble is given by [itex]Z = \sum_n \exp -\beta E_n[/itex]
this corresponds to ``the probability of the system being in state [itex]\psi[/itex] is [itex]\propto \exp - \beta E(\psi)[/itex]''
as only energies of the form E_n are allowed, i.e. eigenvalues of the Hamiltonian, the only psi's with a non-zero probability are eigenfunctions of the Hamiltonian

The state of the canonical ensemble is given by ρ = 1/Z exp(βH), where Z has to be tr{exp(βH)} to ensure that the trace of ρ is equal to one. Z doesn't directly correspond to probabilities, it's a normalization factor. The important quantity really is the density matrix.

If your system is isolated and you know your state at the initial time, you just have ordinary quantum mechanics and don't need statistical mechanics. (Classical) statistics only comes into play when there's uncertainty.

Hm maybe I should've said that in class we didn't saw the density matrix approach, just the [itex]Z = \sum_n \exp - \beta E_n[/itex] statement. Maybe I can't really understand it correctly without understanding the density matrix?

Oh so the [itex]\delta E[/itex] refers to an uncertainty in our knowledge of the energy of the system? Ohhh...

Yes, I think so. I would try to get a feel for the density matrix for spin-1/2 systems first (Sakurai is probably a good reference for this) and maybe do some explicit calculations.

There are some books and lecturers who don't use the density matrix formulation at all. That's ok for doing calculations but it's much harder to grasp the simple quantum mechanical foundations behind the formalism.

You can follow two ways: The first used in equilbrium statistical mechanics just postulates the state for a thermodynamic system at equilibrium. The justification of this postulate is done a posteriori by comparison with experimental data of the predictions done using that state.

The second way is the non-equilibrium one. You start from the irreversible equation of motion and apply it to a system at thermodynamic equilibrium, solving the equation you derive the state.

The equilibrium state is a function of the Hamiltonian because the Hamiltonian is the generator of time translations and verifies [H,ρ]=0 for equilibrium. Outside equilibrium, the state is not given by Hamiltonian alone but also by fluxes that drive the system outside equilibrium.

The classical state that corresponds to thermodynamic equilibrium is just the classical limit of the quantum state. A thermodynamic equilibrium cannot be represented by a point in phase space because equilibrium is the result of dissipation and dissipation breaks trajectories (if any initially).

No the uncertainty is related to the interchange of energy of the system with the heat bath. The energy of the system is not constant (it is not isolated) and varies due to quantum fluctuations, just as the number of molecules [itex]N[/itex] in an open room is not constant due to random entry and exit of molecules inside the room: [itex]\delta N[/itex] has nothing to see with our knowledge.