DFT, Kohn-Sham: failure to describe transition states.

[Derivator]

Hi,

could someone explain the following quote to me, please? It explains, why the Kohn-Sham-Scheme of DFT fails to describe transition states.

Failure to describe transition states:
The total energies predicted for transition states are very unreliable. Both in this
case and in the case of strongly correlated systems the true wave-function has not
only one dominating determinant but several important determinants. This multi-
configuration character of the wave-function becomes visible in the occupation
numbers. Since the occupation number of a certain natural orbital is equal to the
sum of the squared coefficients of all the determinants that contain this natural or-
bital, there will be occupation numbers that are neither close to one or zero. If
there is only one dominating determinant, the occupation number of the occupied
natural orbitals contained in this determinant will be close to one whereas the occu-
pation numbers of all the other virtual orbitals are very small. That the Kohn-Sham
methods fail in this case is not so surprising.

My questions:

1. If $N_i$ ist the occupation number and the wavefunction may be given by $\Psi = \sum_I{c_I \Phi_I}$ where $\Phi_I$ are Slater-Determinants, why does from $N_i = \sum{C_I^2}$ follow that if many $C_I \neq 0$ there will be occupation numbers $N_i$ that are not close to 1 and not close to 0? (this question relates to the red colored text)
2. If there is only one $C_I$ that is not vanishing and all other coefficients are small, why will be the occupation number of the natural orbital that is contained in the Slater-Determinant belonging to the $C_I$ that is not vanishing close to 1? Why are all occupation numbers of the virtual orbitals very small? (this question relates to the blue colored text)

 

[SpectraCat]

Hi,

could someone explain the following quote to me, please? It explains, why the Kohn-Sham-Scheme of DFT fails to describe transition states.



My questions:

1. If $N_i$ ist the occupation number and the wavefunction may be given by $\Psi = \sum_I{c_I \Phi_I}$ where $\Phi_I$ are Slater-Determinants, why does from $N_i = \sum{C_I^2}$ follow that if many $C_I \neq 0$ there will be occupation numbers $N_i$ that are not close to 1 and not close to 0? (this question relates to the red colored text)
2. If there is only one $C_I$ that is not vanishing and all other coefficients are small, why will be the occupation number of the natural orbital that is contained in the Slater-Determinant belonging to the $C_I$ that is not vanishing close to 1? Why are all occupation numbers of the virtual orbitals very small? (this question relates to the blue colored text)

Think of it this way, the solution you get from the DFT calculation is a single Slater determinant that is an approximation for some lower-energy multi-configurational (MC) wavefunction. If it is a close approximation, then the natural orbitals (NO's) extracted from the single Slater determinant electron density from the calculation will be close approximations to the natural orbitals for the true MC wavefunction. As such, the occupied orbitals will have occupations numbers very close to 2 (or 1 for half-full orbitals), and the virtual orbitals will have occupation numbers very close to zero.

On the other hand, if the single Slater determinant electron density is a poor approximation to the true MC electron density, then the natural orbitals from the single Slater determinant answer will be linear combinations of the correct natural orbitals for the MC density. This is true because any set of natural orbitals forms a complete basis for the electron density. So, since the single-determinant NO's are linear combinations of the correct MC NO's (i.e. the ones which have occupation numbers of 0,1 or 2), the occupation numbers will be calculated from a mixture of filled and empty orbitals.

Note that the above description is a theoretical post-rationalization of why the non-integer occupation numbers arise. In actuality, they arise because the unrestricted single Slater-determinant is spin-contaminated, e.g. for a nominal doublet state, there will be contamination from quartet and sextet states. This results in qualitative differences between the alpha and beta spin-orbitals in the unrestricted wavefunction, and those in turn give rise to the partial occupation numbers for the NO's.

 

[Derivator]

Hi SpectraCat,

thank you, for your answers.

Could you please explain my 2nd question again? I still do not unserstand this one.

 

[SpectraCat]

Hi SpectraCat,

thank you, for your answers.

Could you please explain my 2nd question again? I still do not unserstand this one.

Well .. the whole point of natural orbitals is to find the orbital basis that diagonalizes the density matrix corresponding to the wavefunction. Thus the correct natural orbitals for a given system should have occupation numbers of 2 for fully occupied orbitals, 1 for half-occupied orbitals and 0 for virtual orbitals.

In the case where only one configuration contributes significantly to the full multi-configurational wavefunction, then the single Slater determinant answer from the DFT calculation will be a good approximation to the full multi-configurational answer. That therefore means that the natural orbitals constructed based on the electron density in the single determinant calculation should be good approximations to the correct natural orbitals for the full multi-configurational answer, and thus should have occupation numbers close to the ideal values.

 

[cgk]

I'd like to add that the original paragraph does not actually explain anything relating to the failure of KS-DFT for transition states. The whole text could be replaced by "KS-DFT does not work well for strongly correlated systems." (strongly correlated systems are by definition systems that need multiple determinants with comparable weight for a qualitative description). That a theory that is in practice some correction to Hartree-Fock does not deal well with such systems should indeed come as no surprise.

The more interesting point is that quite often transition states are not actually strongly correlated systems, and are described well by correlation methods which use Hartree-Fock as reference, but not by DFT. The deeper reason for this lies in the self-interaction error which all common DFT functionals have (and this is also why the error goes down with increasing HF exchange fraction in hybrid functionals). Google-Scholar for DFT+transition state+self-interaciton error if you want to know more about that.

 

[alxm]

I'm wondering about the context of this quote? It seems to be talking referring to a more specific set of circumstances: "Both in this case and in the case of strongly correlated systems". Is it, perhance, the perennial H₂-dissociation example?

Because I wouldn't say that it's generally true that KS-DFT fails to describe transition-states correctly (in particular their relative energy). If it were regarded as generally true that KS-DFT fails to describe transition-states accurately, it'd have to be the most disregarded fact in all of computational chemistry, since after all, there are papers being published every day with transition-states calculated using DFT. So, while they're certainly less accurate for transition-states compared to minima, they're far from being too unreliable to be useful for transition-states.

I don't think you can say DFT fails on transition states in general, or that transition-states generally require a multireference description. And many which you'd assume require such a description from the kind of rationale quoted here, such as H₂ dissociation, can in fact be described by a single determinant (in principle). For a discussion of this, see e.g. Koch & Holthausen's "A Chemists Guide to Density Functional Theory", sec 5.3.4 "Is the Kohn-Sham Approach a Single Determinant Method?". Where they also warn against these kinds of statements:

We cannot overemphasize how important it is not to let a sloppy way of describing these effects slip in. Again, it is not necessarily the single determinant based regular KS scheme itself which is to blame for the sometimes poor results of approximate Kohn-Sham density functional theory; responsible instead are the deficiencies of the approximations to the exchange-correlation functional.

 

[DrDu]

Remember that the Kohn Sham orbitals refer to a (hypothetical) non-interacting system whith the same density as the interacting system. However, the wavefunction for a non-interacting system can always be represented as a single Slater determinant. So basically it's all about inaccurate XC functionals.
DFT is a semi-empirical theory. This also means that there are always situations where a given functional will fail to give the correct answer.

 

[cgk]

Remember that the Kohn Sham orbitals refer to a (hypothetical) non-interacting system whith the same density as the interacting system. However, the wavefunction for a non-interacting system can always be represented as a single Slater determinant. So basically it's all about inaccurate XC functionals.

While this is true in theory, in practice I don't think that the "in principle DFT is exact" perspective is very helpful, especially not in the Kohn-Sham case. You are right that every density can be represented by a single Slater determinant: In fact, every density can be represented by an infinite number of very different, inequivalent Slater determinants with very different kinetic energies (e.g., see Harriman's original Phys Rev A 25 680 (1981) paper). And up to a global constant multiplier, even by a Slater determinant built for any given number of electrons. What this means is that there is much more significance to the KS determinant than that it reproduces the density. Obviously, it also needs to be sensible in some other way, otherwise KS would not work! And in KS this is effectively realized by creating some kind of theory mixture of Hartree-Fock and actual (orbital free) DFT, which makes sure that the kinetic energy is represented in some meaningful way.

While of course one can claim that "in principle we only need to find the correct xc functional to get everything right", one should also add that there are good reasons to believe that both in theory and in practice the only way of getting the functional is by explicit inversion of a large-basis Full CI calculation. (Note the n-representability problems buried in the xc functional are likely themselves as hard as full CI).