DFT Hybrid Functionals: Inclusion of HF Exact Exchange Energies

[hiltac]

Hello !

How does the inclusion of a fraction of the HF "exact exchange energies" in an hybrid functional (B3LYP for example) can degrade the quality of the results ? I mean that the results are worst than expected because of the HF exchange, but if we call it "exact" how is it possible ?

Thank you for your help !

 

[hiltac]

anyone ?

 

[TeethWhitener]

Becke's paper, "A new mixing of Hartree-Fock and local density-functional theories," gives a pretty good motivation for hybrid functionals. It's unclear from your post how much you already understand about HF and DFT, so let me know if I assume too much.

The rationale for using DFT instead of HF is that HF doesn't explicitly include certain electron correlation effects. There is some correlation already present in HF in the form of the exchange integral, which enforces the Pauli principle (so that electrons with like spins avoid each other). However, the coulomb repulsion between electrons in HF theory is treated in a mean-field way: in effect, each electron sees the average electric field of all the other electrons, instead of the field from each individual electron. This is what we mean when we say that HF doesn't include electron correlation (Technical note: this is sometimes referred to as 'dynamic correlation.' There is another type of correlation called 'static correlation' which refers to situations in which the ground state of the molecule is not well-described by a single Slater determinant. This isn't particularly relevant to the present discussion). Most post-HF methods that do include correlation (Moller-Plesset, configuration interaction, coupled cluster, etc.) are quite computationally expensive, and DFT provides a method which includes correlation explicitly while being somewhat more computationally tractable than other post-HF methods.

As it turns out, the exchange-correlation energy is dominated by the "exchange" portion, with the "correlation" portion being a small correction. This suggests splitting the exchange-correlation energy into two parts:
$$E_{XC} \approx E_X + E_C$$ Typically, some approximation from an electron gas (such as LDA or GGA) is used to give functionals representing exchange $E_X$ and correlation $E_C$. You can, however, use the HF exchange integral for $E_X$ and, e.g., the LDA expression for $E_C$. In the paper mentioned above, Becke notes that this technique improves slightly upon HF, but it's nowhere near chemical accuracy. The reason is because the above equation is actually a terrible approximation and the exchange and correlation parts of the energy can't really be that cleanly separated. In addition, the exchange portion of $E_{XC}$ is not the same for HF as it is for something like LDA or GGA. The approximation gets somewhat better if you mix the overall exchange-correlation from DFT $E^{DFT}_{XC}$ with the pure exchange from HF $E^{HF}_{X}$ in some linear combination. For example, the PBE0 functional has the form:
$$E^{PBE0}_{XC} = \frac{1}{4} E^{HF}_X + \frac{3}{4} E^{PBE}_X + E^{PBE}_C$$ You might ask: If HF is so crappy, why not just use the LDA exchange-correlation functional? Why even drag HF exchange into it in the first place? In fact, the HF exchange does add something to DFT. In general, exchange-correlation energies are nonlocal, but most DFT functionals assume some sort of local approximation (LDA=local density approximation). This causes local DFT to underestimate the true exchange-correlation energy. Including HF exchange as a nonlocal correction improves results for calculations like bond energies and ionization potentials, among other things.

 

[hiltac]

Thank you for that explanation !
So, if results (using B3LYP) are worst because of HF exchange it's due to the fact that take E_XC = E_X + E_C is an approximation ? And it's for the same reason that we cannot only take E_X from HF exchange for hybryd functionnal ?

Do you know why we get best results if we optimize geometry at HF level first and then with DFT ? Is it because we take into account the electron correlation with DFT an not with HF and that we need the HF exchange (E_X) for the hybrid functionnal ? What is we optimize only at DFT level ?

Thank you

 

[TeethWhitener]

So, if results (using B3LYP) are worst because of HF exchange it's due to the fact that take EXC = EX + EC is an approximation ? And it's for the same reason that we cannot only take EX from HF exchange for hybryd functionnal ?

$E_{XC} = E_X + E_C$ is a standard approximation in DFT. It's only when we try to use the exchange integral from HF without mixing in any DFT exchange that things start to go haywire. The approximation isn't that bad, but it's certainly not good enough for thermochemical calculations/reaction energies.

Do you know why we get best results if we optimize geometry at HF level first and then with DFT ? Is it because we take into account the electron correlation with DFT an not with HF and that we need the HF exchange (EX) for the hybrid functionnal ? What is we optimize only at DFT level ?

As long as you're reasonably close to a minimum on the potential energy surface, optimizing with HF and then with DFT won't be any different than only optimizing with DFT. Usually, people optimize at a low level of theory first because the low level calculation is quicker. In a sense, doing HF gets you "close enough" without having to take the extra time that DFT or other post-HF methods require. Once you're "close enough," then you can go back with a higher level of theory and refine the structure.

 

[hiltac]

I thought that solving a problem with hybrid functional was quicker than with HF ?

 

[TeethWhitener]

Sorry, I misspoke in my last post. In principle, pure DFT is quicker than HF (computation time scales as N³ for DFT vs. N⁴ for HF, where N is the number of basis functions used in the calculation). However, the slow part of HF is the computation of the exchange integrals, so adding HF exchange into DFT to make a hybrid functional essentially makes HF and hybrid HF/DFT run at roughly the same speeds (to within a constant scaling factor). But, hybrid HF/DFT generally gives more accurate results than either than pure DFT or HF, so there's really no reason not to start with the hybrid functional.

As for optimizing at HF level before DFT, I've never encountered a situation where this made any meaningful impact on the final result. Maybe (and this is what I meant to say in my earlier post) you're thinking of a situation where you pre-optimize using HF and a small basis set? It's common to use a low level of theory to get close to a final geometry so you're not wasting $O(N^7)$ computational cycles by starting somewhere far away from a minimum and cranking away at it with some giant basis set at a high level of theory.

 

[Einstein Mcfly]

There are times when certain DFT software packages provide a very poor initial guess on the density, and so running a single point HF calculations first to get a set of orbitals that is reasonable is sometimes recommended. That is an instance where running a HF calculation first is a good idea.

I can't think of another, and would like to know what the initial questioner means about how using a hybrid functional "degrades the results". DFT is an approximate method meant to solve a system that is analogous to a real system (that is, they have the same density) but is NOT even in principle meant to solve the Schrodinger equation. There is quite a bit of wiggle room in how you choose your XC functional that can either give you excellent results or very poor results, partially because of things like fortuitous cancellations of errors (exchange energy is a bit too high for a given system, but correlation is a bit low and so the total energy is pretty good etc).

When using wave function methods, you can attempt to approach a more complete solution to the problem by increasing your level of theory (multi-reference methods). You can't do this in DFT and the ONLY way that you can rightly claim to have a DFT model that gives reliable results is to benchmark your calculations with experimental values. If you're looking at thermochemical properties of some set of molecules, you should find experimental values for thermochemical properties of similar molecules and choose an XC functional that gives results closest to those values. Most of the time, that will likely mean using hybrid functionals at least, and sometimes even more exotic functionals which go beyond GGA as well. There are many papers out there who simply using a particular functional because it's "well known" or because they found a paper where someone had used it for a similar system without caring if it was ever bench marked. When I review papers of this sort I always demand that bench marking data is included in a subsequent draft, otherwise the results may be meaningless. There is enough wiggle room in DFT to drive a truck through and if people aren't careful they'll waste their (and everyone else's) time.

 

[hiltac]

There is something I don't get...
If we have to use exchange energy from HF in hybrid functional, how is it possible to be quicker than only HF? We have to solve HF calculation and then kohn sham, right ?
Thank you for all these precision