Why the DMET total energy is not the limit of EWF for very large eta?

[efertitta]

Hello,
I have started playing with the examples provided to test the improvement given by EWF over DMET and I was (perhaps naively) expecting DMET and EWF to give the same answer for the total energy if a very large eta is used or if bathtype='dmet' is called upon (given the same fragmentation scheme). I can see that if I do that, the same active space is indeed constructed (the DMET orbitals) and the solver (I tried both FCI and CCSD) does give the same energy for the fragment calculations whether you use EWF(bathtype='dmet') or DMET, but the total energy is very different.
I assume this is due to a different expression being used for the total energy, so when using CCSD, I played with e_tot and get_dm_energy but this does not seem to change things much. I am aware you have done some work on proper calculations of expectation values for the whole system. Is this perhaps implemented in EWF only? What am I missing something here?

Many thanks

[ghb24]

Hi Edoardo,
So yes, as you suspected, this is due to different total energy expressions that are default in the DMET and EWF approaches, though the fragments and associated bath spaces will be identical in the two calculations. These differences are discussed in more detail in the (admittedly lengthy) https://arxiv.org/pdf/2210.14561.pdf. There are also example Vayesta inputs for the different approaches in the SI of that paper, showing how to get the different energies.

If you want the energies from DMET and EWF to agree, then you have to do two things, a) make sure that the same self-consistency is being performed in both, b) make sure the same energy functional is being used. By default, the EWF branch doesn't have any self-consistency. DMET on the other hand requires at least a chemical potential optimization to ensure the right number of electrons (EWF doesn't, since it has the right number of electrons by construction). You can ask an EWF calculation to optimize a (global) chemical potential by calling emb.optimize_chempot() on the embedding object before running the kernel. Alternatively, if you know the number of electrons in each fragment (rather than just the global number of electrons), then you can add nelectron_target as an optional argument when setting up the fragment, and it will before a self-consistency to ensure the correct number (see e.g. examples/other/52-hubbard-2D.py and molecules/15-chemical-potential.py where both options are shown). The DMET code will also generally perform a full self-consistency with full fragment correlation potential optimization, so this may only agree with EWF for the first iteration...

The next thing, is to make sure that the right energy functional is used in EWF for consistency with DMET. This can be found in an EWF embedding object via emb.get_dmet_energy(part_cumulant=False), which will construct the energy functional from the partitioned RDMs. Note that setting part_cumulant=True results in an almost universally better energy functional, but is not the 'standard' one used in the DMET literature to date.

[efertitta]

Hi George,

Thank you so much for your quick reply and very comprehensive explanation! This explains a lot!
I had indeed run a one-shot DMET (maxiter=1) for the comparison, but I did not think of the chemical potential being updated

A follow-up question: how is the right number of electrons ensured in EWF by construction without requiring a chemical potential? In the limit I am looking at the orbital space is the same, so is it again backed into the the total energy expression? (I will look into the paper for details, but just to understand if I am on the right path)

Also, when you say that part_cumulant=True is a "better" energy functional, this does not necessarily mean that it delivers a lower energy, but instead that it is systematically improvable, right? In the example I ran I got 70mHa more correlation energy from DMET than EWF with DMET bath orbitals

[ghb24]

So I consider DMET to be a density matrix embedding, while EWF is more of a wave function embedding. By applying projectors to wave function amplitudes directly (on the excitation space), rather than projecting the density matrices, the EWF approach always (implicitly) defines a wavefunction, which by construction has the right number of electrons. The energy expression is then built on this wave function, and therefore must have the right number of electrons. When you optimize the number of electrons via a chemical potential in DMET, you are first of all having to define how you compute the number of electrons. This is via the 'democratic partitioning' of density matrices, which is a choice, and is not a number of electrons which is determined from any (global) wave function, but rather by combining the number of electrons from each cluster solution. EWF instead implicitly defines the exact number of electrons globally by construction.

Re. the part_cumulant option, this is an option within the DMET framework (i.e. with democratically partitioned density matrices), but just involves a slightly different partitioning of the 2-body energy contribution (again, see the reference above for more details). It is not variational, so lower doesn't necessarily mean better, but neither is the standard EWF or DMET expressions. Beyond Hubbard models (and perhaps minimal basis Hydrogen systems) however, we have found that it generally produces better results to partition the cumulant rather than 2RDM (corroborated independently in the recent work here, which also uses this approach: https://arxiv.org/abs/2301.06153)

[efertitta]

Thank you George, very clear!