Below is a summary I put together some years ago regarding the cosmological constant problem (or the vacuum catastrophe)-
There are two methods to calculating the density of the vacuum required when calculating Lambda (the cosmological constant); by cosmology
(as discussed in this post) and by quantum field theory (QFT).
Lambda in Quantum Field Theory
One theory for dark energy is that quantum fluctuations lead to the appearance and disappearance of virtual pairs of particles which continuously pop into and then out of existence as the universe expands. They cannot be measured directly but they may produce a small amount of energy that has an affect on the overall density of universe and the curvature of space.
Zero Point Energy is another possible source for vacuum energy, a vibrational energy retained by molecules even at a temperature of absolute zero. Since temperature is a measure of the intensity of molecular motion, molecules would be expected to come to rest at absolute zero. However, if molecular motion were to cease altogether, the atoms would each have a precisely known location and velocity (zero), and the Heisenberg uncertainty principle states that this cannot occur, since precise values of both position and velocity of an object cannot be known simultaneously. Thus, even molecules at absolute zero must have some zero-point energy.
Lambda calculated by Quantum Field Theory
(quantum fluctuations of particles, zero point energy, zero point fluctuations)
'For particles of mass, m, we can expect one virtual particle in each cubical volume with sides given by the
Compton wavelength of the particle,
h/mc, where
h is the Planck's constant.' The density of the vacuum according to QFT is-
\text{Vacuum density}\ (\rho_\Lambda)=\frac{m}{(h/mc)^3}=\frac{m^4c^3}{h^3}
where
h is Planck's constant in Joules/s, c is the speed of light and m would be the Planck mass (m
P), the largest elementary particle mass 'usually considered'
(presumably providing a top end figure for the QFT vacuum density)
Based on QM, the vacuum density works out at-
\text{Vacuum density}\ (\rho_\Lambda)=\frac{(2.176\times10^{-8})^4\ \times\ (3\times10^8)^3}{(6.626\times10^{-34})^3}
= 2.077x10^94 kg/m^3 (!)
This density is the equivalent of the mass of the universe (2x10^53 kg) multiplied by 10^41, squeezed into a cubic metre! Therefore, something seems to be amiss with the QFT equation.
The vacuum density can also be expressed as vacuum energy (UΛ) by multiplying the density by c^2, which works out at 1.6x10^111 erg/cm^3. This is a staggering amount considering the centre of the sun is 2x10^17 erg/cm^3 and the erg figure for the cosmological observation is 0.6x10^-10 ergs/cm^3.
Based on this QM density for the vacuum, Lambda works out at-
\Lambda=\frac{8\pi G}{c^2}\ \times\ \text{Vacuum density}\ (\rho_\Lambda)= \frac{8\ \times\ 3.14159\ \times\ 6.6742\times10^{-11}}{9\times10^{16}}\ \times\ 2.077\times10^{94}
= 3.876x10^68 m^-2 in geometric units
Compare this to Lambda in geometric units calculated using cosmology, Λ = 1.252x10^-52 m^-2, and there is a difference to the order of 120. This large discrepancy between cosmology observations and QFT calculations is often referred to as the
cosmological constant problem and is currently one of the problems facing physicists today.
To account for this large difference, it is thought that there is some device in place where virtually all of this mass/energy is canceled out at quantum level, leaving the small amount we observe today that keeps our universe flat. One analogy would be when using long division to divide a large number; we generate a lot of intermediate numbers which we then discard once we have the answer. Is a lot of this energy temporary scaffolding used for calculations that vanish when the calculations are complete?
'The Quantum Vacuum and the Cosmological Constant Problem' by S.E. Rugh and H. Zinkernagel