I Is the Cosmological Constant Problem a Misunderstanding of Zero-Point Energy?

[AndreasC]

Reading the Wikipedia page on it, one reads:

In cosmology, the cosmological constant problem or vacuum catastrophe is the disagreement between the observed values of vacuum energy density (the small value of the cosmological constant) and theoretical large value of zero-point energy suggested by quantum field theory.

But on the other hand, as far as I know and if I'm not mistaken, zero point energy is not a physical thing, and it is merely a mathematical artifact in QFT. Someone correct me if I'm wrong on that. So if that is the case, then why is it a "problem" that it does not "agree" with the cosmological constant? Isn't trying to plug it into general relativity to calculate the cosmological constant just a non-sensical idea to begin with? It certainly seems so to given that the calculations are off by I don't know how many orders of magnitude (I found a paper claiming that the typical figures of 120 orders of magnitude deviation are not correct, and it is instead "only" about 60. It doesn't seem that much better to me).

 

[PeroK]

The observed vacuum energy must be due to something. It ought to be calculable, not just some unjustifiable observed quantity.

 

[AndreasC]

The observed vacuum energy must be due to something. It ought to be calculable, not just some unjustifiable observed quantity.

But then the problem is that we don't know what it is caused by, not the fact that it does not agree with the calculation of vacuum energy, right? I don't see why that calculation is relevant to the problem if we do not expect vacuum energy to be physical and thus have anything to do with the cosmological constant to begin with... That is why I'm confused about why it comes up.

 

[PeroK]

But then the problem is that we don't know what it is caused by, not the fact that it does not agree with the calculation of vacuum energy, right? I don't see why that calculation is relevant to the problem if we do not expect vacuum energy to be physical and thus have anything to do with the cosmological constant to begin with... That is why I'm confused about why it comes up.

Okay, but don't you think it was worth trying? Just to see?

 

[AndreasC]

Okay, but don't you think it was worth trying? Just to see?

Well, I guess maybe but it seems like there is a lot of emphasis put on that still, and many people still seem to believe they are somehow connected... The Wikipedia article I quoted has some references which seem to imply there is still research claiming that, case in point the paper I mentioned: https://arxiv.org/abs/1205.3365

This paper (which I may be misunderstanding) also seems to talk about quantum vacuum gravitating: https://arxiv.org/abs/1703.00543

So this is why I'm confused.

 

[phyzguy]

Isn't trying to plug it into general relativity to calculate the cosmological constant just a non-sensical idea to begin with?

Yes!

 

[AndreasC]

Yes!

Well then the question becomes, why does this keep getting brought up? Why do so many physicists seemingly treat these as being related?

 

[phyzguy]

Well then the question becomes, why does this keep getting brought up? Why do so many physicists seemingly treat these as being related?

Well, since the source of the cosmological constant isn't known, people keep trying to understand. As @PeroK said, it was worth a try. Maybe it's on the right track and we just dropped a factor of 10^120 somewhere. You might be interested in this paper, "

Why all these prejudices against a constant?"​

https://arxiv.org/abs/1002.3966v3

 

[PeterDonis]

zero point energy is not a physical thing, and it is merely a mathematical artifact in QFT

That depends on which physicist you talk to. There's no way to prove or disprove such statements theoretically; ultimately any such theoretical claim has to be tested against observations. Our best current observations indicate that our universe has a small positive cosmological constant. GR itself allows for a nonzero cosmological constant but makes no prediction at all about what its value should be; so the only theoretical framework we have at present for even investigating the question is quantum mechanics.

 

[PeterDonis]

trying to plug it into general relativity to calculate the cosmological constant

I'm not sure what you mean by this. You don't need to "plug" anything into GR to try to estimate from QM what the energy density of the vacuum should be. And the fact that a nonzero cosmological constant is identical in the Einstein Field Equation to a nonzero energy density of the vacuum is obvious.

Isn't trying to plug it into general relativity to calculate the cosmological constant just a non-sensical idea to begin with?

Yes!

Why? It's not nonsense that the cosmological constant can be nonzero in GR. It's not nonsense that a nonzero cosmological constant is the same, from the standpoint of the Einstein Field Equation, as a nonzero energy density of the vacuum. And it's not nonsense that QM can be used to at least try to estimate the energy density of the vacuum. So what, exactly, are you saying is nonsense here?

 

[phyzguy]

So are you saying your chain of:
Nonzero cosmological constant -> Nonzero vacuum energy density -> QM vacuum fluctuations
makes "sense"? If so, why is it off by 120 orders of magnitude? It seems to me that any reasonable person would discard it as "nonsense" when the agreement is so bad.

 

[PeterDonis]

So are you saying your chain of:
Nonzero cosmological constant -> Nonzero vacuum energy density -> QM vacuum fluctuations
makes "sense"?

I said nothing about "QM vacuum fluctuations".

why is it off by 120 orders of magnitude?

We don't know. But we also don't know how to do the calculation exactly anyway; all of the numbers quoted are approximations, invoking things like ad hoc cutoffs. Obviously we don't have a full understanding of the underlying theory. But that doesn't mean the idea is nonsense.

 

[PeterDonis]

It seems to me that any reasonable person would discard it as "nonsense" when the agreement is so bad.

It seems to me that any reasonable person would want to look at what other models we have before discarding the QM model as "nonsense". So what other models do we have to explain a nonzero cosmological constant?

 

[Vanadium 50]

Is the zero-point energy in the simple harmonic oscillator "real" or "not a physiacl thing"? Before discussing grand and grandiose things like The Entire Universe, shouldn't we get this straight? At a minimum, it tells us that the issue is present in much simpler systems.

Next, there is absolutely no calculation of the cosmological constant from QM. Nada. Zilch. Zip. What we have is an order of magnitude estimate and some plausible numbers as inputs to this estimate, but this is not a calculation. "We don't have a calculation, but if we did, I'm sure it would be too big" is a very different kind of "problem" than "these numbers should be equal and aren't" (or "these numbers have no reason to be equal but are")

 

[TeethWhitener]

Weinberg’s 1988 Rev Mod Phys paper is relevant here and gives a really good (IMO) exposition of the issue:
https://repositories.lib.utexas.edu/bitstream/handle/2152/61094/Weinberg_1989.pdf;jsessionid=F306B9DD843B1834614FC774EC08CBE9?sequence=1
I’m not an expert, so I don’t know what kinds of advances have been made in the past 30 odd years since this paper was written.

 

[strangerep]

It seems to me that any reasonable person would want to look at what other models we have before discarding the QM model as "nonsense".

The so-called dark energy associated with the cosmological constant also has a rather unusual equation of state (EoS) -- very different from all known matter and radiation fields. Hence, it seems unlikely that the VeV of these known fields would have such a different EoS. The "problem" is not just the size of estimated vacuum energy, but also the weird EoS required.

So what other models do we have to explain a nonzero cosmological constant?

Umm, a universal constant?

 

[PeterDonis]

The so-called dark energy associated with the cosmological constant also has a rather unusual equation of state (EoS) -- very different from all known matter and radiation fields.

This equation of state is also the only EoS that is consistent with a nonzero energy density of the vacuum, since any such energy density must be Lorentz invariant. That's why a nonzero energy density of the vacuum is a viable possible explanation for dark energy.

a universal constant?

That isn't an explanation, it's just a bare assertion. It is, of course, possible that our universe has a nonzero cosmological constant simply because it does, with no deeper explanation possible. But we are in no position to just assert this when there is so much we still don't know about other possibilities.

 

[PeterDonis]

it seems unlikely that the VeV of these known fields would have such a different EoS.

Actually, the VeV of a scalar field, like the Higgs field, has a similar EoS, correct?

 

[strangerep]

Actually, the VeV of a scalar field, like the Higgs field, has a similar EoS, correct?

Reference please?
(Maybe I've been away from particle physics for too long.)

 

[PeterDonis]

Reference please?

The general EoS of a scalar field is $\rho = K + V$, $p = K - V$, where $K = \left( \partial \phi \right)^2$ is the kinetic energy and $V$ is the potential energy. For a constant scalar field, which will be the case if the field is in its vacuum state with a nonzero VEV, $K = 0$ and we have $p = - \rho$, which is the vacuum energy EoS. (This is the same general argument that is invoked for the case of slow roll inflation.) See, for example, here:

https://supernova.lbl.gov/~evlinder/umass/sca.html

 

[AndreasC]

And the fact that a nonzero cosmological constant is identical in the Einstein Field Equation to a nonzero energy density of the vacuum is obvious.

That is what I mean. If the energy of the quantum vacuum or zero point energy or fluctuations or whatever is considered to not be physical, why do people try to make this calculation? Unless it is physical which doesn't seem to be at all universally accepted.

 

[AndreasC]

Is the zero-point energy in the simple harmonic oscillator "real" or "not a physiacl thing"? Before discussing grand and grandiose things like The Entire Universe, shouldn't we get this straight? At a minimum, it tells us that the issue is present in much simpler systems.

I'm not at home so I can't check it right now but I remember doing the quantization of the free electromagnetic field via assigning operators to momenta and positions in the classical theory in a slightly different way such that the weird infinite zero point energy of the field disappears. I don't remember what I did exactly, I think I wrote p^2+x^2 as a difference of the squares of p and ix (which is a valid and entirely equivalent way to express the problem classically obviously) and then replaced these with the operators, and used the fact they do not commute (unlike the classical case) to get a slightly different Hamiltonian that does not have that problem. I think the same happens if you write it as (p+x)^2-2px and do the same thing. If it can be removed like this, it seems to me like it's probably not really a physical thing. Unless there is somewhere else where it crops up that I don't know about. But isn't the zero point energy in general and the quantum vacuum energy different things anyways?

 

[haushofer]

Maybe it has been mentioned, but apart from trying the naive hunch this vacuum energy calculation has the peculiar property that it' a UV-divergence with a long distance (IR) effect. So it defeats Wilson's "divide and conquer energy scales"-philosophy in effective field theories. That's an interesting lesson on its own.

 

[AndreasC]

To expand on what I was trying to say before about the harmonic oscillator, let's try to quantize the simple harmonic oscillator. The classical hamiltonian is (setting all constants to 1) $H=p^2/2+x^2/2$. If you try to quantize it now by replacing p and x with the associated quantum operators, and you then rewrite them using the ladder operators, you get:
$$H= [i(a^\dagger-a)/\sqrt{2}]^2/2+[(a^\dagger+a)\sqrt{2}]^2/2 = a^\dagger a - [a^\dagger, a]/2 = a^\dagger a + 1/2 $$
This can be written using the number operator:
$$ H = N+1/2 $$
So there appears to be a "zero point energy", this 1/2 term. But if instead we wrote the classical hamiltonian as $H=(p-ix)(p+ix)/2$, which seems every bit as valid to me, and then you quantize, it is obvious that this is just: $$H=a^\dagger a = N$$ Thus the zero point energy goes away. This procedure can be used to remove the infinity in the quantization of the free EM field, and if I recall correctly these annoying zero point energy terms also appear in statistical physics calculations where they cause issues, and they can be removed as such. So regarding the question of whether the simple harmonic oscillator zero point energy is physical, I think the answer is probably not, unless I'm missing something.

 

[AndreasC]

Maybe it has been mentioned, but apart from trying the naive hunch this vacuum energy calculation has the peculiar property that it' a UV-divergence with a long distance (IR) effect. So it defeats Wilson's "divide and conquer energy scales"-philosophy in effective field theories. That's an interesting lesson on its own.

Hmm, do you have a reference that talks more about that? Otherwise, can you expand on it a bit?

 

[gggnano]

Then what causes the casimir pressure, if you do the math on abysmal distance of a distance a little over the Planck scale the pressure is literally higher than the pressure inside a neutron star! I'm not saying the Casimir formula is ideal though. Yet: what causes spontaneous emission or Lamb shift? It's like saying that gravity is artifact because it's many times weaker than the other 3 forces.
Ruling out quantum fluctuations, ex nihil, will also rule out another interesting idea: the Bolzmann brain, which is unfalsifiable if you think about it but still interesting thought experiment.

 

[PeterDonis]

If the energy of the quantum vacuum or zero point energy or fluctuations or whatever is considered to not be physical, why do people try to make this calculation?

Who said the people trying to make the calculation did not consider the energy density of the vacuum to be physical?

Unless it is physical which doesn't seem to be at all universally accepted.

That's not at all the same as it not being accepted at all. It should be obvious that there are some physicists who think it is physical and some who think it is not, and since we don't have a good underlying theory that makes a prediction about its value that matches what we observe, this area of research is still open.

 

[PeterDonis]

if instead we wrote the classical hamiltonian as $H=(p-ix)(p+ix)/2$, which seems every bit as valid to me, and then you quantize, it is obvious that this is just: $$H=a^\dagger a = N$$

No, it isn't, because $p$ and $x$ don't commute. So you have

$$
H = \left( p^2 - i x p + i p x + x^2 \right) / 2 = \left( p^2 + x^2 + i [p, x] \right) / 2 = N + \frac{1}{2}
$$

 

[AndreasC]

Then what causes the casimir pressure

There are a lot of papers claiming zero point energy is not required to explain that. I actually found tons of discussion in this forum on the subject, just search for Casimir effect physicsforums and you will see. About Lamb shift etc I don't know much.

 

[AndreasC]

No, it isn't, because $p$ and $x$ don't commute. So you have

$$
H = \left( p^2 - i x p + i p x + x^2 \right) / 2 = \left( p^2 + x^2 + i [p, x] \right) / 2 = N + \frac{1}{2}
$$

Check your calculation again, the ladder operators are p-ix and p+ix, both divided by the square root of 2. The number operator N is their product. It is exactly because they don't commute that this result is different from the same result with just p^2 and x^2.

To be more clear:
$$ H = (p-ix)(p+ix)/2 = \frac{(p-ix)}{\sqrt{2}} \frac{(p+ix)}{\sqrt{2}} = a^\dagger a = N $$