I Is the Cosmological Constant Problem a Misunderstanding of Zero-Point Energy?

[AndreasC]

Reading the Wikipedia page on it, one reads:

In cosmology, the cosmological constant problem or vacuum catastrophe is the disagreement between the observed values of vacuum energy density (the small value of the cosmological constant) and theoretical large value of zero-point energy suggested by quantum field theory.

But on the other hand, as far as I know and if I'm not mistaken, zero point energy is not a physical thing, and it is merely a mathematical artifact in QFT. Someone correct me if I'm wrong on that. So if that is the case, then why is it a "problem" that it does not "agree" with the cosmological constant? Isn't trying to plug it into general relativity to calculate the cosmological constant just a non-sensical idea to begin with? It certainly seems so to given that the calculations are off by I don't know how many orders of magnitude (I found a paper claiming that the typical figures of 120 orders of magnitude deviation are not correct, and it is instead "only" about 60. It doesn't seem that much better to me).

 

[PeroK]

The observed vacuum energy must be due to something. It ought to be calculable, not just some unjustifiable observed quantity.

 

[AndreasC]

The observed vacuum energy must be due to something. It ought to be calculable, not just some unjustifiable observed quantity.

But then the problem is that we don't know what it is caused by, not the fact that it does not agree with the calculation of vacuum energy, right? I don't see why that calculation is relevant to the problem if we do not expect vacuum energy to be physical and thus have anything to do with the cosmological constant to begin with... That is why I'm confused about why it comes up.

 

[PeroK]

But then the problem is that we don't know what it is caused by, not the fact that it does not agree with the calculation of vacuum energy, right? I don't see why that calculation is relevant to the problem if we do not expect vacuum energy to be physical and thus have anything to do with the cosmological constant to begin with... That is why I'm confused about why it comes up.

Okay, but don't you think it was worth trying? Just to see?

 

[AndreasC]

Okay, but don't you think it was worth trying? Just to see?

Well, I guess maybe but it seems like there is a lot of emphasis put on that still, and many people still seem to believe they are somehow connected... The Wikipedia article I quoted has some references which seem to imply there is still research claiming that, case in point the paper I mentioned: https://arxiv.org/abs/1205.3365

This paper (which I may be misunderstanding) also seems to talk about quantum vacuum gravitating: https://arxiv.org/abs/1703.00543

So this is why I'm confused.

 

[phyzguy]

Isn't trying to plug it into general relativity to calculate the cosmological constant just a non-sensical idea to begin with?

Yes!

 

[AndreasC]

Yes!

Well then the question becomes, why does this keep getting brought up? Why do so many physicists seemingly treat these as being related?

 

[phyzguy]

Well then the question becomes, why does this keep getting brought up? Why do so many physicists seemingly treat these as being related?

Well, since the source of the cosmological constant isn't known, people keep trying to understand. As @PeroK said, it was worth a try. Maybe it's on the right track and we just dropped a factor of 10^120 somewhere. You might be interested in this paper, "

Why all these prejudices against a constant?"​

https://arxiv.org/abs/1002.3966v3

 

[PeterDonis]

zero point energy is not a physical thing, and it is merely a mathematical artifact in QFT

That depends on which physicist you talk to. There's no way to prove or disprove such statements theoretically; ultimately any such theoretical claim has to be tested against observations. Our best current observations indicate that our universe has a small positive cosmological constant. GR itself allows for a nonzero cosmological constant but makes no prediction at all about what its value should be; so the only theoretical framework we have at present for even investigating the question is quantum mechanics.

 

[PeterDonis]

trying to plug it into general relativity to calculate the cosmological constant

I'm not sure what you mean by this. You don't need to "plug" anything into GR to try to estimate from QM what the energy density of the vacuum should be. And the fact that a nonzero cosmological constant is identical in the Einstein Field Equation to a nonzero energy density of the vacuum is obvious.

Isn't trying to plug it into general relativity to calculate the cosmological constant just a non-sensical idea to begin with?

Yes!

Why? It's not nonsense that the cosmological constant can be nonzero in GR. It's not nonsense that a nonzero cosmological constant is the same, from the standpoint of the Einstein Field Equation, as a nonzero energy density of the vacuum. And it's not nonsense that QM can be used to at least try to estimate the energy density of the vacuum. So what, exactly, are you saying is nonsense here?

 

[phyzguy]

So are you saying your chain of:
Nonzero cosmological constant -> Nonzero vacuum energy density -> QM vacuum fluctuations
makes "sense"? If so, why is it off by 120 orders of magnitude? It seems to me that any reasonable person would discard it as "nonsense" when the agreement is so bad.

 

[PeterDonis]

So are you saying your chain of:
Nonzero cosmological constant -> Nonzero vacuum energy density -> QM vacuum fluctuations
makes "sense"?

I said nothing about "QM vacuum fluctuations".

why is it off by 120 orders of magnitude?

We don't know. But we also don't know how to do the calculation exactly anyway; all of the numbers quoted are approximations, invoking things like ad hoc cutoffs. Obviously we don't have a full understanding of the underlying theory. But that doesn't mean the idea is nonsense.

 

[PeterDonis]

It seems to me that any reasonable person would discard it as "nonsense" when the agreement is so bad.

It seems to me that any reasonable person would want to look at what other models we have before discarding the QM model as "nonsense". So what other models do we have to explain a nonzero cosmological constant?

 

[Vanadium 50]

Is the zero-point energy in the simple harmonic oscillator "real" or "not a physiacl thing"? Before discussing grand and grandiose things like The Entire Universe, shouldn't we get this straight? At a minimum, it tells us that the issue is present in much simpler systems.

Next, there is absolutely no calculation of the cosmological constant from QM. Nada. Zilch. Zip. What we have is an order of magnitude estimate and some plausible numbers as inputs to this estimate, but this is not a calculation. "We don't have a calculation, but if we did, I'm sure it would be too big" is a very different kind of "problem" than "these numbers should be equal and aren't" (or "these numbers have no reason to be equal but are")

 

[TeethWhitener]

Weinberg’s 1988 Rev Mod Phys paper is relevant here and gives a really good (IMO) exposition of the issue:
https://repositories.lib.utexas.edu/bitstream/handle/2152/61094/Weinberg_1989.pdf;jsessionid=F306B9DD843B1834614FC774EC08CBE9?sequence=1
I’m not an expert, so I don’t know what kinds of advances have been made in the past 30 odd years since this paper was written.

 

[strangerep]

It seems to me that any reasonable person would want to look at what other models we have before discarding the QM model as "nonsense".

The so-called dark energy associated with the cosmological constant also has a rather unusual equation of state (EoS) -- very different from all known matter and radiation fields. Hence, it seems unlikely that the VeV of these known fields would have such a different EoS. The "problem" is not just the size of estimated vacuum energy, but also the weird EoS required.

So what other models do we have to explain a nonzero cosmological constant?

Umm, a universal constant?

 

[PeterDonis]

The so-called dark energy associated with the cosmological constant also has a rather unusual equation of state (EoS) -- very different from all known matter and radiation fields.

This equation of state is also the only EoS that is consistent with a nonzero energy density of the vacuum, since any such energy density must be Lorentz invariant. That's why a nonzero energy density of the vacuum is a viable possible explanation for dark energy.

a universal constant?

That isn't an explanation, it's just a bare assertion. It is, of course, possible that our universe has a nonzero cosmological constant simply because it does, with no deeper explanation possible. But we are in no position to just assert this when there is so much we still don't know about other possibilities.

 

[PeterDonis]

it seems unlikely that the VeV of these known fields would have such a different EoS.

Actually, the VeV of a scalar field, like the Higgs field, has a similar EoS, correct?

 

[strangerep]

Actually, the VeV of a scalar field, like the Higgs field, has a similar EoS, correct?

Reference please?
(Maybe I've been away from particle physics for too long.)

 

[PeterDonis]

Reference please?

The general EoS of a scalar field is $\rho = K + V$, $p = K - V$, where $K = \left( \partial \phi \right)^2$ is the kinetic energy and $V$ is the potential energy. For a constant scalar field, which will be the case if the field is in its vacuum state with a nonzero VEV, $K = 0$ and we have $p = - \rho$, which is the vacuum energy EoS. (This is the same general argument that is invoked for the case of slow roll inflation.) See, for example, here:

https://supernova.lbl.gov/~evlinder/umass/sca.html

 

[AndreasC]

And the fact that a nonzero cosmological constant is identical in the Einstein Field Equation to a nonzero energy density of the vacuum is obvious.

That is what I mean. If the energy of the quantum vacuum or zero point energy or fluctuations or whatever is considered to not be physical, why do people try to make this calculation? Unless it is physical which doesn't seem to be at all universally accepted.

 

[AndreasC]

Is the zero-point energy in the simple harmonic oscillator "real" or "not a physiacl thing"? Before discussing grand and grandiose things like The Entire Universe, shouldn't we get this straight? At a minimum, it tells us that the issue is present in much simpler systems.

I'm not at home so I can't check it right now but I remember doing the quantization of the free electromagnetic field via assigning operators to momenta and positions in the classical theory in a slightly different way such that the weird infinite zero point energy of the field disappears. I don't remember what I did exactly, I think I wrote p^2+x^2 as a difference of the squares of p and ix (which is a valid and entirely equivalent way to express the problem classically obviously) and then replaced these with the operators, and used the fact they do not commute (unlike the classical case) to get a slightly different Hamiltonian that does not have that problem. I think the same happens if you write it as (p+x)^2-2px and do the same thing. If it can be removed like this, it seems to me like it's probably not really a physical thing. Unless there is somewhere else where it crops up that I don't know about. But isn't the zero point energy in general and the quantum vacuum energy different things anyways?

 

[haushofer]

Maybe it has been mentioned, but apart from trying the naive hunch this vacuum energy calculation has the peculiar property that it' a UV-divergence with a long distance (IR) effect. So it defeats Wilson's "divide and conquer energy scales"-philosophy in effective field theories. That's an interesting lesson on its own.

 

[AndreasC]

To expand on what I was trying to say before about the harmonic oscillator, let's try to quantize the simple harmonic oscillator. The classical hamiltonian is (setting all constants to 1) $H=p^2/2+x^2/2$. If you try to quantize it now by replacing p and x with the associated quantum operators, and you then rewrite them using the ladder operators, you get:
$$H= [i(a^\dagger-a)/\sqrt{2}]^2/2+[(a^\dagger+a)\sqrt{2}]^2/2 = a^\dagger a - [a^\dagger, a]/2 = a^\dagger a + 1/2 $$
This can be written using the number operator:
$$ H = N+1/2 $$
So there appears to be a "zero point energy", this 1/2 term. But if instead we wrote the classical hamiltonian as $H=(p-ix)(p+ix)/2$, which seems every bit as valid to me, and then you quantize, it is obvious that this is just: $$H=a^\dagger a = N$$ Thus the zero point energy goes away. This procedure can be used to remove the infinity in the quantization of the free EM field, and if I recall correctly these annoying zero point energy terms also appear in statistical physics calculations where they cause issues, and they can be removed as such. So regarding the question of whether the simple harmonic oscillator zero point energy is physical, I think the answer is probably not, unless I'm missing something.

 

[AndreasC]

Maybe it has been mentioned, but apart from trying the naive hunch this vacuum energy calculation has the peculiar property that it' a UV-divergence with a long distance (IR) effect. So it defeats Wilson's "divide and conquer energy scales"-philosophy in effective field theories. That's an interesting lesson on its own.

Hmm, do you have a reference that talks more about that? Otherwise, can you expand on it a bit?

 

[gggnano]

Then what causes the casimir pressure, if you do the math on abysmal distance of a distance a little over the Planck scale the pressure is literally higher than the pressure inside a neutron star! I'm not saying the Casimir formula is ideal though. Yet: what causes spontaneous emission or Lamb shift? It's like saying that gravity is artifact because it's many times weaker than the other 3 forces.
Ruling out quantum fluctuations, ex nihil, will also rule out another interesting idea: the Bolzmann brain, which is unfalsifiable if you think about it but still interesting thought experiment.

 

[PeterDonis]

If the energy of the quantum vacuum or zero point energy or fluctuations or whatever is considered to not be physical, why do people try to make this calculation?

Who said the people trying to make the calculation did not consider the energy density of the vacuum to be physical?

Unless it is physical which doesn't seem to be at all universally accepted.

That's not at all the same as it not being accepted at all. It should be obvious that there are some physicists who think it is physical and some who think it is not, and since we don't have a good underlying theory that makes a prediction about its value that matches what we observe, this area of research is still open.

 

[PeterDonis]

if instead we wrote the classical hamiltonian as $H=(p-ix)(p+ix)/2$, which seems every bit as valid to me, and then you quantize, it is obvious that this is just: $$H=a^\dagger a = N$$

No, it isn't, because $p$ and $x$ don't commute. So you have

$$
H = \left( p^2 - i x p + i p x + x^2 \right) / 2 = \left( p^2 + x^2 + i [p, x] \right) / 2 = N + \frac{1}{2}
$$

 

[AndreasC]

Then what causes the casimir pressure

There are a lot of papers claiming zero point energy is not required to explain that. I actually found tons of discussion in this forum on the subject, just search for Casimir effect physicsforums and you will see. About Lamb shift etc I don't know much.

 

[AndreasC]

No, it isn't, because $p$ and $x$ don't commute. So you have

$$
H = \left( p^2 - i x p + i p x + x^2 \right) / 2 = \left( p^2 + x^2 + i [p, x] \right) / 2 = N + \frac{1}{2}
$$

Check your calculation again, the ladder operators are p-ix and p+ix, both divided by the square root of 2. The number operator N is their product. It is exactly because they don't commute that this result is different from the same result with just p^2 and x^2.

To be more clear:
$$ H = (p-ix)(p+ix)/2 = \frac{(p-ix)}{\sqrt{2}} \frac{(p+ix)}{\sqrt{2}} = a^\dagger a = N $$

 

[AndreasC]

I am correcting my previous post, since I made a small error regarding the definitions of the operators. The ladder operators are (x+ip) and (x-ip). My point still stands as long as we look at the Hamiltonian:
$$ H = (x-ip)(x+ip)/2 $$
...And carry out the same procedure. The end result is indeed N. The calculation @PeterDonis was trying to carry out gives $H=(p^2+x^2+i[x,p])/2$, where a factor $i[x,p]/2=-1/2$ is added to the previous Hamiltonian, and thus eliminates the 1/2 "zero point energy" term, which definitely seems to solve many issues for me.

Now that I think about it, we could actually have a little more fun with it and come up with whatever "zero point energy" we like, by noticing that terms involving $(x-ip)(x+ip)/2$ add 0 to the number operator, while terms involving $(x+ip)(x-ip)/2$ add 1, yet classically they are all the same. One could write the Hamiltonian as:
$$ H = c(x+ip)(x-ip)/2 + (1-c)(x-ip)(x+ip)/2 $$
Which of course classically is just p^2+x^2, but when you quantize, you get N+c. So I wouldn't take the harmonic oscillator seriously, to answer the question of @Vanadium 50 .

 

[gggnano]

There are a lot of papers claiming zero point energy is not required to explain that. I actually found tons of discussion in this forum on the subject, just search for Casimir effect physicsforums and you will see. About Lamb shift etc I don't know much.

There are. But don't forget that in Michael Faraday's time no one was able to explain the simple experiment of a dipole over a solenoid and hence the name "Faraday paradox"...then they discovered the electron...

I'm not saying vacuum energy is indeed what Feynman and others claim to be of the absurd values of over 10^60 joules/m3 (thought I'm not complaining if we can use even 0.000000000001% of such energy...) yet I'm hoping for explanations better than "it's an error of measurement", "magnets cause it" or "attraction between atoms causes it".

The pressure is 1.3Gpa which is about the same pressure when the first human-made diamond was synthesized! You get ridiculous values if you decrease it even more to a point I'd be curious what the temperature is on solid matter with such pressure, forget the energy:

https://www.wolframalpha.com/input?i=casimir+pressure+1nm

 

[gggnano]

Check your calculation again, the ladder operators are p-ix and p+ix, both divided by the square root of 2. The number operator N is their product. It is exactly because they don't commute that this result is different from the same result with just p^2 and x^2.

To be more clear:
$$ H = (p-ix)(p+ix)/2 = \frac{(p-ix)}{\sqrt{2}} \frac{(p+ix)}{\sqrt{2}} = a^\dagger a = N $$

By the way, I haven't read your discussion with the other user but the Hamiltonian of a particle in QM can be expressed as:

(1/(2m))*p^2+0.5*m*w^2*x^2

And maybe you're trying to show that it has a root for m different that zero over the real line, that is p is zero and w is zero too...but I'm not sure what's the point since this is a rule that electrons or photons or any real particle should obey?? Remember, virtual particles exist for t < Planck-time and thus can violate everything including photons that have mass.

 

[AndreasC]

By the way, I haven't read your discussion with the other user but the Hamiltonian of a particle in QM can be expressed as:

(1/(2m))*p^2+0.5*m*w^2*x^2

And maybe you're trying to show that it has a root for m different that zero over the real line, that is p is zero and w is zero too...but I'm not sure what's the point since this is a rule that electrons or photons or any real particle should obey?? Remember, virtual particles exist for t < Planck-time and thus can violate everything including photons that have mass.

The point is that @Vanadium 50 said that the matter of the simple harmonic oscillator and its zero point energy should be settled first before tackling something more grandiose, like the entire universe. My conclusion so far in absence of further evidence is that in the case of the oscillator it is unphysical, since it only appears as a consequence of quantizing the classical field in a specific way which is ambiguous and far from unique.

Specifically, I took the Hamiltonian you mentioned and set all the constants to 1 for ease, then showed that when you replace x and p with operators, you can get whatever "zero point energy" you want by making algebraic manipulations to the form classical equation that don't change its essence at all.

 

[PeterDonis]

Check your calculation again

If you get two different answers for the same Hamiltonian by two different methods, one of them must be wrong. The result $H = a^\dagger a + 1/2$ is the one that is accepted as correct in the literature.

The ladder operators are (x+ip) and (x-ip).

(More precisely, there should be a $1 / \sqrt{2}$ factor on each one, as you state later on in your post.)

If that is the case, then we have $H = a^\dagger a + 1/2 = \left( x + ip \right) \left( x - ip \right) /2 + 1/2$, not $H = \left( x + ip \right) \left( x - ip \right) / 2$.

The ladder operators are (x+ip) and (x-ip). My point still stands as long as we look at the Hamiltonian:
$$ H = (x-ip)(x+ip)/2 $$

And if your definition of the ladder operators is correct, then this is the wrong Hamiltonian. See above.

In short, it is not possible for all of the claims you are making to be true, since they are not all consistent with each other. Your original calculation of the Hamiltonian in terms of the ladder operators, which is the one that ends up with $H = a^\dagger a + 1/2$, is the one that is accepted as correct in the literature. So your alternative calculation is making a mistake in one of your claims.

 

[PeterDonis]

It is exactly because they don't commute that this result is different from the same result with just p^2 and x^2.

You have the effect of this backwards. If you want the Hamiltonian to be $\left( x^2 + p^2 \right) / 2$, then you cannot just say $H = \left( x + ip \right) \left( x - ip \right)$, because the product of those two does not give you $\left( x^2 + p^2 \right) / 2$ in the quantum case, because of the non-commutation of $x$ and $p$. That's why you have to add the extra correction term to the number operator $N$ to get $H = \left( x^2 + p^2 \right) / 2$.

In other words, the correct statement is $H = \left( x^2 + p^2 \right) / 2 = \left( x + ip \right) \left( x - ip \right) + 1/2 = N + 1/2$. (I did not express this correctly in post #28.) Whereas you are claiming, incorrectly, that $H = \left( x^2 + p^2 \right) / 2 - 1/2 = \left( x + ip \right) \left( x - ip \right) = N$.

 

[PeterDonis]

I took the Hamiltonian you mentioned and set all the constants to 1 for ease, then showed that when you replace x and p with operators, you can get whatever "zero point energy" you want by making algebraic manipulations to the form classical equation that don't change its essence at all.

You have not shown this. See my previous posts, especially post #36.

 

[Vanadium 50]

My conclusion so far in absence of further evidence is that in the case of the oscillator it is unphysical

OK, so then why do you consider the ZPE of the entire universe - which uses exactly the dame equations as the SHO - to be physical?

 

[strangerep]

If you get two different answers for the same Hamiltonian by two different methods, one of them must be wrong. The result $H = a^\dagger a + 1/2$ is the one that is accepted as correct in the literature.

... and then the inconvenient vacuum energy has to be swept away by appealing to the fact that only energy differences are actually measured. Cf. Peskin & Schroeder pp21-22 and p790.

Strictly speaking it's not the "same" Hamiltonian. The $p^2 + x^2$ is a naive transcription from classical to quantum, whereas $(x+ip)(x-ip)$ could be regarded as a better quantum Hamiltonian that still gives the familiar classical version in the limit $\hbar\to 0$.

Related types of classical-vs-quantum quandries occurs in other situations. E.g., for the Hydrogen atom, a straightforward transcription of the classical LRL vector gives a non-hermitian quantum operator. The correct resolution (by Pauli) was to modify the quantum version in such a way that it becomes Hermitian, yet still reduces to the classical version when $\hbar\to 0$.

This is associated with the Groenewold van Hoove theorem about essential ambiguities when trying to pass from classical to quantum.

 

[PeterDonis]

and then the inconvenient vacuum energy has to be swept away by appealing to the fact that only energy differences are actually measured.

But this isn't true in the presence of gravity, because energy gravitates. So absolute values of energy are meaningful. For example, the measured mass of a gravitating body like the Earth is an absolute value of energy, not an energy difference.

The $p^2 + x^2$ is a naive transcription from classical to quantum

No, it isn't, it's the standard quantum kinetic energy operator, $p^2 / 2m$ (the poster I was responding to set all constants to 1, including $m$), and the quantum harmonic oscillator potential energy operator. So it's kinetic plus potential energy, which is the standard way to define a Hamiltonian in QM.

for the Hydrogen atom, a straightforward transcription of the classical LRL vector gives a non-hermitian quantum operator.

But this is not the case for the harmonic oscillator, both $p^2$ and $x^2$ are Hermitian so there's no need to correct anything.

 

[PeterDonis]

This is associated with the Groenewold van Hoove theorem about essential ambiguities when trying to pass from classical to quantum.

As far as I can gather from a quick search, the issues identified by this theorem do not arise if the only observables we are concerned with are at most quadratic in $p$ and $x$, so they would not arise for the harmonic oscillator Hamiltonian.

Also, the theorem does not claim or imply that zero point energy can always be eliminated, so as far as the topic of this thread is concerned, it doesn't appear to resolve anything.

 

[strangerep]

My extra remarks about the LRL vector, and the GvH were intended under a more general umbrella that "sometimes one must adjust the quantum operator version of a classical quantity in order to get a good physical theory". (For the QHO, it was about dispensing with the nonzero vacuum energy, for Hydrogen it was about ensuring hermiticity, while the mention of GvH was for more general cases.)

As for gravity and the QHO, we are still so far from a quantum gravity theory that I don't see much point going down that rabbit hole in this thread.

Responding to the opening post, all I really wanted to say is "yes, I regard the vacuum energy of the QHO as an artifact".

 

[AndreasC]

OK, so then why do you consider the ZPE of the entire universe - which uses exactly the dame equations as the SHO - to be physical?

I don't, I'm just confused by why it seems to be considered physical by many physicists, and wondering if I'm missing something.

 

[AndreasC]

you want the Hamiltonian to be (x2+p2)/2, then you cannot just say H=(x+ip)(x−ip), because the product of those two does not give you (x2+p2)/2 in the quantum case,

Exactly, I do not want it to be x^2+p^2 in the quantum case. That's my point. That by merely quantizing the classical equation in a different manner (I found out that it actually has a name and it is called Wick and anti-Wick quantization), you get a different quantum Hamiltonian that gets rid of or amplifies the zero point energy.

In other words, the correct statement is

No, the first term in the "difference of squares" term is the a operator, and the second is a dagger. Combined they give $aa^\dagger=N+1$. This is anti-Wick quantization. This is the case where the "zero point energy" gets increased. If you wrote the classical Hamiltonian "difference of squares" term the other way round, you get a dagger a, which is just N. This is Wick quantization. And combining the two you can get any "zero point energy" you want. The classical Hamiltonian you posted in that last post is not even correct, you have added an extraneous factor of 1/2. Although one could claim that it is not extraneous since you can add whatever constant to a classical Hamiltonian.

Please check your calculations, and remember $a=(x+ip)/\sqrt2$, $a^\dagger=(x-ip)/\sqrt2$.

 

[AndreasC]

If you get two different answers for the same Hamiltonian by two different methods, one of them must be wrong.

Not really, because you can add whatever overall constant to the Hamiltonian and the physics does not change. The equations of motion do not care about constants.

The result H=a†a+1/2 is the one that is accepted as correct in the literature.

Except for when it isn't because it causes problems, for instance in many cases in statistical mechanics, or the quantization of the free EM field, where the 1/2 term is ignored because it gives weird unnecessary large quantities or infinities.

In short, it is not possible for all of the claims you are making to be true, since they are not all consistent with each other

They are once you accept that constants added to the Hamiltonian are unphysical and don't matter, and only pop up as a result of quantization ambiguity, or simply convention.

 

[AndreasC]

... and then the inconvenient vacuum energy has to be swept away by appealing to the fact that only energy differences are actually measured. Cf. Peskin & Schroeder pp21-22 and p790.

Strictly speaking it's not the "same" Hamiltonian. The $p^2 + x^2$ is a naive transcription from classical to quantum, whereas $(x+ip)(x-ip)$ could be regarded as a better quantum Hamiltonian that still gives the familiar classical version in the limit $\hbar\to 0$.

Related types of classical-vs-quantum quandries occurs in other situations. E.g., for the Hydrogen atom, a straightforward transcription of the classical LRL vector gives a non-hermitian quantum operator. The correct resolution (by Pauli) was to modify the quantum version in such a way that it becomes Hermitian, yet still reduces to the classical version when $\hbar\to 0$.

This is associated with the Groenewold van Hoove theorem about essential ambiguities when trying to pass from classical to quantum.

Exactly, however I am thinking that maybe we shouldn't even be saying that one is correct and the other isn't, because even when you get back to classical mechanics, adding a constant to the Hamiltonian makes no difference physically.

 

[AndreasC]

As far as I can gather from a quick search, the issues identified by this theorem do not arise if the only observables we are concerned with are at most quadratic in p and x, so they would not arise for the harmonic oscillator Hamiltonian.

Yes, but it shows that there is no one "perfect" quantization scheme which would imply it should be preferred. In terms quadratic in x and p, you still have different available schemes, pseudo differential quantization, symmetrized pseudo differential, Wick, anti-Wick, or others for instance the weird mix of Wick and anti-Wick I came up with to get the zero point energy to be whatever we like. Now if one of these schemes gave mathematically preferred answers always, then we could maybe say it is better and more "correct" than the others. But GvH says this does not exist.

 

[AndreasC]

As far as I can gather from a quick search, the issues identified by this theorem do not arise if the only observables we are concerned with are at most quadratic in $p$ and $x$, so they would not arise for the harmonic oscillator Hamiltonian.

Also, the theorem does not claim or imply that zero point energy can always be eliminated, so as far as the topic of this thread is concerned, it doesn't appear to resolve anything.

Hey, if you know an example where it can NOT be eliminated for some reason, I'd love to see it, it's kind of the point of my thread. But the SHO is not such an example for sure.

 

[vanhees71]

The point is that within Newtonian or special-relativistic mechanics only energy differences are observable, i.e., any zero-point energy can be subtracted. Particularly in the here discussed case of relativistic quantum field theory it's very convenient to make the ground-state energy 0 and introduce "normal ordering" for the usual perturbative calculations.

Now this changes within general relativity, because here the energy-momentum-stress tensor, describing the density of energy and momentum and their current densities, enters the right-hand side of Einstein's equation. It's important that here this energy-momentum tensor is uniquely defined as given by the variation of the matter Lagrangian under variations of the pseudo-metric components $g_{\mu \nu}$, because that's how it enters the Einstein equation when derived from the Einstein-Hilbert action.

Now when evaluating the energy-momentum-stress tensor for a relativistic interacting QFT you have to renormalize it at some energy-momentum scale, and the renormalization-group equations tell you, how it changes when changing this renormalization scale. Now when just using the Standard Model of elementary particle physics and you look at this "running" of energy and momentum you obtain some $10^{120}$-factor discrapancy between the observed value of the cosmological constant, which describes the acceleration of the Hubble expansion of our universe, which can be measured by combining observations of the cosmic-microwave-background fluctuations and the distance-redshift relation from Supernovae. The main culprit in this huge discrepancy between theory and observation is the Higgs boson, whose self-energy is quadratically divergent, and afaik there's no really convincing idea, how to cure this deficiency in a "natural way", i.e., to extend the Standard Model somehow such that some mathematical feature of the new model enforces the cosmological constant as predicted by the renormalization group calculation to stay small.

 

[PeterDonis]

I do not want it to be x^2+p^2 in the quantum case.

Why not? As I've already pointed out in response to @strangerep, that's the kinetic energy plus the potential energy, i.e., the standard way to form the Hamiltonian for a quantum system. If you want it to be something else, you need to at least give some kind of physical argument for why in the quantum case the total energy should not be the kinetic energy plus the potential energy.