I Is the Cosmological Constant Problem a Misunderstanding of Zero-Point Energy?

[AndreasC]

I am correcting my previous post, since I made a small error regarding the definitions of the operators. The ladder operators are (x+ip) and (x-ip). My point still stands as long as we look at the Hamiltonian:
$$ H = (x-ip)(x+ip)/2 $$
...And carry out the same procedure. The end result is indeed N. The calculation @PeterDonis was trying to carry out gives $H=(p^2+x^2+i[x,p])/2$, where a factor $i[x,p]/2=-1/2$ is added to the previous Hamiltonian, and thus eliminates the 1/2 "zero point energy" term, which definitely seems to solve many issues for me.

Now that I think about it, we could actually have a little more fun with it and come up with whatever "zero point energy" we like, by noticing that terms involving $(x-ip)(x+ip)/2$ add 0 to the number operator, while terms involving $(x+ip)(x-ip)/2$ add 1, yet classically they are all the same. One could write the Hamiltonian as:
$$ H = c(x+ip)(x-ip)/2 + (1-c)(x-ip)(x+ip)/2 $$
Which of course classically is just p^2+x^2, but when you quantize, you get N+c. So I wouldn't take the harmonic oscillator seriously, to answer the question of @Vanadium 50 .

 

[gggnano]

There are a lot of papers claiming zero point energy is not required to explain that. I actually found tons of discussion in this forum on the subject, just search for Casimir effect physicsforums and you will see. About Lamb shift etc I don't know much.

There are. But don't forget that in Michael Faraday's time no one was able to explain the simple experiment of a dipole over a solenoid and hence the name "Faraday paradox"...then they discovered the electron...

I'm not saying vacuum energy is indeed what Feynman and others claim to be of the absurd values of over 10^60 joules/m3 (thought I'm not complaining if we can use even 0.000000000001% of such energy...) yet I'm hoping for explanations better than "it's an error of measurement", "magnets cause it" or "attraction between atoms causes it".

The pressure is 1.3Gpa which is about the same pressure when the first human-made diamond was synthesized! You get ridiculous values if you decrease it even more to a point I'd be curious what the temperature is on solid matter with such pressure, forget the energy:

https://www.wolframalpha.com/input?i=casimir+pressure+1nm

 

[gggnano]

Check your calculation again, the ladder operators are p-ix and p+ix, both divided by the square root of 2. The number operator N is their product. It is exactly because they don't commute that this result is different from the same result with just p^2 and x^2.

To be more clear:
$$ H = (p-ix)(p+ix)/2 = \frac{(p-ix)}{\sqrt{2}} \frac{(p+ix)}{\sqrt{2}} = a^\dagger a = N $$

By the way, I haven't read your discussion with the other user but the Hamiltonian of a particle in QM can be expressed as:

(1/(2m))*p^2+0.5*m*w^2*x^2

And maybe you're trying to show that it has a root for m different that zero over the real line, that is p is zero and w is zero too...but I'm not sure what's the point since this is a rule that electrons or photons or any real particle should obey?? Remember, virtual particles exist for t < Planck-time and thus can violate everything including photons that have mass.

 

[AndreasC]

By the way, I haven't read your discussion with the other user but the Hamiltonian of a particle in QM can be expressed as:

(1/(2m))*p^2+0.5*m*w^2*x^2

And maybe you're trying to show that it has a root for m different that zero over the real line, that is p is zero and w is zero too...but I'm not sure what's the point since this is a rule that electrons or photons or any real particle should obey?? Remember, virtual particles exist for t < Planck-time and thus can violate everything including photons that have mass.

The point is that @Vanadium 50 said that the matter of the simple harmonic oscillator and its zero point energy should be settled first before tackling something more grandiose, like the entire universe. My conclusion so far in absence of further evidence is that in the case of the oscillator it is unphysical, since it only appears as a consequence of quantizing the classical field in a specific way which is ambiguous and far from unique.

Specifically, I took the Hamiltonian you mentioned and set all the constants to 1 for ease, then showed that when you replace x and p with operators, you can get whatever "zero point energy" you want by making algebraic manipulations to the form classical equation that don't change its essence at all.

 

[PeterDonis]

Check your calculation again

If you get two different answers for the same Hamiltonian by two different methods, one of them must be wrong. The result $H = a^\dagger a + 1/2$ is the one that is accepted as correct in the literature.

The ladder operators are (x+ip) and (x-ip).

(More precisely, there should be a $1 / \sqrt{2}$ factor on each one, as you state later on in your post.)

If that is the case, then we have $H = a^\dagger a + 1/2 = \left( x + ip \right) \left( x - ip \right) /2 + 1/2$, not $H = \left( x + ip \right) \left( x - ip \right) / 2$.

The ladder operators are (x+ip) and (x-ip). My point still stands as long as we look at the Hamiltonian:
$$ H = (x-ip)(x+ip)/2 $$

And if your definition of the ladder operators is correct, then this is the wrong Hamiltonian. See above.

In short, it is not possible for all of the claims you are making to be true, since they are not all consistent with each other. Your original calculation of the Hamiltonian in terms of the ladder operators, which is the one that ends up with $H = a^\dagger a + 1/2$, is the one that is accepted as correct in the literature. So your alternative calculation is making a mistake in one of your claims.

 

[PeterDonis]

It is exactly because they don't commute that this result is different from the same result with just p^2 and x^2.

You have the effect of this backwards. If you want the Hamiltonian to be $\left( x^2 + p^2 \right) / 2$, then you cannot just say $H = \left( x + ip \right) \left( x - ip \right)$, because the product of those two does not give you $\left( x^2 + p^2 \right) / 2$ in the quantum case, because of the non-commutation of $x$ and $p$. That's why you have to add the extra correction term to the number operator $N$ to get $H = \left( x^2 + p^2 \right) / 2$.

In other words, the correct statement is $H = \left( x^2 + p^2 \right) / 2 = \left( x + ip \right) \left( x - ip \right) + 1/2 = N + 1/2$. (I did not express this correctly in post #28.) Whereas you are claiming, incorrectly, that $H = \left( x^2 + p^2 \right) / 2 - 1/2 = \left( x + ip \right) \left( x - ip \right) = N$.

 

[PeterDonis]

I took the Hamiltonian you mentioned and set all the constants to 1 for ease, then showed that when you replace x and p with operators, you can get whatever "zero point energy" you want by making algebraic manipulations to the form classical equation that don't change its essence at all.

You have not shown this. See my previous posts, especially post #36.

 

[Vanadium 50]

My conclusion so far in absence of further evidence is that in the case of the oscillator it is unphysical

OK, so then why do you consider the ZPE of the entire universe - which uses exactly the dame equations as the SHO - to be physical?

 

[strangerep]

If you get two different answers for the same Hamiltonian by two different methods, one of them must be wrong. The result $H = a^\dagger a + 1/2$ is the one that is accepted as correct in the literature.

... and then the inconvenient vacuum energy has to be swept away by appealing to the fact that only energy differences are actually measured. Cf. Peskin & Schroeder pp21-22 and p790.

Strictly speaking it's not the "same" Hamiltonian. The $p^2 + x^2$ is a naive transcription from classical to quantum, whereas $(x+ip)(x-ip)$ could be regarded as a better quantum Hamiltonian that still gives the familiar classical version in the limit $\hbar\to 0$.

Related types of classical-vs-quantum quandries occurs in other situations. E.g., for the Hydrogen atom, a straightforward transcription of the classical LRL vector gives a non-hermitian quantum operator. The correct resolution (by Pauli) was to modify the quantum version in such a way that it becomes Hermitian, yet still reduces to the classical version when $\hbar\to 0$.

This is associated with the Groenewold van Hoove theorem about essential ambiguities when trying to pass from classical to quantum.

 

[PeterDonis]

and then the inconvenient vacuum energy has to be swept away by appealing to the fact that only energy differences are actually measured.

But this isn't true in the presence of gravity, because energy gravitates. So absolute values of energy are meaningful. For example, the measured mass of a gravitating body like the Earth is an absolute value of energy, not an energy difference.

The $p^2 + x^2$ is a naive transcription from classical to quantum

No, it isn't, it's the standard quantum kinetic energy operator, $p^2 / 2m$ (the poster I was responding to set all constants to 1, including $m$), and the quantum harmonic oscillator potential energy operator. So it's kinetic plus potential energy, which is the standard way to define a Hamiltonian in QM.

for the Hydrogen atom, a straightforward transcription of the classical LRL vector gives a non-hermitian quantum operator.

But this is not the case for the harmonic oscillator, both $p^2$ and $x^2$ are Hermitian so there's no need to correct anything.

 

[PeterDonis]

This is associated with the Groenewold van Hoove theorem about essential ambiguities when trying to pass from classical to quantum.

As far as I can gather from a quick search, the issues identified by this theorem do not arise if the only observables we are concerned with are at most quadratic in $p$ and $x$, so they would not arise for the harmonic oscillator Hamiltonian.

Also, the theorem does not claim or imply that zero point energy can always be eliminated, so as far as the topic of this thread is concerned, it doesn't appear to resolve anything.

 

[strangerep]

My extra remarks about the LRL vector, and the GvH were intended under a more general umbrella that "sometimes one must adjust the quantum operator version of a classical quantity in order to get a good physical theory". (For the QHO, it was about dispensing with the nonzero vacuum energy, for Hydrogen it was about ensuring hermiticity, while the mention of GvH was for more general cases.)

As for gravity and the QHO, we are still so far from a quantum gravity theory that I don't see much point going down that rabbit hole in this thread.

Responding to the opening post, all I really wanted to say is "yes, I regard the vacuum energy of the QHO as an artifact".

 

[AndreasC]

OK, so then why do you consider the ZPE of the entire universe - which uses exactly the dame equations as the SHO - to be physical?

I don't, I'm just confused by why it seems to be considered physical by many physicists, and wondering if I'm missing something.

 

[AndreasC]

you want the Hamiltonian to be (x2+p2)/2, then you cannot just say H=(x+ip)(x−ip), because the product of those two does not give you (x2+p2)/2 in the quantum case,

Exactly, I do not want it to be x^2+p^2 in the quantum case. That's my point. That by merely quantizing the classical equation in a different manner (I found out that it actually has a name and it is called Wick and anti-Wick quantization), you get a different quantum Hamiltonian that gets rid of or amplifies the zero point energy.

In other words, the correct statement is

No, the first term in the "difference of squares" term is the a operator, and the second is a dagger. Combined they give $aa^\dagger=N+1$. This is anti-Wick quantization. This is the case where the "zero point energy" gets increased. If you wrote the classical Hamiltonian "difference of squares" term the other way round, you get a dagger a, which is just N. This is Wick quantization. And combining the two you can get any "zero point energy" you want. The classical Hamiltonian you posted in that last post is not even correct, you have added an extraneous factor of 1/2. Although one could claim that it is not extraneous since you can add whatever constant to a classical Hamiltonian.

Please check your calculations, and remember $a=(x+ip)/\sqrt2$, $a^\dagger=(x-ip)/\sqrt2$.

 

[AndreasC]

If you get two different answers for the same Hamiltonian by two different methods, one of them must be wrong.

Not really, because you can add whatever overall constant to the Hamiltonian and the physics does not change. The equations of motion do not care about constants.

The result H=a†a+1/2 is the one that is accepted as correct in the literature.

Except for when it isn't because it causes problems, for instance in many cases in statistical mechanics, or the quantization of the free EM field, where the 1/2 term is ignored because it gives weird unnecessary large quantities or infinities.

In short, it is not possible for all of the claims you are making to be true, since they are not all consistent with each other

They are once you accept that constants added to the Hamiltonian are unphysical and don't matter, and only pop up as a result of quantization ambiguity, or simply convention.

 

[AndreasC]

... and then the inconvenient vacuum energy has to be swept away by appealing to the fact that only energy differences are actually measured. Cf. Peskin & Schroeder pp21-22 and p790.

Strictly speaking it's not the "same" Hamiltonian. The $p^2 + x^2$ is a naive transcription from classical to quantum, whereas $(x+ip)(x-ip)$ could be regarded as a better quantum Hamiltonian that still gives the familiar classical version in the limit $\hbar\to 0$.

Related types of classical-vs-quantum quandries occurs in other situations. E.g., for the Hydrogen atom, a straightforward transcription of the classical LRL vector gives a non-hermitian quantum operator. The correct resolution (by Pauli) was to modify the quantum version in such a way that it becomes Hermitian, yet still reduces to the classical version when $\hbar\to 0$.

This is associated with the Groenewold van Hoove theorem about essential ambiguities when trying to pass from classical to quantum.

Exactly, however I am thinking that maybe we shouldn't even be saying that one is correct and the other isn't, because even when you get back to classical mechanics, adding a constant to the Hamiltonian makes no difference physically.

 

[AndreasC]

As far as I can gather from a quick search, the issues identified by this theorem do not arise if the only observables we are concerned with are at most quadratic in p and x, so they would not arise for the harmonic oscillator Hamiltonian.

Yes, but it shows that there is no one "perfect" quantization scheme which would imply it should be preferred. In terms quadratic in x and p, you still have different available schemes, pseudo differential quantization, symmetrized pseudo differential, Wick, anti-Wick, or others for instance the weird mix of Wick and anti-Wick I came up with to get the zero point energy to be whatever we like. Now if one of these schemes gave mathematically preferred answers always, then we could maybe say it is better and more "correct" than the others. But GvH says this does not exist.

 

[AndreasC]

As far as I can gather from a quick search, the issues identified by this theorem do not arise if the only observables we are concerned with are at most quadratic in $p$ and $x$, so they would not arise for the harmonic oscillator Hamiltonian.

Also, the theorem does not claim or imply that zero point energy can always be eliminated, so as far as the topic of this thread is concerned, it doesn't appear to resolve anything.

Hey, if you know an example where it can NOT be eliminated for some reason, I'd love to see it, it's kind of the point of my thread. But the SHO is not such an example for sure.

 

[vanhees71]

The point is that within Newtonian or special-relativistic mechanics only energy differences are observable, i.e., any zero-point energy can be subtracted. Particularly in the here discussed case of relativistic quantum field theory it's very convenient to make the ground-state energy 0 and introduce "normal ordering" for the usual perturbative calculations.

Now this changes within general relativity, because here the energy-momentum-stress tensor, describing the density of energy and momentum and their current densities, enters the right-hand side of Einstein's equation. It's important that here this energy-momentum tensor is uniquely defined as given by the variation of the matter Lagrangian under variations of the pseudo-metric components $g_{\mu \nu}$, because that's how it enters the Einstein equation when derived from the Einstein-Hilbert action.

Now when evaluating the energy-momentum-stress tensor for a relativistic interacting QFT you have to renormalize it at some energy-momentum scale, and the renormalization-group equations tell you, how it changes when changing this renormalization scale. Now when just using the Standard Model of elementary particle physics and you look at this "running" of energy and momentum you obtain some $10^{120}$-factor discrapancy between the observed value of the cosmological constant, which describes the acceleration of the Hubble expansion of our universe, which can be measured by combining observations of the cosmic-microwave-background fluctuations and the distance-redshift relation from Supernovae. The main culprit in this huge discrepancy between theory and observation is the Higgs boson, whose self-energy is quadratically divergent, and afaik there's no really convincing idea, how to cure this deficiency in a "natural way", i.e., to extend the Standard Model somehow such that some mathematical feature of the new model enforces the cosmological constant as predicted by the renormalization group calculation to stay small.

 

[PeterDonis]

I do not want it to be x^2+p^2 in the quantum case.

Why not? As I've already pointed out in response to @strangerep, that's the kinetic energy plus the potential energy, i.e., the standard way to form the Hamiltonian for a quantum system. If you want it to be something else, you need to at least give some kind of physical argument for why in the quantum case the total energy should not be the kinetic energy plus the potential energy.

 

[PeterDonis]

you can add whatever overall constant to the Hamiltonian and the physics does not change.

As I have already pointed out in response to @strangerep, in the presence of gravity this is not the case, because energy gravitates. So absolute values of energy matter; you can't just add or remove arbitrary constants. And since in this thread we are talking about the cosmological constant problem, we are certainly not ignoring gravity.,

 

[AndreasC]

The point is that within Newtonian or special-relativistic mechanics only energy differences are observable, i.e., any zero-point energy can be subtracted. Particularly in the here discussed case of relativistic quantum field theory it's very convenient to make the ground-state energy 0 and introduce "normal ordering" for the usual perturbative calculations.

Now this changes within general relativity, because here the energy-momentum-stress tensor, describing the density of energy and momentum and their current densities, enters the right-hand side of Einstein's equation. It's important that here this energy-momentum tensor is uniquely defined as given by the variation of the matter Lagrangian under variations of the pseudo-metric components $g_{\mu \nu}$, because that's how it enters the Einstein equation when derived from the Einstein-Hilbert action.

Now when evaluating the energy-momentum-stress tensor for a relativistic interacting QFT you have to renormalize it at some energy-momentum scale, and the renormalization-group equations tell you, how it changes when changing this renormalization scale. Now when just using the Standard Model of elementary particle physics and you look at this "running" of energy and momentum you obtain some $10^{120}$-factor discrapancy between the observed value of the cosmological constant, which describes the acceleration of the Hubble expansion of our universe, which can be measured by combining observations of the cosmic-microwave-background fluctuations and the distance-redshift relation from Supernovae. The main culprit in this huge discrepancy between theory and observation is the Higgs boson, whose self-energy is quadratically divergent, and afaik there's no really convincing idea, how to cure this deficiency in a "natural way", i.e., to extend the Standard Model somehow such that some mathematical feature of the new model enforces the cosmological constant as predicted by the renormalization group calculation to stay small.

Hmm ok, this definitely goes a bit further as far as I can tell to justify the whole idea. But I want to understand here what exactly goes into figuring out that stress energy tensor via the quantum theory.

 

[AndreasC]

As I have already pointed out in response to @strangerep, in the presence of gravity this is not the case, because energy gravitates. So absolute values of energy matter; you can't just add or remove arbitrary constants. And since in this thread we are talking about the cosmological constant problem, we are certainly not ignoring gravity.,

Yes but the point is that if you want to do that, you have to come up with an "official" level, and the entire point is that this choice is arbitrary. There is no theoretical reason why one or the other Hamiltonian is "better" within the confines of the quantum or the classical theory.

 

[AndreasC]

Why not? As I've already pointed out in response to @strangerep, that's the kinetic energy plus the potential energy, i.e., the standard way to form the Hamiltonian for a quantum system. If you want it to be something else, you need to at least give some kind of physical argument for why in the quantum case the total energy should not be the kinetic energy plus the potential energy.

Or the potential energy could be the same as your potential energy plus whatever constant you want since it doesn't matter, and you haven't given a good theoretical reason why one is better than the other. As I showed even just quantizing the very same Hamiltonian in a slightly different and perfectly valid way can add a constant to it.

 

[PeterDonis]

Or the potential energy could be the same as your potential energy plus whatever constant you want since it doesn't matter

Yes, it does matter in the presence of gravity. I have already stated why, several times. Either respond to what I said about that, or please stop repeating this statement as if it has not been challenged.

if you want to do that, you have to come up with an "official" level

The obvious "official" level is the sum of the kinetic and potential energy, as I have already said. (I believe this is consistent with what @vanhees71 has said about the stress energy tensor of a quantum field, but I'll let him respond further to that as he knows more about that aspect than I do.) Again, if you are not going to respond to that obvious physical argument for why that Hamiltonian should be preferred, at least do not keep repeating the same statement as though it has not been challenged.

 

[strangerep]

[..] need to at least give some kind of physical argument for why in the quantum case the total energy should not be the kinetic energy plus the potential energy.

Last time I checked, the separation of total energy into kinetic+potential energies is not form invariant under the full group of canonical transformations -- which, in the quantum case, means under general Bogoliubov transformations that preserve that canonical commutation relations. These transformations can mix c/a operators (hence mix position and momentum operators) in complicated ways, or add/subtract constants.

In the no-gravity case, this explains why one is reasonably justified in modifying the total Hamiltonian if it yields better-behaved physics predictions.

In the with-gravity case,... who knows? Until the Einstein Field Equations are somehow reconciled to fit consistently with (some generalization of) quantum Bogoliubov transformations (QBTs), I won't be making any premature assertions. Heck, it might be that careful use of QBTs in a generalized renormalization group context could help with the "gravity-gravitates" difficulty, but without a well-developed concrete theory this is speculative, so I'll stop here.

Instead, I'll let the numbers (quantum VeVs versus the value of the cosmological constant) speak for themselves as to why the two probably have little to do with each other.

 

[vanhees71]

Yes, it does matter in the presence of gravity. I have already stated why, several times. Either respond to what I said about that, or please stop repeating this statement as if it has not been challenged.The obvious "official" level is the sum of the kinetic and potential energy, as I have already said. (I believe this is consistent with what @vanhees71 has said about the stress energy tensor of a quantum field, but I'll let him respond further to that as he knows more about that aspect than I do.) Again, if you are not going to respond to that obvious physical argument for why that Hamiltonian should be preferred, at least do not keep repeating the same statement as though it has not been challenged.

Of course, in relativistic physics the notion of "a potential" is not so clear, but what you have in relativistic field theory is a Lagrangian, from which you can derive the Hamilton (density) of the fields. As long as you do special relativity an additive overall constant to the Hamiltonian doesn't matter since it's just providing a common phase factor to the states, which is not observable.

This changes when taking into account gravitation and general relativity. There additive constants in energy matter, because the energy-momentum-stress tensor is the source of the gravitational field according to Einstein's field equation.

 

[AndreasC]

Yes, it does matter in the presence of gravity. I have already stated why, several times. Either respond to what I said about that, or please stop repeating this statement as if it has not been challenged.

I have responded many times but you keep misinterpreting what I said.

The obvious "official" level is the sum of the kinetic and potential energy, as I have already said

Cool, let's go with that, now what is the potential energy and why are you so convinced it is what you are saying it is and not some other, theoretically equivalent version where a constant has been added to it? Why is your quantization scheme correct and not Wick ordered quantization, or anti Wick or whatever else? If you answer "because in gravity it matters", yes indeed it does, but it is missing the whole point because it is circular. I understand that you are saying it makes a difference when you are considering gravity, but saying it is different on its own doesn't invalidate one or the other option. I'm asking you why you believe your option for the Hamiltonian is the valid one and "mine" isn't, and you are telling me "it's because yours is different from mine and mine is the correct one". Ok but I'm trying to figure out what if any is the theoretical or observational justification for one being preferred to the other. I'm eager to hear if there is such a reason, but you have to give me that, not tell me that it matters in gravity, that's not the point. Saying it is the kinetic plus the potential energy isn't a good reason either because I can just say the constant is part of the potential energy.

I apologize if this comes off as rude, it's not my intention, I just think you have not completely understood what I'm asking.

 

[vanhees71]

There is no right or wrong quantization scheme. As I repeatedly tried to explain in special-relativistic QFT the absolute value of the total energy is not observable, and you have to renormalize the total energy anyway. The standard renormalization is such to make the energy of the ground state (vacuum) 0.

In general-relativistic QFT, i.e., QFT in a given "classical background spacetime" you have the same problem, and if you then calculate the energy-momentum tensor of the "matter fields" you have to renormalize it too, and to adjust it to what's expected to be observed at the Planck scale needs finetuning, which is considered "unnatural" by most physicists. That's why one looks for some (symmetry) principle which explains the value of the cosmological constant, but there's no such model, let alone a solution for the notorious problem of quantizing also the gravitational interaction in a consistent way. Until there's no such theory, it's quite speculative to guess, how this problem might be solved.

 

[PeterDonis]

what is the potential energy

For the harmonic oscillator, which was the specific case we were considering, it is $x^2 / 2$, where $x$ is the displacement from equilibrium. That's the physical definition: the potential energy is zero at the equilibrium point. To quantize that you just make $x$ the position operator, as with any other potential in QM.

More generally, the "correct" potential energy is the one whose zero point corresponds to the state that is physically picked out as "equilibrium" or something similar. For example, for an isolated gravitating body the potential energy is zero at infinity. That's how you pick out the "correct" potential from all the other mathematically possible options.