Negative variance of an observable quantity

[Mandragonia]

Quantum mechanics has a well-known procedure for evaluating the expectation value of an observable quantity in a given quantum state. First one must obtain the quantum operator O that is associated with the observable quantity. Then the rule for computing the expectation value is: Apply O to the wave function psi, multiply the result with psi*, and finally integrate over spatial variables.

The procedure described above can be extended to evaluate the variance of an observable quantity. Now the rule is: apply the operator O twice in succession to the wave function psi, multiply with psi* and integrate over space. From this result, subtract the square of the expectation value.

All this is in clear analogy with the standard rules of statistics. However, in QM things are less clear-cut. For example, it can be demonstrated that the variance may acquire a negative value!

My question is: How should one interpret a negative value for the variance of an observable quantity?

 

[Bill_K]

No, this is false. In QM as well, the variance of an observable is nonnegative.

<A²> = <ψ|A²|ψ> = <ψ|A(Aψ)> = <Aψ|Aψ> ≥ 0

 

[atyy]

Maybe the OP was thinking about negative entropy?
http://arxiv.org/abs/quant-ph/9512022
http://arxiv.org/abs/quant-ph/0702225

 

[Mandragonia]

No, this is false. In QM as well, the variance of an observable is nonnegative.

Okay, thank you! I think I have a counter-example, but I could be wrong of course.

<A²> = <ψ|A²|ψ> = <ψ|A(Aψ)> = <Aψ|Aψ> ≥ 0

Sorry, but I don't understand your third equality. Applying the operator A twice to the wave function on the right may well result in something that correlates negatively with the wave function on the left. However, if you are allowed to shift one operator to the left, then the result becomes indeed positive (or better non-negative) as you conclude.

For example, let psi(x) = sin(x) and operator A is the first derivative with respect to x.
Then your third term equals -sin(x)*sin(x) whereas your fourth term equals cos(x)*cos(x).

 

[The_Duck]

All observables are Hermitian; the definition of a Hermitian operator is that it satisfies

$\langle \phi | A \psi \rangle = \langle A \phi | \psi \rangle$

for any states $| \phi \rangle$ and $| \psi \rangle$. As Bill_K showed, this immediately implies that $A^2$ is "positive semidefinite," i.e., that its expectation value is always nonnegative. This is intuitive if you know that Hermitian operators have only real eigenvalues. Squaring the operator squares the eigenvalues, so the squared operator has only real nonnegative eigenvalues.

 

[Mandragonia]

Thank you. I will have to check whether my example is in accordance with the Hermitian rules.

 

[strangerep]

I will have to check whether my example is in accordance with the Hermitian rules.

You might also want to check whether your example is indeed a normalizable wave function.

If it's not, then it's not in the usual Hilbert space and everything becomes far more challenging...

 

[Mandragonia]

I spent a considerable amount of time checking my calculations. There are things I can't get right.

Observable quantity: Kinetic energy
Associated operator: constant * Laplace operator (L)
Wave function: ground state of Hydrogen atom in spherical coordinates (psi)

A simple example, everything is real. Now according to the Hermitian postulate (psi, LL psi) = (L psi, L psi). But this can not possibly be true! Here is just of many reasons: L takes first and second derivative with respect to r, LL third and fourth derivative. So if psi is a polynomial of degree 1 or 2, then the left hand side is zero, whereas the right hand is some positive function.

 

[strangerep]

A simple example, everything is real. Now according to the Hermitian postulate (psi, LL psi) = (L psi, L psi). But this can not possibly be true! Here is just of many reasons: L takes first and second derivative with respect to r, LL third and fourth derivative. So if psi is a polynomial of degree 1 or 2, then the left hand side is zero, whereas the right hand is some positive function.

You're hitting well-known problems associated with unbounded operators and continuous spectra.

If you want more explicit help, you'll have to show your work...

Firstly, write down your example $\psi$ explicitly (here) as such a polynomial. Then calculate $(\psi,\psi)$ explicitly (here). I presume your inner product is just an integral? What do you get?

 

[Mandragonia]

Okay, thank you very much for your offer! Here is my calculation. For clarity, I will omit all physical parameters and constants such as Planck's constant, electron mass, Bohr radius and pi.

psi(r) = exp(-r) ....... [wave function; the n=1 orbital of the Hydrogen atom]
L = (2/r)*(d/dr) + d^2/dr^2 ..... .. [radial part of the Laplace operator in spherical coordinates]
LL = (4/r)*(d^3/dr^3) + d^4/dr^4 ... [radial part of the Laplace operator applied twice in succession]

I calculate the expectation value of the Kinetic energy squared in two ways, which should be identical [if the Hermitian criteria are met]. First I give the result of the projection of the wave functions (f and g); next the result of multiplying by (r^2)*dr and integrating over the spatial variables, yielding F and G. Note that a * means multiplication (not complex conjugate!).

[1] f(r) = psi * (L(L psi)) = (-4/r + 1) * exp(-2*r)
F = integral (0 to inf) f(r) * r^2 * dr = -3/4

[2] g(r) = (L psi) * (L psi) = (4/r^2 - 4/r + 1) * exp(-2*r)
G = integral (from 0 to inf) g(r) * r^2 * dr = +5/4

I chose method [1], but this leads to a negative value. Why? Method [2] should be identical, but in fact leads to a different result (positive, and therefore perhaps the correct result?).

 

[Bill_K]

L = (2/r)*(d/dr) + d^2/dr^2 ..... .. [radial part of the Laplace operator in spherical coordinates]
LL = (4/r)*(d^3/dr^3) + d^4/dr^4 ... [radial part of the Laplace operator applied twice in succession]

No, when you calculate LL, there's a couple of terms you forgot. (... d²/dr² )(2/r d/dr + ...) when expanded produces d/dr and d²/dr² terms in addition to the ones you have.

 

[Mandragonia]

Did I make a mistake? I believe I performed the calculation with considerable care. The result for LL is what I got after expanding all the terms; in the final result most of the terms (those with the lower order derivatives) cancel. I also performed the calculation by first computing L(psi) explicitly and only then applying the second operator L.

 

[Bill_K]

Sorry, your expression is right.

 

[Mandragonia]

Thank you! The mystery continues...

In my calculations I obtained the general expression for applying N consecutive (radial) Laplace operators:

L^N = (2*N/r) * d^(2*N-1)/dr^(2*N-1) + d^(2*N)/dr^(2*N).

 

[strangerep]

Mandragonia,

Well, congratulations! Your calculations of those integrals above look correct to me.
So then it took me most of today to figure out WTF is going on here...

Here's an exercise to help you resolve the puzzle:

Exercise: Determine (carefully!) whether $L$ is/isn't self-adjoint. I.e., complete the following:
$$
(\phi, L\psi) ~\equiv~ \int_0^\infty\!\! dr\; r^2 \phi^* (2r^{-1}\partial_r
~+~ \partial^2_r ) \psi ~=~ \cdots ~,
$$where $\phi,\psi$ are functions of $r$. (Hint: use integration by parts twice.) What conditions on the wavefunctions must be satisfied for $L$ to be self-adjoint? I.e., under what conditions is it true that:
$$(\phi, L\psi) ~=~ (L\phi, \psi) ~~?$$
And btw, PLEASE learn how to use Latex on this forum.

 

[Mandragonia]

Thank you very much for your time and assistance!

I followed your advice and performed partial integration. The result I get is that the Laplace operator indeed satisfies the Hermitian criterium. However it is necessary that there is no contribution from the boundary terms. These terms can be combined into a single function f.

Define: f(r) = (r^2) (phi* psi' - psi phi*').

[* = complex conjugate; ' = derivative with respect to r]

The condition is that f(inf) - f(0) must be zero.

 

[stevendaryl]

I'm not sure if it is relevant to the original post, but Feynman wrote a little note about the use of negative probabilities in quantum mechanics here:
http://cds.cern.ch/record/154856/files/pre-27827.pdf

 

[Mandragonia]

I will continue the calculation started in my previous post. We are ready to apply the Hermitian rule derived in my previous post to the case discussed in this thread. So we set phi = exp(-r) and psi = L(phi) = (1 - 2/r)*exp(-r). It turns out that the
r -> inf boundary term vanishes, but the r=0 term doesn't. It picks up a contribution equal to -2.

Therefore, in this particular case, the Hermitian rule adjusts to:

(phi, L psi) = (L phi, psi) - 2 or equivalently (phi , LL phi) = (L phi, L phi) - 2.

This explains the difference between the two integrals, evaluated as -3/4 and +5/4 respectively!
An elegant line of reasoning, leading to a surprising and thought provoking result!

 

[strangerep]

Correct so far, afaict. For wavefunctions which vanish at infinity like $e^{-r}$, i.e., faster than any power of $r$, the condition is indeed:
$$
\lim_{r\to 0} \Big( r^2 (\phi^* \psi_r - \phi^*_r \psi) \Big) ~=~ 0 ~,~~~ \forall \phi, \psi ~,
$$ (where my subscripts denote a partial derivatives). However, this must hold for arbitrary $\phi,\psi$, so probably a stronger condition that each term vanish separately is justified.

[...] This explains the difference between the two integrals, evaluated as -3/4 and +5/4 respectively!
An elegant line of reasoning, leading to a surprising and thought provoking result!

Indeed. Here's some more food for thought...

Back in post #9, I made the cryptic remark:

You're hitting well-known problems associated with unbounded operators and continuous spectra.

The problems that arise in this case can broadly be expressed in the Hellinger--Toeplitz theorem:
en.wikipedia.org/wiki/Hellinger-Toeplitz_theorem

Wiki's explanation is rather brief, so a Functional Analysis textbook is needed to delve deeper. Kreyszig's textbook on FA is one that's reasonably friendly to physicsts and engineers, though still quite difficult. Typically, most QM textbooks don't explore such arcane results of Functional Analysis in much detail. The better texts might make a brief remark about how one must be careful about "domains of definition" and "domains of self-adjointness" when working with such operators.

But this still doesn't explain thoroughly why your kinetic energy operator behaves in such a recalcitrant manner for this (supposedly basic) problem of the nonrelativistic H-atom. Naively, one might complain: "well, I solved the Schrodinger equation and normalized the resulting wavefunctions, so what's gone wrong??"

If you'd like to explore further, try this:

Exercise: Repeat the earlier self-adjointness exercise, but this time for:
(i) the potential energy operator $V$ associated with this problem,
and then,
(ii) the full Hamiltonian $H := L+V$ .

 

[Hypersphere]

I'm not sure if it is relevant to the original post, but Feynman wrote a little note about the use of negative probabilities in quantum mechanics here:
http://cds.cern.ch/record/154856/files/pre-27827.pdf

I think it is slightly off the topic Mandragonia eventually wanted, but you might find the Wigner function interesting. It is basically an attempt to provide a probability distribution in phase space, but in some regions associated with unphysical states, the function takes negative values.

 

[bohm2]

I think it is slightly off the topic Mandragonia eventually wanted, but you might find the Wigner function interesting. It is basically an attempt to provide a probability distribution in phase space, but in some regions associated with unphysical states, the function takes negative values.

There is also Khrennikov's interpretation of negative probabilities:

...negative probabilities are absurd objects in the framework of the standard Kolmogorov theory of probability. We present a large class of non-Kolmogorovean probability models where negative probabilities are well defined on the frequency basis.These are models with probabilities which belong to the so-called field of p-adic numbers.

.
p-adic probability prediction of correlations between particles in the two-slit and neutron interferometry experiments
http://arxiv.org/pdf/0906.0509v1.pdf

 

[Mandragonia]

If you'd like to explore further, try this:

Exercise: Repeat the earlier self-adjointness exercise, but this time for:
(i) the potential energy operator $V$ associated with this problem,
and then,
(ii) the full Hamiltonian $H := L+V$ .

Ha! I had already completed all these calculations some time ago...! The peculiar result for the variance of the kinetic energy motivated me to start this thread. Well, I am glad this issue has been resolved.

It turns out the +5/8 result is satisfactory, because now the variance of the kinetic energy is equal to the variance of the potential energy. This makes physical sense. Psi is an eigen state of H; hence for every value of r the deviation from the mean for L(psi) equals minus the deviation from the mean for V(psi). Integrating the squares of these deviations over r, the two variances must be identical.

It is interesting to point out that the standard deviation in the kinetic energy is equal to twice the average value of the kinetic energy. This result suggests that there is a fair probability that the kinetic energy of the electron is negative! In classical physics this make little sense. However in QM the probability distribution stretches well beyond the boundary where the kinetic energy becomes negative (r > 2 * Bohr radius). To get a crude estimate of this probability, I suppose one may use a Gaussian model. The result is 31 %.

 

[George Jones]

The problems that arise in this case can broadly be expressed in the Hellinger--Toeplitz theorem:
en.wikipedia.org/wiki/Hellinger-Toeplitz_theorem

Wiki's explanation is rather brief, so a Functional Analysis textbook is needed to delve deeper. Kreyszig's textbook on FA is one that's reasonably friendly to physicsts and engineers, though still quite difficult. Typically, most QM textbooks don't explore such arcane results of Functional Analysis in much detail. The better texts might make a brief remark about how one must be careful about "domains of definition" and "domains of self-adjointness" when working with such operators.

There is a new book that, at a glance, seems to have nice treatments of all this, "Quantum Theory for Mathematicians" by Brian Hall.

https://www.amazon.com/dp/146147115X/?tag=pfamazon01-20

This looks like a really wonderful book. It has "the basics of L^2 spaces and Hilbert spaces" as prerequisites. Most of the functional analysis presented in the book is in chapters 6 - 10, through which the author gives several paths "I have tried to design this section of the book in such a way that a reader can take in as much or as little of the mathematical details as desired."

The Hellinger-Toeplitz theorem is not named, but it is presented and proved as Corollary 9.9.

 

[stevendaryl]

I think it is slightly off the topic Mandragonia eventually wanted, but you might find the Wigner function interesting. It is basically an attempt to provide a probability distribution in phase space, but in some regions associated with unphysical states, the function takes negative values.

A place where negative probabilities pop up is in the attempt to come up with a hidden-variables model for spin-1/2 twin particle EPR. To simplify the analysis, let's confine spin measurements to being along three axes:

A = the x-axis
B = the direction in the xy plane at 120 degrees clockwise from A.
C = the direction in the xy plane at 240 degrees clockwise from A.

So we assume that there is a hidden variable, \lambda that has 8 possible values:
\lambda_{ABC}, \lambda_{AB\overline{C}}, \lambda_{A\overline{B}C}, \lambda_{A\overline{B}\overline{C}}, \lambda_{\overline{A}BC}, \lambda_{\overline{A}B\overline{C}}, \lambda_{\overline{A}\overline{B}C}, \lambda_{\overline{A}\overline{B}\overline{C}}

where \lambda_{ABC} means that a spin measurement along any of the axes will produce spin-up, \lambda_{AB\overline{C}} means that a spin measurement along A or B will produce spin-up, while a measurement along axis C will produce spin-down, etc. (We assume, because of the perfect anti-correlation, that if one particle has spin-up along one axis, then the twin ha spin-down along that axis).

You can show that this hidden variables theory can only reproduce the quantum mechanical predictions if we assume the following probabilities for the various values of \lambda:

P(\lambda = \lambda_{AB\overline{C}}) <br /> = P(\lambda = \lambda_{A \overline{B}C})<br /> = P(\lambda = \lambda_{\overline{A} B C})<br /> =P(\lambda = \lambda_{\overline{A}B\overline{C}}) <br /> = P(\lambda = \lambda_{\overline{A} \overline{B}C})<br /> = P(\lambda = \lambda_{A \overline{B} \overline{C}}<br /> = \frac{3}{16})

P(\lambda = \lambda_{ABC}) <br /> = P(\lambda = \lambda_{\overline{A} \overline{B}\overline{C}})<br /> = -\frac{1}{16}

The impossibility of negative probabilities shows that no such hidden variables theory can exist.

 

[rlduncan]

A place where negative probabilities pop up is in the attempt to come up with a hidden-variables model for spin-1/2 twin particle EPR. To simplify the analysis, let's confine spin measurements to being along three axes:

A = the x-axis
B = the direction in the xy plane at 120 degrees clockwise from A.
C = the direction in the xy plane at 240 degrees clockwise from A.

So we assume that there is a hidden variable, \lambda that has 8 possible values:
\lambda_{ABC}, \lambda_{AB\overline{C}}, \lambda_{A\overline{B}C}, \lambda_{A\overline{B}\overline{C}}, \lambda_{\overline{A}BC}, \lambda_{\overline{A}B\overline{C}}, \lambda_{\overline{A}\overline{B}C}, \lambda_{\overline{A}\overline{B}\overline{C}}

where \lambda_{ABC} means that a spin measurement along any of the axes will produce spin-up, \lambda_{AB\overline{C}} means that a spin measurement along A or B will produce spin-up, while a measurement along axis C will produce spin-down, etc. (We assume, because of the perfect anti-correlation, that if one particle has spin-up along one axis, then the twin ha spin-down along that axis).

You can show that this hidden variables theory can only reproduce the quantum mechanical predictions if we assume the following probabilities for the various values of \lambda:

P(\lambda = \lambda_{AB\overline{C}}) <br /> = P(\lambda = \lambda_{A \overline{B}C})<br /> = P(\lambda = \lambda_{\overline{A} B C})<br /> =P(\lambda = \lambda_{\overline{A}B\overline{C}}) <br /> = P(\lambda = \lambda_{\overline{A} \overline{B}C})<br /> = P(\lambda = \lambda_{A \overline{B} \overline{C}}<br /> = \frac{3}{16})

P(\lambda = \lambda_{ABC}) <br /> = P(\lambda = \lambda_{\overline{A} \overline{B}\overline{C}})<br /> = -\frac{1}{16}

The impossibility of negative probabilities shows that no such hidden variables theory can exist.

Actually what you have shown (in a concise and eloquent manner using probabilities) is that the last two conditions are impossible. At least one of the three must be + or -, but all three simultaneously cannot have the same sign at the specified angles.

 

[dauto]

This hydrogen wavefunction is a solution to the Schrodinger equation everywhere except at the origin where the solution forms a cusp. The cusp means that the function is not differentiable at the origin, so it can't be a solution to a differential equation. That may, as you found out, cause boundary problems at the origin. The source of that problem is, off course, the fact that the Coulomb potential diverges at the origin. One possible way to attempt to solve that problem would be to replace the point charge with a Gaussian-like distribution of charge at the origin.

 

[strangerep]

Ha! I had already completed all these calculations some time ago...!

So... what results did you get for $(\psi, H^2\psi)$ and $(H\psi, H\psi)$ ?

The peculiar result for the variance of the kinetic energy motivated me to start this thread. Well, I am glad this issue has been resolved.

Hmm, so you think it has been fully resolved?

It turns out the +5/8 result is satisfactory,

What "5/8" result? Previously, you only quoted "5/4".

because now the variance of the kinetic energy is equal to the variance of the potential energy.

Please expand your reasoning here. I don't follow.

This makes physical sense. Psi is an eigen state of H; hence for every value of r the deviation from the mean for L(psi) equals minus the deviation from the mean for V(psi). Integrating the squares of these deviations over r, the two variances must be identical.

I don't follow this either. Please translate your words into math so I can see what you're really talking about.

 

[Mandragonia]

Time-independent Schroedinger equation: H psi = L psi + V psi = E psi ... with E constant
Multiply by psi and integrate: (psi, L psi) + (psi, V psi) = (psi, E psi)
This can be written alternatively as: <L> + <V> = E ... (where the brackets denote averages)
Substitute this into the Schroedinger eq.: (L-<L>) psi + (V-<V>) psi = 0
Define the difference operator D as L-<L>, which is therefore also equal to -(V-<V>).
Then (D psi, D psi) = <D^2> = variance of L = variance of V.

 

[stevendaryl]

Actually what you have shown (in a concise and eloquent manner using probabilities) is that the last two conditions are impossible. At least one of the three must be + or -, but all three simultaneously cannot have the same sign at the specified angles.

That's the same as saying that

P(\lambda = \lambda_{ABC}) = P(\lambda = \lambda_{\overline{A}\overline{B}\overline{C}}) = 0

But that assumption can't possibly reproduce the predictions of QM for this experiment.

Okay, here's a little more of the argument. By symmetry, let's assume that 6 of the probabilities are the same, and the other two are the same:
P(\lambda = \lambda_{\overline{A} B C})<br /> = P(\lambda = \lambda_{A \overline{B} C})<br /> = P(\lambda = \lambda_{A B \overline{C}})<br /> = P(\lambda_{A \overline{B} \overline{C}})<br /> = P(\lambda_{\overline{A} B \overline{C}})<br /> = P(\lambda_{\overline{A} \overline{B} C})

P(\lambda = \lambda_{A B C}) = P(\lambda = \lambda_{\overline{A} \overline{B} \overline{C}})

So let \alpha be P(\lambda = \lambda_{\overline{A} B C}) and let \beta be P(\lambda = \lambda_{A B C})

The QM probability that one particle is detected to have spin-up along axis A while the other particle is detected to have spin-down along axis B is:

\frac{1}{2}sin^2(\frac{120}{2}) = \frac{1}{8}

In terms of this hidden-variables model, there are two ways to get this result:

1. \lambda = \lambda_{A B C}
2. \lambda = \lambda_{A B \overline{C}}

So for the probabilities to work out, we would need:
P(\lambda = \lambda_{A B C}) + P(\lambda = \lambda_{A B \overline{C}}) = \frac{1}{8}

So in terms of \alpha and \beta, we have:
\beta + \alpha = \frac{1}{8}

The QM probability that one particle is detected to have spin-up along axis A while the other particle is detected to have spin-up along axis B is:

\frac{1}{2}cos^2(\frac{120}{2}) = \frac{3}{8}

In terms of our hidden-variables model, this implies:

P(\lambda = \lambda_{A \overline{B} C}) + P(\lambda = \lambda_{A \overline{B} \overline{C}}) = \frac{3}{8}

So in terms of \alpha and \beta, this means:
2 \alpha = \frac{3}{8}

So we have \alpha =\frac{3}{16} and \beta=-\frac{1}{16}

 

[Mandragonia]

Strangrep -- Thank you for showing an interest in my calculations. Apart from the <LL> problem, I encountered one other oddity. Namely that the operators VL and LV seemed to have different expectation values. I thought this effect could presumably be explained by the non-commutativity of the two operators. But I had no real clue how to resolve the issue.

Thanks to the discussions in this thread, I also found a satisfactory answer to this problem. Using Hermitian properties the expectation values should be rewritten in the form: (V psi, L psi) and (L psi, V psi). Now for the VL case the procedure is trivial (since V is multiplicative). For the LV case however, the Hermitian procedure leads to a boundary term. Taking this term into account, the LV and VL operators have the same expectation value. Nice!

It is still a bit mysterious to me that different representations co-exist, with sometimes different results. This raises the question how one can know a priori what the correct representation is. Suppose one is interested in three observables with operators A, B and C. First of all there are 6 permutations ABC, ACB etc. Secondly, each of these can be represented in four forms: (psi, ABC psi); (A psi, BC psi); (BA psi, C psi) and (CBA psi, psi). A messy situation. How should one proceed?

 

[strangerep]

Thanks to the discussions in this thread, I also found a satisfactory answer to this problem. Using Hermitian properties the expectation values should be rewritten in the form: (V psi, L psi) and (L psi, V psi). Now for the VL case the procedure is trivial (since V is multiplicative). For the LV case however, the Hermitian procedure leads to a boundary term. Taking this term into account, the LV and VL operators have the same expectation value. Nice!

Actually, I think you have got hold of the wrong end of the stick...

The boundary terms account for why the various integrals differ. But that does not answer deeper questions like: "how may one define variance sensibly when unbounded operators are involved?". In that sense, the "answer" so far is not yet satisfactory.

[...] A messy situation. How should one proceed?

Before we delve into that, I note that you ignored my earlier question about what values you got for $(\psi,H^2\psi)$ and $(H\psi, H\psi)$.

Similarly, what values did you get for $(\psi, VL\psi)$, $(V\psi, L\psi)$, $(L\psi, V\psi)$, and $(\psi, LV\psi)$ ?

 

[Mandragonia]

I note that you ignored my earlier question about what values you got for $(\psi,H^2\psi)$ and $(H\psi, H\psi)$.

I am sorry, I skipped the question because the answer seems self-evident. The system is in a Hamiltonian eigen state. So we are free to replace the Hamilton operator by its eigen value E. This is a constant, so it follows directly that (psi, HH psi) = (H psi, H psi) = E^2 = +1/4.

Similarly, what values did you get for $(\psi, VL\psi)$, $(V\psi, L\psi)$, $(L\psi, V\psi)$, and $(\psi, LV\psi)$ ?

The four values I found are: -3/2, -3/2, -3/2, +1/2. The first three appear to be correct. The difference between results 3 and 4 is due to a boundary term at r=0.

 

[Mandragonia]

Now I would like to present the final results for all the expectation values of the observables, as I believe them to be correct. I denote the kinetic energy by T, whereas the Laplace operator (at its core) is denoted by L. This time I have inserted all the physical parameters and constants where they belong. Having done so, it becomes apparent that the most convenient energy unit is the Rydberg constant of +13.606 eV. Scaling the expectation values for T, V, H and E in terms of the Rydberg constant, all the results become integers.

<T> = +1 .... <V> = -2 ...... <H> = -1 ..... <E> = E = -1

<TT> = +5 ... <TV> = -6 ...... <TH> = -1 ...... <TE> = -1
<VT> = -6 ... <VV> = +8 ...... <VH> = +2 ...... <VE> = +2
<HT> = -1 ... <HV> = +2 ...... <HH> = +1 ...... <HE> = +1

variance(T) = +4 ... variance(V) = +4... variance(H) = 0 ..... variance(E) = 0

The results satisfy the requirements of symmetry <AB> = <BA> and additivity <AT> + <AV> = <AH> for every A and B. They are in agreement with the fact that the system is in an eigen state of the Hamiltonian: <AH> = <AE> = E*<A> and hence variance(H) = 0. The proper way to obtain an expectation value <AB> is apparently by evaluating the integral (A psi, B psi).

 

[Mandragonia]

There is a fair probability that the kinetic energy of the electron is negative! To get a crude estimate of this probability, I suppose one may use a Gaussian model. The result is 31 %.

The exact result is: p = 13*exp(-4) = 23.8 %.

 

[dextercioby]

Mandragonia, your statements in this thread make little to absolutely no sense, at least from a mathematical perspective.

 

[strangerep]

[Mandragonia's] statements in this thread make little to absolutely no sense, at least from a mathematical perspective.

I know what you mean. The only reason I've been able to follow him is because I've worked out the various integrals. The goal in my responses in this thread is (eventually) to explain in some detail why statements like the following are incorrect:

The proper way to obtain an expectation value <AB> is apparently by evaluating the integral (A psi, B psi).

(But not right now.)

 

[bhobba]

There is a new book that, at a glance, seems to have nice treatments of all this, "Quantum Theory for Mathematicians" by Brian Hall.

https://www.amazon.com/dp/146147115X/?tag=pfamazon01-20

This looks like a really wonderful book. It has "the basics of L^2 spaces and Hilbert spaces" as prerequisites. Most of the functional analysis presented in the book is in chapters 6 - 10, through which the author gives several paths "I have tried to design this section of the book in such a way that a reader can take in as much or as little of the mathematical details as desired."

The Hellinger-Toeplitz theorem is not named, but it is presented and proved as Corollary 9.9.

Checked it out.

Nice book - must get a copy.

What I like is, horror of horrors, it actually includes the Dirac Delta function formalism most physicists actually use, as well as a bit of a review of distribution theory. It isn't just a rehash of Von Neumann's classic.

As an aside all physicists and mathematicians should, IMHO, have knowledge of distribution theory. A book I got years ago on it has been one of the best I ever got:
https://www.amazon.com/dp/0521558905/?tag=pfamazon01-20

Thanks
Bill

 

[strangerep]

Suppose one is interested in three observables with operators A, B and C. First of all there are 6 permutations ABC, ACB etc. Secondly, each of these can be represented in four forms: (psi, ABC psi); (A psi, BC psi); (BA psi, C psi) and (CBA psi, psi). A messy situation. How should one proceed?
[...]
The proper way to obtain an expectation value <AB> is apparently by evaluating the integral (A psi, B psi).

Let's take a temporary detour...

Forget about QM for a moment, and consider the following problem in ordinary (classical) statistics. Suppose that $f(x)$ is a probability distribution for the random variable $x$. So, among other things, it's normalized as $\int f(x) dx = 1$, and the mean of the distribution is calculated by $\int xf(x) dx$.

Exercises: Consider the specific distribution: $f(x) := \frac{1}{\pi(1+x^2)}$, where $x$ takes values on the whole real line.

1) Calculate whether the distribution $f(x)$ is already normalized to 1.

2) Calculate the mean of the distribution $f(x)$ .

3) Write down the definition of variance (in ordinary statistics), and calculate the variance of the distribution $f(x)$.

 

[Mandragonia]

The boundary terms account for why the various integrals differ. But that does not answer deeper questions like: "how may one define variance sensibly when unbounded operators are involved?". In that sense, the "answer" so far is not yet satisfactory.

I agree. I can't help you though, because I am insufficiently familiar with operators etc. However, today I came up with an idea that might help to avoid the problem altogether. The energy relation in Rydberg units reads T - 2/r = -1. This is equivalent to r = 2/(T+1). Using this 1-1 relation between r and T, it is straightforward to convert the probability function for r into a probability function for T. The result is:

rho(T) = 32 * (T+1)^-4 * exp{-4/(T+1)} ... where T ranges from -1 to +infinity.

This function allows us to evaluate all properties of the kinetic energy distribution DIRECTLY. There is no longer any need for the Laplace operator! Thus we side-step all problems associated with differentation, commutation, boundary effects etc. The results are UNAMBIGUOUS.

One may verify that the function is correctly normalized. The first two moments of T are easily computed, and found to be: <T> = +1 and <T^2> = +5. So the variance of T equals +4. It is also straightforward to derive properties for the potential energy V, since it can be expressed as V = -(T+1). One obtains <V> = -2 and <V^2> = +8. Therefore the variance of V equals +4. Note that these results are fully consistent with those I previously posted.

 

[strangerep]

The energy relation in Rydberg units reads T - 2/r = -1. This is equivalent to r = 2/(T+1). Using this 1-1 relation between r and T, it is straightforward to convert the probability function for r

What "probability function for r" are you talking about? I have no idea what you're doing.

into a probability function for T. The result is:

rho(T) = 32 * (T+1)^-4 * exp{-4/(T+1)} ... where T ranges from -1 to +infinity.

 

[Mandragonia]

What "probability function for r" are you talking about? I have no idea what you're doing.

I am referring to the probability density for the radial coordinate r (which represents the distance between the electron and the nucleus). In units of the Bohr radius, it is given by:

rho(r) = 4 * (r^2) * exp(-2*r) ... where r ranges from 0 to infinity.

It can be used to calculate properties of the orbital that depend only on r, such as the average distance of the electron from the nucleus <r>. Obviously it can not be used to calculate expectation values of operators that depend on differentiation! However, in the case of the kinetic energy one can side-step this problem by making clever use of the energy relation. This allows one to construct a similar probability density, but now for the kinetic energy T.

 

[Mandragonia]

Consider the distribution: $f(x) := \frac{1}{\pi(1+x^2)}$, where $x$ takes values on the whole real line.

Ha! I recognized the function immediately. It is a well-known example of a distribution with the property that its variance diverges. It is commonly presented to undergraduate math students, so that they learn that not every normalized bell-shaped distribution necessarily has a finite variance.

 

[strangerep]

Ha! I recognized the function immediately. It is a well-known example of a distribution with the property that its variance diverges. It is commonly presented to undergraduate math students, so that they learn that not every normalized bell-shaped distribution necessarily has a finite variance.

OK, so... I guess there's no need for me to elaborate on how this is relevant to your example.

 

[Mandragonia]

Strangerep -- I acknowledge that one has to be careful when doing calculations on the hydrogen orbital; because it is ill-defined at r=0.
I think Dauto has formulated it well:

This hydrogen wavefunction is a solution to the Schrodinger equation everywhere except at the origin where the solution forms a cusp. The cusp means that the function is not differentiable at the origin, so it can't be a solution to a differential equation. That may, as you found out, cause boundary problems at the origin. The source of that problem is, off course, the fact that the Coulomb potential diverges at the origin. One possible way to attempt to solve that problem would be to replace the point charge with a Gaussian-like distribution of charge at the origin.

The equation I derived for the probability density of the kinetic energy is certainly not immune to this problem. However, since it only involves straightforward integration, one can pinpoint the problem more clearly and take appropriate measures.

My question to you: have you reached a conclusion on my formula for the probability density of the kinetic energy?

 

[strangerep]

[...] have you reached a conclusion on my formula for the probability density of the kinetic energy?

You have still not explained yourself clearly and thoroughly. I'm not going to try anymore to guess what you mean.

 

[Mandragonia]

Multiply the wave function Psi(r) by its complex conjugate and by spherical shell 4*pi*r^2 and you get the probability density Rho(r). The latter has a straightforward physical meaning. It tells you that at any moment in time the electron has a probability given by Rho(r)*dr to be at a distance in the interval (r, r+dr). One can obtain the cumulative probability P(R), by integrating Rho(r) from 0 to R. One may also define Q(R) as 1-P(R). The results for the n=1 orbital in the hydrogen atom are:

P1(R) = 1 - (2*R^2 + 2*R + 1) * exp(-2*R)
Q1(R) = (2*R^2 + 2*R + 1) * exp(-2*R)

These functions have the following meaning.
[1a] P1(R) is the probability that the electron is at a distance r smaller than R from the nucleus.
[1b] Q1(R) is the probability that the electron is at a distance r larger than R from the nucleus.

Now by logical induction, we are free to reformulate these statements as follows:
[2a] There is a probability P2(1/R) that the electron is at a reciprocal distance 1/r larger than 1/R.
[2b] There is a probability Q2(1/R) that the electron is at a reciprocal distance 1/r smaller than 1/R.

Consistency demands that P2=P1 and Q2=Q1 for all distances r. It turns out to be very easy to derive the explicit formulas for P2 and Q2 in terms of P1 and Q1.

P2(S) = 1 - (2/S^2 +2/S + 1) * exp(-2/S)
Q2(S) = (2/S^2 + 2/S + 1) * exp(-2/S)

Setting the reciprocal distance S equal to 1/R, one verifies that indeed P2=P1 and Q2=Q1.
The next step is to recognize that in reduced units the potential energy V(r) is equal to -2/r. This differs by just a constant factor (-2) from the reciprocal distance S = 1/r. Substitution of S = -V/2 in P2 and Q2 yields:

P3(V) = 1 - (8/V^2 - 4/V +1) *exp (4/V)
Q3(V) = (8/V^2 - 4/V + 1) * exp(4/V)

The physical interpretation is again straightforward.
[3a] P3(V) is the probability that the potential energy is smaller than V = -2/R.
[3b] Q3(V) is the probability that the potential energy is larger than V = -2/R.

Next we use H = T+V = E. Since E equals -1 in reduced units, we can write the energy balance as V = -(T+1). Apply this to P3 and Q3.

P4(T) = 1 - {8/(T+1)^2 + 4/(T+1) +1} *exp{-4/(T+1)}
Q4(T) = {8/(T+1)^2 + 4/(T+1) + 1} * exp{-4/(T+1)}

With the physical interpretation:
[4a] P4(T) is the probability that the kinetic energy is larger than T = -1 + 2/R.
[4b] Q4(T) is the probability that the kinetic energy is smaller than T = -1 + 2/R.

Taking the derivative of Q4 with respect to T, we obtain the probability density Rho(T) for the kinetic energy.

Rho(T) = 32 * (T+1)^-4 * exp{-4/(T+1)} for T ranging from -1 to +infinity.

The physical interpretation of this formula is that Rho(T)*dT is the probability that the kinetic energy is in the interval (T, T+dT). We can use the last formula to evaluate properties of the kinetic energy, without having to apply the Laplace operator to the wave function.

 

[strangerep]

[...] We can use the last formula to evaluate properties of the kinetic energy, without having to apply the Laplace operator to the wave function.

All that stuff boils down to nothing more than a change of variable in an integral, of the form: $\rho(r) dr \to \rho(V) dV~$ or $\rho(r) dr \to \rho(T) dT$.

Such a change of variable relies on an equation like: $$T = E + 1/r ~, $$where one tacitly assumes we act only on the state $\psi$. Indeed, for the state $\psi$ which is an eigenstate of the Hamiltonian, i.e., $H\psi = E\psi$, the $T$ operator acts like a multiplication operator:
$$T\psi ~=~ (E+1/r)\psi ~.$$You extrapolate this to mean that the substitution $T = E + 1/r$ can be applied more generally. However, it is not generally true that a higher power of $T$, such as $T^2$, is equivalent to $(E+1/r)^2$ when acting on $\psi$. The reason is that $T\psi$ is not an eigenstate of $H$ with eigenvalue $E$ .

This would invalidate your attempt to use probability distributions in the way you describe to calculate higher moments of the distribution such as the variance of kinetic energy.

[BTW, I might make an effort to reply sooner if you make an effort to learn and use basic Latex on this forum. No one wants to read all that ugly ascii math. Instructions for getting started with Latex can be found by following one of the pulldown menus near the top of the PF main page. I.e., SITEINFO->FAQ. There's really no excuse.]

 

[Mandragonia]

It is not generally true that a higher power of $T$, such as $T^2$, is equivalent to $(E+1/r)^2$ when acting on $\psi$. The reason is that $T\psi$ is not an eigenstate of $H$ with eigenvalue $E$ .

The key point of this thread is that the operator $T^2$ is not to be trusted, since it can lead to unphysical results, such as a negative variance. The right representation is apparently with two operators $T$, working on the left and right wavefunction $\psi$ respectively. Now in that representation it is immediately clear that both operators $T$ can indeed be replaced by their multiplicative cousin $(E+1/r)$. So up to second order the concept works well.

It is possible that my idea fails to describe the third and higher order moments of the kinetic energy correctly. We don't know that. As yet it is not even clear how to represent these higher moments in operator formalism.

 

[strangerep]

The key point of this thread is that the operator $T^2$ is not to be trusted, since it can lead to unphysical results, such as a negative variance.

In fact, kinetic energy is not a good observable for the dynamical system of the hydrogen atom. This occurs in other cases too -- in the sum $H=T+V$, the 3 operators are not necessarily well-defined on a common domain. Although $H$ may be well-defined as an observable on the Hilbert space constructed from its eigenstates, this doesn't necessarily mean that the $T$ operator is also a good observable on that Hilbert space. That's the key lesson to be learned in this thread.

(Indeed, making the mistake of thinking that the kinetic, potential, and total energy operators are sensibly well-defined on the same Hilbert space is one of the underlying causes of infinities in QFT.)

One might also ponder (but not too long) on how/whether the kinetic energy of a bound electron could ever be measured directly.

The right representation is apparently with two operators $T$, working on the left and right wavefunction $\psi$ respectively.

If you mean that this is the correct representation for variance of kinetic energy, then no. It does not correspond to any standard definition of "variance".

Herein lies the point of my suggested exercise involving the Cauchy distribution, (which you dismissed so readily). Not all probability distributions have well-defined higher moments (and some even have ill-defined means). In such cases, one must adopt alternative measures of "average" and "dispersion".

 

[Mandragonia]

I have finally found a satisfactory solution to the problem that I posed in this thread !

Careful examination of the boundary terms that arise at r=0 indicates that the problem of non-self-adjointness of the kinetic energy operator T is associated with the presence of odd powers of r in the expansion of the wave function around r=0. This is an indication that the "true" wave function has only even terms in its expansion! In fact, this result makes a lot of sense, both mathematically and physically.

In order to get rid of the odd terms of r in Psi, I chose to adjust the wave function in a small region around the origin (0, epsilon). One can then calculate the higher moments of the kinetic energy operator. Finally taking the limit of epsilon -> 0, the results indeed satisfy the self-adjointness criterium. The computation itself is far from trivial. It involves many terms with higher order derivatives of the delta function, which are generated at r=epsilon, the boundary between the thin interface and the bulk region.

In conclusion I can say that the standard solution to the SE, Psi = exp(-r), is merely an approximation of the correct wave function. Sometimes this approximation works well, and sometimes it doesn't. In the latter case, one has to use a (mathematically) more appropriate version of the wave function to obtain the right results.

Perhaps I shall write a short article about my little project.