Dyson's View Of Wavefunction Collapse

[martinbn]

May be a bit off topic, but this article may help understand Dyson a bit better.

https://www.damtp.cam.ac.uk/user/tong/em/dyson.pdf

And this article, from it

I have observed in teaching quantum mechanics, and also in learning it, that students go through an experience similar to the one that Serbian physicist, Mihajlo Idvorsky Pupin describes. The student begins by learning the tricks of the trade. He learns how to make calculations in quantum mechanics and get the right answers, how to calculate the scattering of neutrons by protons and so forth. To learn the mathematics of the subject and to learn how to use it takes about six months. This is the first stage in learning quantum mechanics, and it is comparatively painless.

The second stage comes when the student begins to worry because he does not understand what he has been doing. He worries because he has no clear physical picture in his head. He gets confused in trying to arrive at a physical explanation for each of the mathematical tricks he has been taught. He works very hard and gets discouraged because he does not seem to be able to think clearly. This second stage often lasts six months or longer. It is strenuous and unpleasant.

Then, unexpectedly, the third stage begins. The student suddenly says to himself, “I understand quantum mechanics,” or rather he says, “I understand now that there isn’t anything to be understood.” The difficulties which seemed so formidable have mysteriously vanished. What has happened is that he has learned to think directly and unconsciously in quantum-mechanical language. He is no longer trying to explain everything in terms of prequantum conceptions.

The duration and severity of the second stage are decreasing as the years go by. Each new generation of students learns quantum mechanics more easily than their teachers learned it. The students are growing more detached from prequantum pictures. There is less resistance to be broken down before they feel at home with quantum ideas. Ultimately, the second stage will disappear entirely. Quantum mechanics will be accepted by students from the beginning as a simple and natural way of thinking, because we shall all have grown used to it. By that time, if science progresses as we hope, we shall be ready for the next big jump into the unknown.

 

[Peter Morgan]

That precisely was Dyson's point. (What Scientific Idea Is Ready For Retirement?)


I share Dyson's view and used the same quote in what was probably my first post on this forum. And I agree that it is puzzling (and sad!) that such discussions are still going on in the International Quantum Year.

The root of the problem, in my opinion, is the exaggerated role of the time-dependent wave function and Schrödinger's equation. There is an almost irrepressible urge to describe a quantum object in a pseudo-Markovian (quasi-Newtonian?) fashion as having at all times some definite state evolving continuously and even deterministically. It simply doesn't square with the discontinuities and randomness that experiments seem to reveal in the real world. There's a reason why von Neumann introduced "measurement" (collapse) as a separate process besides unitary evolution. But it's an uneasy combination and, for some physicists, has created a new "measurement problem". I think Schrödinger's equation is but one piece of the mathematical apparatus of quantum mechanics, and it must not be pulled apart from the other essential component of the machinery, namely Born's rule.

It is said that the Heisenberg picture is completely equivalent to the Schrödinger picture. But this is true only if one considers the full picture, and the wave function by itself does not provide the full picture! What is the place of wave function collapse in the Heisenberg picture? In the Heisenberg picture the state (wave function) just remains constant!

I think there is real graininess (discontinuities) in the real world, and that it is correctly described by quantum theory. But it's not caused by collapsing wave functions. In my view, quantum theory must be seen as a stochastic theory.

Thank you for the URL for Dyson's quote on Edge.org.
A Stochastic Process is commonly defined as a collection of random variables indexed by time, which I suppose is only a starting point for a more elaborate stochastic theory that would be as complete as quantum field theory. In the first instance, I suggest there are three more-or-less clear differences: a quantum field is commonly defined as an operator-valued distribution indexed by time and space(1), which generate a noncommutative(2) algebra of measurement operators, and there are two measures of dispersion, both kT for thermal noise and ℏ for quantum noise(3).
We can accommodate (1) and (3) into a classical formalism quite straightforwardly, by working with a random-variable-valued distribution indexed by space and time(1), and by noticing that quantum noise is Lorentz invariant whereas thermal noise, which is a symmetry property that can be adopted into classical physics(3). Noncommutativity can be thought of as classically natural, as a way of accommodating different experimental contexts into a single formalism instead of taking each initial condition as a different model universe, but of course that is a considerable extension.
Although I agree that there is a real graininess in the experimental data I suggest that it is not a graininess that survives in a stochastic formalism when we consider the fine-grained signals out of an apparatus. At a timescale that is sub-picosecond, say, every measured signal exhibits a finite rise time. In any case, at such scales and with extreme amplification there is always too much noise to be sure of much except the fine-grained signal convolved with a relatively coarse-grained window function, which is as smooth as the window function we choose.
In the article I linked to above, in JPhysA 2022 (arXiv, DOI there), I introduced a new kind of 'picture', which I call the super-Heisenberg picture, which is not related to the Heisenberg picture and Schrödinger picture by a unitary transformation. I call it the super-Heisenberg picture because it absorbs the collapse of the quantum state as well as the unitary evolution into the measurement operators, in contrast to the Heisenberg picture, which absorbs only the unitary evolution into the measurement operators. Applid in the QFT setting, that results in a Hilbert space formalism that can be thought of as (a)a commutative QFT; as (b)a commutative algebraic model of a stochastic theory with (1) and (3) added; or as (c) a QFT with (2) subtracted. There are connections with Koopman's Hilbert space formalism for classical mechanics, for anyone who knows that construction.

 

[Demystifier]

May be a bit off topic, but this article may help understand Dyson a bit better.

https://www.damtp.cam.ac.uk/user/tong/em/dyson.pdf

And this article, from it

I see your point, but how to explain that, in the last 30 years or so, the interest in quantum foundations increases, rather than decreases? There is more and more mature physicists who think that QM as such is not sufficiently clear, that there is something to understand about QM which is not explained by its formalism.

 

[martinbn]

I see your point, but how to explain that, in the last 30 years or so, the interest in quantum foundations increases, rather than decreases? There is more and more mature physicists who think that QM as such is not sufficiently clear, that there is something to understand about QM which is not explained by its formalism.

That is a question for Dyson, I am not sure my view is entirely the same as his. There is a lot of interest in the last 30 years, but what is the progress made?

 

[Demystifier]

That is a question for Dyson, I am not sure my view is entirely the same as his. There is a lot of interest in the last 30 years, but what is the progress made?

There is no much progress, I admit. But that can be said for many other branches of physics as well. Actually, the Nobel prize in physics last year is given for a research which did not make a progress in physics at all, which indicates that physics as a whole is in crisis. But now I'm off-topic.

 

[martinbn]

There is no much progress, I admit.

But there is some, right?

 

[Peter Morgan]

That is a question for Dyson, I am not sure my view is entirely the same as his. There is a lot of interest in the last 30 years, but what is the progress made?

Working physicists have slowly been making QM seem less and less 'weird'. As Philip Ball put it in the title of one of his popular-level books, QM is becoming "Beyond Weird". It has been a drop-by-drop process that can be dismissed as just Shut-Up-And-Calculate, but IMO that kind of dismissal misses the bigger picture.
I suggest, in particular, that the distance between classical stochastic models and quantum models has been steadily eroding, so that there is now almost a flood of articles about classical-quantum models. There are now many physicists pursuing stochastic models in ways that would have been thought laughable 20 years ago.
't Hooft weathered years of ridicule for his approach to QM, but Oppenheim, Barandes, Wetterich, Khrennikov, Hossenfelder, Palmer, Carcassi, as only those who come to mind in a moment, are now developing similar ideas. Wolfram and Weinberg are well-known on the periphery of Physics, and there are many more physicists pursuing similar ideas less visibly. Nobody has succeeded in making a stochastic process 'story' compelling, but I think there's now more of a watch-this-space feeling than there was even five years ago.

What was called "quantum probability" in the 1990s is now called "Generalized Probability Theory", and the nature of that generalization is just a mathematical issue of different measurement contexts, not about physics at all. The Measurement Theory literature now is almost unrecognizably different from the literature in 1990.

I could belabor this some more, but the Oxford Philosophy of Physics Seminar was willing to listen to the approach to the relationship between classical physics and quantum physics that I suggest, so here that is: , title "A Dataset&Signal Analysis Interpretation of Quantum Field Theory".
To be clear, I only want to claim that this is progress, it's very far from perfect, but that's what you asked for: progress, not perfection. In some ways this channels SUAC and Copenhagen to construct a much more potent classical physics than we have been used to. Here's the abstract:

 

[Demystifier]

But there is some, right?

Of course.

 

[martinbn]

Of course.

Like?

 

[Morbert]

I see your point, but how to explain that, in the last 30 years or so, the interest in quantum foundations increases, rather than decreases? There is more and more mature physicists who think that QM as such is not sufficiently clear, that there is something to understand about QM which is not explained by its formalism.

This is likely due to the protean character of QM, and hence the mutual independence of different quantum foundation projects. Progress made in Everettian interpretations will not close QBism research projects.

QM does not uniquely select among equally complete projects, so instead of branches being pruned as per usual, you get a divergent mess of branches offering more and more opportunities for paper writing.

 

[Demystifier]

Like?

Better theoretical and experimental understanding of decoherence, various new no-go theorems, sharpening of various interpretations, ...

 

[Peter Morgan]

This is likely due to the protean character of QM, and hence the mutual independence of different quantum foundation projects. Progress made in Everettian interpretations will close QBism research projects.

QM does not uniquely select among equally complete projects, so instead of branches being pruned as per usual, you get a divergent mess of branches offering more and more opportunities for paper writing.

There are convergences as well as divergences. In particular, Koopman's Hilbert space formalism for classical mechanics allows a unification of classical mechanics with quantum mechanics. It has taken since the 1931 appearance of Koopman's original paper suggesting that formalism because the issues are subtle, but those subtleties are slowly coalescing.

 

[A. Neumaier]

A wave-function is a description of a probability, and a probability is a statement of ignorance. Ignorance is not a physical object, and neither is a wave-function. When new knowledge displaces ignorance, the wave-function does not collapse; it merely becomes irrelevant.

A subjective probability is a statement of ignorance, and one can learn and make the ignorance go away.

But an objective probability is a well-defined property of an ensemble. If you observe one item of the ensemble, the probability does not change but persists.

Many physicists have the view that probability is objective!

There is nothing in the quote to suggest that Dyson makes such an assumption.

Observations are physical in Dyson's text quoted in #1, since they are taken as objective pieces of evidence. Since they do not appear in the wave function, they are additional input to reality.

What is the place of wave function collapse in the Heisenberg picture? In the Heisenberg picture the state (wave function) just remains constant!

But in the Heisenberg picture, dynamics is always unitary.

How do you describe in the Heisenberg picture two consecutive measurements of noncommuting observables? The states changes nonunitarily in between!

 

[Morbert]

But in the Heisenberg picture, dynamics is always unitary.

How do you describe in the Heisenberg picture two consecutive measurements of noncommuting observables? The states changes nonunitarily in between!

Wouldn't the Heisenberg picture just evolve the projectors unitarily? E.g. Two consecutive measurements at $t_1$ and $t_2$ would yield respective outcomes $a$ and $b$ with probability $\mathrm{tr}\Pi_a(t_1)\rho\Pi_a(t_1)\Pi_b(t_2)$

 

[WernerQH]

How do you describe in the Heisenberg picture two consecutive measurements of noncommuting observables? The states changes nonunitarily in between!

All that quantum theory is concerned with are operator products and traces over them.
We can calculate correlation functions of any order ($n$-point functions) and compare them with experimental data. What is missing, in your opinion?

 

[A. Neumaier]

Wouldn't the Heisenberg picture just evolve the projectors unitarily? E.g. Two consecutive measurements at $t_1$ and $t_2$ would yield respective outcomes $a$ and $b$ with probability $\mathrm{tr}\Pi_a(t_1)\rho\Pi_a(t_1)\Pi_b(t_2)$

Can you write down the details of how this should follow from the Heisenberg dynamics and some assumed form of Born's rule for a single measurement? Or do you need a separate axiom for the probability of 2,3,4,,... consecutive experiments?

 

[A. Neumaier]

All that quantum theory is concerned with are operator products and traces over them.

This is just the mathematical formalism, without its link to reality.

We can calculate correlation functions of any order ($n$-point functions) and compare them with experimental data. What is missing, in your opinion?

How do you get Born's rule for two consecutive measurements (which is traditionally used to compare with consecutive measurements) from these $n$-point functions?

 

[martinbn]

Observations are physical in Dyson's text quoted in #1, since they are taken as objective pieces of evidence. Since they do not appear in the wave function, they are additional input to reality.

Not sure what you mean by this and how it relates to my post! Are you saying that Dyson takes observations for lambda?

 

[gentzen]

Can you write down the details of how this should follow from the Heisenberg dynamics and some assumed form of Born's rule for a single measurement? Or do you need a separate axiom for the probability of 2,3,4,,... consecutive experiments?

It looks to me like Morbert just wrote down the formulas from the Consistent Histories formalism. Which makes sense, because those formulas take their simplest form in the Heisenberg picture.

But since there is no collapse in the Consistent Histories formalism, this may not be the answer (or formula) you are looking for.

 

[Morbert]

It looks to me like Morbert just wrote down the formulas from the Consistent Histories formalism. Which makes sense, because those formulas take their simplest form in the Heisenberg picture.

But since there is no collapse in the Consistent Histories formalism, this may not be the answer (or formula) you are looking for.

Consistent Histories uses these formulas a lot but the formulas are not exclusive to that interpretation.

They come from more general reasoning about consecutive measurements, where outcomes a then b can be computed from products of conditional probabilities $p(a|\rho)p(b|a,\rho)$ which results in expressions like the one I gave. While it can be written down with only unitary dynamics, deriving it from Born's rule without recourse to collapse or filtering seems difficult.

[edit] - And actually, you will quickly run into decoherence issues unless the degrees of freedom the measurement apparatuses are explicitly included.

 

[WernerQH]

How do you get Born's rule for two consecutive measurements (which is traditionally used to compare with consecutive measurements) from these $n$-point functions?

Why not just use the Born rule? You may have a desire to derive it from some other special principle describing "measurements". But it doesn't clarify what constitutes a measurement, and as John Bell has argued (Against Measurement), there's really no place for such a concept in a microscopic theory.

The Heisenberg picture differs from the Schrödinger picture in that "measurement" is not thought to happen in an instant. The entire experiment is considered -- lasting for an interval of time including "state preparation" and "measurement". A probability can be calculated for any particular history, but you are not restricted to the use of projection operators (as Consistent Histories may have it). You can, for example, evaluate correlation functions for the positions of a harmonic oscillator at different times. And people have applied these to study the quantum behaviour of the LIGO-mirrors.

 

[Nugatory]

And this article, from it

…. The duration and severity of the second stage are decreasing as the years go by. Each new generation of students learns quantum mechanics more easily than their teachers learned it. The students are growing more detached from prequantum pictures. There is less resistance to be broken down before they feel at home with quantum ideas. Ultimately, the second stage will disappear entirely. Quantum mechanics will be accepted by students from the beginning as a simple and natural way of thinking, because we shall all have grown used to it.

”Ultimately” is a long time, but I am not so optimistic. Early childhood makes everyone an Aristotelian, and (as witness the B-level threads in Classical and Relativity) there is a substantial conceptual shift to get from there to Newtonian physics and Galilean relativity. Those who have made it through that shift tend to be heavily invested in the classical model that they’ve worked so hard to internalize.

 

[A. Neumaier]

Not sure what you mean by this and how it relates to my post! Are you saying that Dyson takes observations for lambda?

I am saying that Dyson takes certain things for real. These must be described by real physics, and play the role of what Demystifier calles lambda.

Why not just use the Born rule?

Because Born's rule is formulated in the Schrödinger picture, and is for single measurements only. For a sequence of measurements one needs state reduction, and hence must find an alternative expression of this when one claims that n-point functions are everything.

as John Bell has argued (Against Measurement), there's really no place for such a concept in a microscopic theory.

But then there must be a replacement doing the same job that Born's rule does. Bell is silent on this.

 

[martinbn]

I am saying that Dyson takes certain things for real. These must be described by real physics, and play the role of what Demystifier calles lambda.

I don't think so. Demystitier calls lambda sometging more restrictive than just anything real. For him lambda is the real state of the quantum system if the wave function represents our knowledge.

 

[PeterDonis]

For him lambda is the real state of the quantum system if the wave function represents our knowledge.

Not quite. The PBR theorem assumes that the wave function represents some kind of probability distribution over an underlying set of states; the latter are $\lambda$. But that concept of $\lambda$ is not limited to the context of the PBR theorem.

For there not to be any $\lambda$, it would have to be the case that it is impossible to describe the allowed real states of the system using any set of states. The only QM interpretation I'm aware of that makes any claim sort of like that is the version of Copenhagen that says it's impossible to have a more complete description than the probabilistic description QM provides. But that version doesn't seem to be very popular; certainly it's not the interpretation @Demystifier favors. And I'm not sure it's the interpretation Dyson would have favored either.

 

[Demystifier]

I don't think so. Demystitier calls lambda sometging more restrictive than just anything real. For him lambda is the real state of the quantum system if the wave function represents our knowledge.

Not quite. The PBR theorem assumes that the wave function represents some kind of probability distribution over an underlying set of states; the latter are $\lambda$. But that concept of $\lambda$ is not limited to the context of the PBR theorem.

For there not to be any $\lambda$, it would have to be the case that it is impossible to describe the allowed real states of the system using any set of states. The only QM interpretation I'm aware of that makes any claim sort of like that is the version of Copenhagen that says it's impossible to have a more complete description than the probabilistic description QM provides. But that version doesn't seem to be very popular; certainly it's not the interpretation @Demystifier favors. And I'm not sure it's the interpretation Dyson would have favored either.

PeteDonis is right, in general lambda can by anything "real", including the wave function. In a special case, when the wave function is interpreted as a purely epistemological entity, the lambda does not include wave function, but this is just a special case. The PBR theorem rules this case out, at least if the assumptions of the theorem are satisfied.

 

[martinbn]

PeteDonis is right, in general lambda can by anything "real", including the wave function. In a special case, when the wave function is interpreted as a purely epistemological entity, the lambda does not include wave function, but this is just a special case. The PBR theorem rules this case out, at least if the assumptions of the theorem are satisfied.

Yes, I agree with this, but Neumaier said the observations are real. I don't see how the observations can be the lambda. My understanding is that the lambda stands for some parameters that describe some real state. Not just anything that is real. You cannot say electrons are real so they are the lambda, right? The lambda needs to be some characteristic of the electrons.

 

[Demystifier]

Yes, I agree with this, but Neumaier said the observations are real. I don't see how the observations can be the lambda. My understanding is that the lambda stands for some parameters that describe some real state. Not just anything that is real. You cannot say electrons are real so they are the lambda, right? The lambda needs to be some characteristic of the electrons.

Would you agree that observations can be described by lambda?

 

[martinbn]

Would you agree that observations can be described by lambda?

How? Say I make a measurement and observe the particle at some place. What lambda describes that observation?

 

[Demystifier]

How? Say I make a measurement and observe the particle at some place. What lambda describes that observation?

The measurement involves a pointer of a macroscopic apparatus, it is supposed to be described by some lambda too. For instance, in Bohmian mechanics this is positions of particles which constitute the measuring apparatus.