What does it take to solve the measurement problem? (new paper published)

[vanhees71]

It's in any textbook of quantum theory or quantum field theory. There is an energy-momentum conserving $\delta$-distribution in the S-matrix.

 

[A. Neumaier]

It's in any textbook of quantum theory or quantum field theory. There is an energy-momentum conserving $\delta$-distribution in the S-matrix.

Together with the probabilistic interpretation of the S-matrix, this gives indeed energy-momentum conservation violations with probability zero. (Thus finitely many exceptions are still allowed, and we can do only finitely many observations.)

But

the argument by @CoolMint was about particle number conservation,

for which the S-matrix argument does not apply. $N$ photons in does not imply $N$ photons out.

 

[vanhees71]

Of course not. The photon number is not conserved. Nobody has ever claimed this.

 

[CoolMint]

Photons are the entities resembling particles the least.

 

[A. Neumaier]

Photons are the entities resembling particles the least.

Gluons, W-bosons, Higgs, etc. are also not conserved but are still considered particles.

 

[gentzen]

It depends how one counts. Those discussing the unique outcome problem say that there are two outcomes, one at each detector. This is not observed, hence should be explained. The quest for an explanation is the unique outcome problem.

But the "unique outcome problem" is solved by Born's probabilistic interpretation of the state. So, where's the problem?

And now you come and ask „where‘s the problem“, the „unique outcome problem“ is solved by Born‘s …
..., this reaction feels very strange to me.

Why?

Because the discussion seemed to be about "what is the unique outcome problem" or "why should there be a unique outcome problem," and not about solving it by experimental observations or by postulating specific (reasonable) axioms.

Physics is an empirical science, and Schrödinger's interpretation of the physical meaning of his wave function was not in accordance with the observations, i.e., single electrons were not detected as little "smeared clouds of continuous charge distributions" but as dots on a photoplate.

I don't think that Schrödinger's "very first" interpretation was the subject of this discussion. Additionally, dots on a photoplate without further context cannot "solve" the problem either. All they do is exhibit a context where the "smeared clouds of continuous charge distributions" interpretation is not applicable. Of course, that naive "smeared" interpretation is almost never a good interpretation, even so smeared charge distributions do occur in the density functional theory.My overall problem with this "strange style" of discussion is that it makes it difficult for me to dive into subtle issues in details. For example, the discussion has now touched topics like "conservation laws event by event" or that "photons are the entities resembling particles the least". As we have seen in discussions with RUTA, boundary conditions can sometimes prevent conservation laws from holding event by event, while still enforcing conservation laws for the quantum expectations.

Also with respect to solving the "unique outcome problem" by "postulating specific (reasonable) axioms," there would be interesting subtle details: What sort of postulate would that be? Would it be more of the "2+2=4" type, the "ZFC is consistent" type, or rather the "PA is consistent" type? You see, "2+2=4" is simply true, and "ZFC is consistent" is simply undecidable. But "PA is consistent" is subtle. It is true, it can be proved true in at least three fundamentally different informal ways, but it must be postulated nevertheless.

Maybe more fundamental, I would have found it nice to clarify A. Neumaier's view on Ensembles in quantum field theory. I don't even understand why he thought that you have to "repeatedly prepare a quantum field extending over all of spacetime" in order to use an ensemble interpretation of QFT. But if already the fact that an ensemble interpretation is not universally applicable is never acknowledged, not even in simple examples, then this makes it difficult for me to dive into such subtle issues.

 

[vanhees71]

"Particle" has a very specific meaning in modern relativistic QFT.

I don't think that Schrödinger's "very first" interpretation was the subject of this discussion. Additionally, dots on a photoplate without further context cannot "solve" the problem either. All they do is exhibit a context where the "smeared clouds of continuous charge distributions" interpretation is not applicable. Of course, that naive "smeared" interpretation is almost never a good interpretation, even so smeared charge distributions do occur in the density functional theory.

The dots on the photoplate are the empirical facts, and your mathematical framework of the theory supposed to describe these facts together with the "interpretation" of this framework has to be consistent with these observations. Indeed, the distribution of many equally prepared electrons on the photoplate is given by $|\psi(t,\vec{x})|^2$, the solution of Schrödinger's equation, but the single electron is not detected as a smeared charge distribution but as a dot. This apparent contradiction (known as the "wave-particle dualism" in the old quantum theory) is resolved by Born's interpretation of $|\psi(t,\vec{x})$ as the probability density for the position of the electron, when it is detected at time $t$. No observations so far contradict this interpretation and it has nobody come up with a satisfactory alternative description. That's why I consider the statistical interpretation of the quantum state a la Born as the solution of the wave-particle-dualism as well as the unique-outcome problem. The unique outcome of measurements, given appropriate measurement devices, are an empirical fact too, and again all there is according to the best currently available theory, QT, are the probabilities for these outcome as predicted by this theory.

My overall problem with this "strange style" of discussion is that it makes it difficult for me to dive into subtle issues in details. For example, the discussion has now touched topics like "conservation laws event by event" or that "photons are the entities resembling particles the least". As we have seen in discussions with RUTA, boundary conditions can sometimes prevent conservation laws from holding event by event, while still enforcing conservation laws for the quantum expectations.

I don't know, what you are referring to here. The conservation laws just say that conserved quantities are conserved. If the system is prepared in an eigenstate of some conserved quantities, then the state at later times stays in such an eigenstate, because the Hamiltonian commutes with the correspond operators representing these conserved quantities. That's a mathematical property of the theory and tells you the event-by-event conservation of conserved quantities. I don't know, which "boundary conditions" you are talking about.

Also with respect to solving the "unique outcome problem" by "postulating specific (reasonable) axioms," there would be interesting subtle details: What sort of postulate would that be? Would it be more of the "2+2=4" type, the "ZFC is consistent" type, or rather the "PA is consistent" type? You see, "2+2=4" is simply true, and "ZFC is consistent" is simply undecidable. But "PA is consistent" is subtle. It is true, it can be proved true in at least three fundamentally different informal ways, but it must be postulated nevertheless.

I don't know, what the fundamental axioms of math have to do with this problem. QT is formulated in terms of standard functional analysis.

Maybe more fundamental, I would have found it nice to clarify A. Neumaier's view on Ensembles in quantum field theory. I don't even understand why he thought that you have to "repeatedly prepare a quantum field extending over all of spacetime" in order to use an ensemble interpretation of QFT. But if already the fact that an ensemble interpretation is not universally applicable is never acknowledged, not even in simple examples, then this makes it difficult for me to dive into such subtle issues.

I've also no clue. Just look at, where (relativistic) QFT is applied to experiments, and you see that the ensemble interpretation perfectly fits. It has a good reason that the LHC and all of its big detectors were upgraded for more luminosity enabling "more statistics"!

 

[gentzen]

Indeed, the distribution of many equally prepared electrons on the photoplate is given by $|\psi(t,\vec{x})|^2$, the solution of Schrödinger's equation, but the single electron is not detected as a smeared charge distribution but as a dot.

The modern formalism assigns physical meaning to quantities that can be associated to self-adjoint measurement operators. As long as no qualification on the physically relevant measurement operators (spin, energy, momentum, ...) and the physically relevant quantities (probabilities for single outcomes, expectation values, ...) are added, this remains a pure description of a mathematical framework (like ordinay differential equations are the framework for classical point particle physics, partial differential equations are the framework for classical field theories, ...), to which the physical content must still be added.

No observations so far contradict this interpretation and it has nobody come up with a satisfactory alternative description.

And still this brute force interpretation might miss some subtle details, like I tried to illustrate above.

(Edit: I remember a huge table from "Quantenchemie: Eine Einführung" by Michael Springborg, where many relevant physical interpretations of quantities occurring in quantum chemistry computations were given. If I can find it again, I will include it in another reply.)

I don't know, what you are referring to here.

The discussions with RUTA about conservation on average only

I don't know, which "boundary conditions" you are talking about.

The easiest way to break a symmetry of the equations of motion is by having physically relevant boundary conditions that are incompatibe with that symmetry, which would have given rise to the conserved quantity. The example from RUTA seem to be easiest explained in terms of such boundary conditions, from my point of view. Of course, symmetries can also be broken for other reasons (at least in classical continuum physics), but those reasons did not apply to his examples, from my point of view.

I don't know, what the fundamental axioms of math have to do with this problem. QT is formulated in terms of standard functional analysis.

They illustrate the subtleties which can still remain despite the agreement that axioms or postulates are indispensible.

I've also no clue. Just look at, where (relativistic) QFT is applied to experiments, and you see that the ensemble interpretation perfectly fits. It has a good reason that the LHC and all of its big detectors were upgraded for more luminosity enabling "more statistics"!

If you always say "it is simple," or "I don't understand your problem," or "...", then this might be harmless as long as your conversation partner is right anyway and doesn't need your input. But you get him into trouble in the occasional cases where he is wrong, and would have benefitted from you input to see this for himself.

 

[PeterDonis]

As we have seen in discussions with RUTA, boundary conditions can sometimes prevent conservation laws from holding event by event

This is a claim RUTA makes, but I don't think it is generally accepted.

 

[vanhees71]

I don't know anybody else, who claims this.

 

[gentzen]

This is a claim RUTA makes, but I don't think it is generally accepted.

I think RUTA wants to make more general claims than that. By "conservation laws holding event by event," I explicitly don't mean the quantum intermediate results, but only the measured classical results. If you measure those classical results in a configuration that they have no chance of both satisfying the quantum expectations, and the conservation laws of the classical results for the specific event, then the quantum expectations will "win".

People will use different words to describe this situation, but they will probably agree on what quantum mechanics predicts to happen in that situation.

 

[PeterDonis]

By "conservation laws holding event by event," I explicitly don't mean the quantum intermediate results, but only the measured classical results.

But the measured classical results are only for the measured system. They don't take into account conserved quantities possessed by the measuring apparatus. This means the measured classical results are useless for evaluating conservation laws, because during the measurement the measured system interacts with the measuring apparatus, so the measured system is not a closed system and you should not expect it to satisfy conservation laws in isolation.

If you measure those classical results in a configuration that they have no chance of both satisfying the quantum expectations, and the conservation laws of the classical results for the specific event, then the quantum expectations will "win".

Of course the results will match QM predictions, that's been established by countless experiments. But, as above, that's irrelevant to assessing conservation laws.

 

[gentzen]

But the measured classical results are only for the measured system. They don't take into account conserved quantities possessed by the measuring apparatus.

Agreed. And the measuring apparatus itself is not modeled on the quantum level, it only enters in the form of "boundary conditions" (or some more appropriate word).

This means the measured classical results are useless for evaluating conservation laws, because during the measurement the measured system interacts with the measuring apparatus, so the measured system is not a closed system

Indeed, the idea behind a closed system would be that you don't have to worry about "boundary conditions". But if you perform measurements on the system, then it surely is not a closed system.

and you should not expect it to satisfy conservation laws in isolation.

Indeed, I don't expect this. The violation of conservation laws event by event (in such a situation) is much less surprising than the preservation of conservation laws for the quantum expectations.

 

[PeterDonis]

The violation of conservation laws event by event

To be clear, I don't think there is any actual violation of conservation laws--it's just that if you only look at the measured system, you aren't taking into account all the relevant conserved quantities.

 

[bhobba]

To be clear, I don't think there is any actual violation of conservation laws--it's just that if you only look at the measured system, you aren't taking into account all the relevant conserved quantities.

If there were, there would be something wrong with Noether.

Thanks
Bill

 

[gentzen]

(Edit: I remember a huge table from "Quantenchemie: Eine Einführung" by Michael Springborg, where many relevant physical interpretations of quantities occurring in quantum chemistry computations were given. If I can find it again, I will include it in another reply.)

I found the huge table again:

$n_R$	$n_E$	$n_B$	$n_\Sigma$	Relevanz für:
0	0	0	0	Gesamtenergie
1	0	0	0	Kräfte; Strukturoptimierung
0	1	0	0	Elektrisches Dipolmoment
0	0	1	0	Magnetisches Dipolmoment
0	0	0	1	Hyperfeinstruktur
2	0	0	0	Harmonische Schwingungsfrequenzen und -Moden
0	2	0	0	Elektrische Polarizabilität
0	0	2	0	Magnetische Polarizabilität
0	0	0	2	Kopplung von Spins verschiedener Kerne
1	1	0	0	Infrarot Intensitäten
0	1	1	0	Circulardichroismus
3	0	0	0	Anharmonische Korrekturen zu Schwingungsfrequenzen
0	3	0	0	Erste elektrische Hyperpolarisierbarkeit
1	2	0	0	Raman Intensitäten
4	0	0	0	Anharmonische Korrekturen zu Schwingungsfrequenzen
0	4	0	0	Zweite elektrische Hyperpolarisierbarkeit

Of course, this immediately raises the questions of the meaning of those $n_R$, $n_E$, $n_B$ and $n_\Sigma$ in the header or rather their integer numbers in the rows of the table. The book explains it as follows:

14.15 Experimental quantities​

Many quantities measured in experiment can also be theoretically determined. In particular, the dependences of the total energy on the structure, the spin of the nuclei, and the components of electric and/or magnetic field vectors are important, i. e., quantities like
$$\frac{\partial E^{n_R+n_E+n_B+n_\Sigma}}{\partial R^{n_R} \partial \mathcal{E}^{n_E}\partial \mathcal{B}^{n_B}\partial \Sigma^{n_\Sigma}}.\qquad(14.91)$$
This notation implies that the experimentally relevant quantities are determined by the $n_R$-, $n_E$-, $n_B$-, and $n_Σ$-fold derivatives of the total energy $E$ with respect to the nuclear coordinates, the vector components of the electric field, the vector components of the magnetic field, and the components of the nuclear spin.

(So the table interprets quantities arising in the context of eigenvalue computations. Which makes sense, because those are related to "quasi-equilibrium properties," which are often more relevant than "scattering properties" in chemistry.) The caption of the table then references back to that description in the text:

Table 14.2: Some experimentally determinable quantities obtained by means of derivatives of the
total energies of the type of equation (14.91).

$n_R$	$n_E$	$n_B$	$n_\Sigma$	Relevance
0	0	0	0	Total energy
1	0	0	0	Forces: structural optimization
0	1	0	0	Electric dipole moment
0	0	1	0	Magnetic dipole moment
0	0	0	1	Hyperfine structure
2	0	0	0	Harmonic vibrational frequencies and modes
0	2	0	0	Electric polarizability
0	0	2	0	Magnetic susceptibility
0	0	0	2	Coupling of spins of different nuclei
1	1	0	0	Infrared intensities
0	1	1	0	Circular dichroism
3	0	0	0	Anharmonic corrections to vibrational frequencies
0	3	0	0	First electrical hyperpolarizability
1	2	0	0	Raman intensities
4	0	0	0	Anharmonic corrections to vibrational frequencies
0	4	0	0	Second electrical hyperpolarizability

 

[Demystifier]

That is still a single outcome.

Unless each of them realizes in another "world".

 

[Demystifier]

I don't understand the issue with single outvomes. How can there be anything else? What is a multiple outcome?

The issue is not whether the outcome is single, because it clearly is (except in the many world interpretation), but how QM explains single outcomes. The standard QM avoids a need for explanation because it postulates that a single outcome appears when a measurement is performed. However, the measurement itself in standard QM is usually not described in terms of something more elementary. Instead, measurement is taken as a primitive notion that does not need to be precisely defined. It works fine in practice, but it's not satisfying from a deeper point of view where one wants to introduce a wave function of the measuring apparatus. But when one does that, then wave function of the apparatus and of the measured system should a priori be treated on an equal footing, and from this point of view it is not at all obvious what's special about measurement so that single outcomes appear.

 

[Demystifier]

No, quantum mechanics implies conservation laws event by event.

That's true when both initial and final value of the conserved quantity is measured. But if the initial state is in the superposition of different values, while final measured state has a definite value, then the initial mean value is not equal to the final definite value.

 

[PeterDonis]

if the initial state is in the superposition of different values, while final measured state has a definite value, then the initial mean value is not equal to the final definite value.

In general this is true, but it doesn't mean conservation laws are violated. It just means the system is not a closed system--it interacts with the measuring apparatus during measurement. If a system is not closed, you should not expect it to obey conservation laws in isolation.

 

[vanhees71]

Of course, event-by-event conservation means that you must have a state, for which the conserved quantity takes a definite value.

 

[PeterDonis]

event-by-event conservation means that you must have a state

You will if you include everything that interacts, which means including the measuring apparatus. As @Demystifier points out, we don't currently have a formulation of QM that does that and also explains (instead of just postulating) single outcomes; that means we don't currently have a formulation of QM that allows us to test conservation laws during measurements.

 

[Demystifier]

In general this is true, but it doesn't mean conservation laws are violated. It just means the system is not a closed system--it interacts with the measuring apparatus during measurement. If a system is not closed, you should not expect it to obey conservation laws in isolation.

But the full system, namely measured system and measuring apparatus, is closed. And yet, the final definite energy of the full closed system is not equal to its initial mean energy.

Sure, the conservation law is not violated if one does not compare mean values with definite values. But if one insists that energy is conserved even in this case, then one must accept that the initial state had a definite energy different from the initial mean value, which is tantamount to accepting the existence of hidden variables.

 

[A. Neumaier]

But the full system, namely measured system and measuring apparatus, is closed.

No. it always iteracts with its environment.

 

[Demystifier]

No. it always iteracts with its environment.

Then the full system, namely measured system, apparatus and environment, is closed, and everything I said before applies to this full system.

 

[PeterDonis]

Then the full system, namely measured system, apparatus and environment, is closed, and everything I said before applies to this full system.

Not necessarily; that full system might be in an eigenstate of whatever conserved quantity you are assessing, even if subsystems of it are not.

 

[A. Neumaier]

Then the full system, namely measured system, apparatus and environment, is closed

This full system is the whole universe! How do you propose to measure it?

 

[Demystifier]

Not necessarily; that full system might be in an eigenstate of whatever conserved quantity you are assessing, even if subsystems of it are not.

I would say that the full system cannot be in the full Hamiltonian eigenstate. For if it was, the full wave function would have a trivial time dependence proportional to $e^{-iE_{\rm full}t/\hbar}$, so no decoherence or any other change could happen due to the Schrodinger evolution, which would correspond to a totally dull universe very unlike our own. (BTW such a dull universe is also related to the problem of time in quantum gravity, but that's another issue.)

 

[Demystifier]

This full system is the whole universe! How do you propose to measure it?

By neglecting the influence of environment. In real quantum optics experiments, a measured system is often very well isolated from the environment. The apparatus, of course, is not isolated from the environment, but the exchange of energy between apparatus and environment has a negligible influence on the measured system.

 

[A. Neumaier]

The apparatus, of course, is not isolated from the environment, but the exchange of energy between apparatus and environment has a negligible influence on the measured system.

This is not true.

The interaction of the apparatus with the environment is the essential ingredient in the derivation of decoherence properties for the measured system + apparatus!!!

 

[vanhees71]

But for sure, you don't need the "entire universe", whatever this unobservable entity might be, for that. Also the decoherence of the observed system through interaction with the measurement apparatus is suffcient too.

 

[Demystifier]

The interaction of the apparatus with the environment is the essential ingredient in the derivation of decoherence properties for the measured system + apparatus!!!

No. Since apparatus itself has many degrees of freedom, it is the apparatus that serves as "environment" needed for decoherence of the measured system. The air and other particles around apparatus is not important.

EDIT: Now I saw that @vanhees71 said the same.

 

[Fra]

but how QM explains single outcomes
...
measurement is taken as a primitive notion that does not need to be precisely defined
...
then wave function of the apparatus and of the measured system should a priori be treated on an equal footing, and from this point of view it is not at all obvious what's special about measurement so that single outcomes appear.

I think solving this problem, requires nothing less than understanding the dual perspectives of dynamics as described by a hamiltonian, and the "evolution" describes by agents making measurements on each other. This is exactly the heart of the problem.

One can choose between two problems
(1) complain that the measurement is a primitive notion, but accept an unexplained finetune hamiltonian and hope to have it explain the illustion of measuerment/collapse in some limite

(2) complain that the hamiltonian is not intrinsic, but accept the measurement as primitive and hope to explain the illustion of a hamiltionin in some limit

I choose the second problem because I see a better chance of solving it over the other one.

/Fredrik

 

[Demystifier]

I think solving this problem, requires nothing less than understanding the dual perspectives of dynamics as described by a hamiltonian, and the "evolution" describes by agents making measurements on each other.

So the evolution of agents is not described by Hamiltonian, right? Do you have some concrete model of agent evolution in mind?

 

[Fra]

So the evolution of agents is not described by Hamiltonian, right? Do you have some concrete model of agent evolution in mind?

The simple answer would be yes, if by agent we refer to the intrinsic perspective, it's not described by a fixed Hamiltonian simply because a Hamiltonian itself implicitly encodes information that bypassed proper inference. Instead the evolution of the agent from it's own perspective is simply an evolutionary learning process.

Still I imagine that one agent can have an emergent hamiltonian description, relative to another agent. This is the way in where must be a correspondence. The quest is to understand exactly how and which "hamiltonians" or that are likely abundant in nature.

The traditional method is differential equation based, which is a top down method, based on constraints. I largely consider an agent based model, which are commong in other fields, such as epidemology, social theory and economy. It's a bottom up model, where you simulate interactions of the part as per some sort of evolutionary rules, and see what population scale patterns that emerge. Predictions of these patterns from the model would add explanatory value.

Unless that basic ABM oncept is clear already see this paper (not specific to physics though!)

Validation and Inference of Agent Based Models
"Agent-based model (ABM) is a simulation based modeling technique that aims to describe complex dynamic processes, such as the spread of infectious disease. ABM provides considerable flexibility by explaining the complex dynamic process using simple rules that incorporate characteristics of individual
entities, called agents, and their interactions. Thus, mechanisms which are often difficult or impossible to model directly at the population level can be incorporated at the smaller scale into the development of ABM [Hooten and Wikle, 2010, Grimm and Railsback, 2013]. ABM can capture emergent phenomena resulting from the interactions of agents, and can be used to simulate counterfactual outcomes in hypothetical experiments which are impossible or unethical to conduct in the real world."
-- https://arxiv.org/abs/2107.03619

The idea is simple; simulate interactions of "agents", and identify from the emergent phenomena, emergent dynamical laws. Here the ABM model, has the potential to "explain" what in equation based modelling are constraints as emergent.

I certainly don't have any concrete models that reproduce the standardmodel in a unified manner. Even though that is obviously the goal. If I find it, i promise to publish it ;)

These ideas are not without problems though, but I prefer thoese problems over the alternative.

I have only some concrete toy models, but these are obiously personal theories at this point and even besides off limits here. I need to work on this alot more before I will officially publish it anywhere. But the general guiding principles can I think be understood without the details.

/Fredrik