Statistical ensemble interpretation done right

[PeterDonis]

This is again splitting hairs.

No, it isn't. It is, once again, observing that you refuse to be consistent in your use of terms.

I only said that it is a logical possibility to assume that indeed not all observables take always predetermined values. Then QT, in the minimal statistical interpretation, can be "considered complete".

Which, again, is not the minimal interpretation. The minimal interpretation does not say anything about "logical possibilities" or whether QT can be "considered complete". It is silent on all such matters.

It is very confusing to me that you, who have repeatedly claimed that you have no interest in interpretation discussions, continue to post in them, without, apparently, even understanding your own viewpoint.

I did not claim that QT were complete.

Really?

If you simply accept that QT is a complete description

I wonder if you even read what you post.

 

[Lynch101]

Which problems? If you simply accept that QT is a complete description, then you must simply accept the "irreducible randomness", and there are no more problems, which only arise, because you claim that QT were incomplete, because you think that Nature must be deterministic, and thus only a deterministic theory can be complete, i.e., the only "allowed randomness" is due to ignorance, as in classical statistical physics.

Of course, you can never prove that any theory is a complete description, because it may well be that you find some empirical evidence that clearly contradicts the theory, and then you need to find a new theory.

In the case of QT it's also clear that it's intrinsically incomplete, because there's no satisfactory theory of the gravitational interaction compatible with it, but this may or may not be related to the probabilistic Nature of QT, i.e., whether a more comprehensive theory, including gravitation, will be deterministic again or not, one cannot say without having found this theory.

What is it a complete description of though? Is it a complete description of a statistical sample (or "ensemble") or is it a complete description of each individual element of the statistical sample (or "ensemble")?

 

[PeterDonis]

What is it a complete description of though? Is it a complete description of a statistical sample (or "ensemble") or is it a complete description of each individual element of the statistical sample (or "ensemble")?

Since this thread is about the statistical ensemble interpretation, we are discussing here interpretations of the first type (with the caveat that "ensemble" is not the same thing as "statistical sample"--this was discussed earlier in the thread). However, AFAIK not all versions of the statistical ensemble interpretation claim that the description given by QT of the statistical ensemble is complete.

 

[Lynch101]

Since this thread is about the statistical ensemble interpretation, we are discussing here interpretations of the first type (with the caveat that "ensemble" is not the same thing as "statistical sample"--this was discussed earlier in the thread). However, AFAIK not all versions of the statistical ensemble interpretation claim that the description given by QT of the statistical ensemble is complete.

Cheers PD. I read through the thread alright and saw the discussion about the use of the term "ensemble". I put it in brackets and quote marks moreso because Vanhees uses the term in place of statistical sample.

To what extent would you say the SEI (or perhaps some expressions of it) are actually interpretations as distinct from "shut up an calculate"?

 

[PeterDonis]

I put it in brackets and quote marks moreso because Vanhees uses the term in place of statistical sample.

And, as I said, that has been corrected in previous discussion. The quantum state in the statistical ensemble interpretation represents, as the name says, the ensemble (with the definition given in Ballentine that I posted earlier), which, as I said, is not the same thing as the finite statistical sample that we actually get from running experiments.

To what extent would you say the SEI (or perhaps some expressions of it) are actually interpretations as distinct from "shut up an calculate"?

This question is unanswerable. If you have a specific reference from the literature or a specific post earlier in this thread, you can ask about that.

 

[vanhees71]

What is it a complete description of though? Is it a complete description of a statistical sample (or "ensemble") or is it a complete description of each individual element of the statistical sample (or "ensemble")?

Of course, probabilistic notions make only sense for statistical samples (as proxies of an ensemble). That's very accurate lnguage, and we agreed to use it in this thread. In everyday discussions among physicsts (particularly experimentalists) "ensemble" is the usual lingo. Nobody talks about "statistical samples", but that's admittedly imprecise and may lead to confusion when it comes to fine details about interpretation.

 

[Lord Jestocost]

“The standard empirical interpretation of quantum mechanics is already statistical. However, a statistical (ensemble) interpretation can also be treated as a semantic interpretation which provides an understanding of empirical data. In contrast to the standard (Copenhagen) interpretation, the statistical interpretation does not refer to an individual object but it refers to a collective (ensemble) of similarly prepared ones.”

Alexander Pechenkin in “The Statistical (Ensemble) Interpretation of Quantum Mechanics” (Chapter 50 of “The Oxford Handbook of the History of Quantum Interpretations”, Oxford University Press (2022))

 

[Lynch101]

Since this thread is about the statistical ensemble interpretation, we are discussing here interpretations of the first type (with the caveat that "ensemble" is not the same thing as "statistical sample"--this was discussed earlier in the thread). However, AFAIK not all versions of the statistical ensemble interpretation claim that the description given by QT of the statistical ensemble is complete.

In those versions of the SEI which claim that the description of the statistical ensemble (given by QT) is complete, do they say anything about the process by which the statistical ensemble becomes populated, as in, the process whereby the individual elements of the ensemble come to be part of the ensemble?

Am I using the correct terminology when I say that the finite statistical sample used in experiments, acts as a proxy for testing the predictions QT makes with regard to an abstract ensemble?

The thing I'm trying to get at is, I know there is an experimental process which gives rise to the statistical sample. I'm just wondering if the aforementioned (or indeed any) versions of SEI describe both the process and the ensemble (where the statistical sample acts as a proxy for the ensemble), or does it just describe the ensemble?

 

[vanhees71]

In those versions of the SEI which claim that the description of the statistical ensemble (given by QT) is complete, do they say anything about the process by which the statistical ensemble becomes populated, as in, the process whereby the individual elements of the ensemble come to be part of the ensemble?

In the minimal interpretation the quantum state (mathematically represented by the statistical operator of the system under consideration) describes a preparation procedure on the individual system. The ensemble is given by infinitely many equally and independently prepared individual systems.

Am I using the correct terminology when I say that the finite statistical sample used in experiments, acts as a proxy for testing the predictions QT makes with regard to an abstract ensemble?

That's right. As with any probaiblistic prediction you have to statistically estimate the error/statistical significance of the estimate of the probability due to the finite sample. In addition you also have systematic uncertainties, which also have to be carefully analyzed.

The thing I'm trying to get at is, I know there is an experimental process which gives rise to the statistical sample. I'm just wondering if the aforementioned (or indeed any) versions of SEI describe both the process and the ensemble (where the statistical sample acts as a proxy for the ensemble), or does it just describe the ensemble?

Theoretical physicists are concerned with the predictions according to QT, i.e., the properties of the abstract, idealized ensemble. The experimentalists then have to figure out, how to measure it and also do the statistical and systematic analysis :-).

 

[Lynch101]

In the minimal interpretation the quantum state (mathematically represented by the statistical operator of the system under consideration) describes a preparation procedure on the individual system. The ensemble is given by infinitely many equally and independently prepared individual systems.

I know this might seem like splitting hairs, but I'm just trying to get a more precise understanding of it. If there is more precise terminology I should use, please let me know. I'm striving to articulate my own thinking as clearly as possible.

Although I understand how they are intrinsically linked, I would distinguish between the following:

• the preparation procedure (using specific apparatus)
• the measurement procedure* (using other specific apparatus)
• the process between

*the measurement procedure results in specific measurement outcomes/observations which form the elements of the statistical sample.

If we were to say that the SEI describes ensemble, for which the statistical sample serves as a proxy, I would interpret that as meaning, it describes ratio of the different elements of the sample i.e. the ratio of one measurement outcome relative to another. For example: measurement outcomes where 50% spin up: 50% spin down.

If we were to say that the SEI describes the preparation procedure, I would interpret that as being distinct from a description of the statistical sample.

To use a classical analogy, the preparation procedure might be the preparation of fair coins, while the ensemble (approximated by the statistical sample) would be the outcomes of N-trials of flipping those fair coins.

The process in between then would be the process by which the coins are flipped.

Does that make sense?

 

[PeterDonis]

In those versions of the SEI which claim that the description of the statistical ensemble (given by QT) is complete, do they say anything about the process by which the statistical ensemble becomes populated, as in, the process whereby the individual elements of the ensemble come to be part of the ensemble?

The ensemble does not get "populated" since it does not refer to any actual runs of the preparation procedure or individual systems produced by it. It refers to the abstract (infinite) set of all possible systems that can be produced by the preparation procedure. (See, for example, Ballentine, which I believe I've referred to earlier in this thread.)

Am I using the correct terminology when I say that the finite statistical sample used in experiments, acts as a proxy for testing the predictions QT makes with regard to an abstract ensemble?

We use a finite statistical sample to test the QT predictions, yes. But we do that whether we're using an ensemble interpretation or not.

To use a classical analogy, the preparation procedure might be the preparation of fair coins, while the ensemble (approximated by the statistical sample) would be the outcomes of N-trials of flipping those fair coins

No, the ensemble would not be that. See above.

 

[Lynch101]

The ensemble does not get "populated" since it does not refer to any actual runs of the preparation procedure or individual systems produced by it. It refers to the abstract (infinite) set of all possible systems that can be produced by the preparation procedure. (See, for example, Ballentine, which I believe I've referred to earlier in this thread.)

Ah yes, of course. I think I had it clearer in my mind in my subsequent post. Is the terminology I'm using accurate, if I talk about the the statistical sample being populated by individual elements?

We use a finite statistical sample to test the QT predictions, yes. But we do that whether we're using an ensemble interpretation or not.

Thanks PD, I'm clear on that.

No, the ensemble would not be that. See above.

Am I right in saying the statistical sample would be the outcomes of N*-trials of flipping those fair coins, while the ensemble would be the, as you mention, abstract infinite set.

*The use of N implying a finite set.

 

[PeterDonis]

Am I right in saying the statistical sample would be the outcomes of N*-trials of flipping those fair coins, while the ensemble would be the, as you mention, abstract infinite set.

Yes.

 

[vanhees71]

I know this might seem like splitting hairs, but I'm just trying to get a more precise understanding of it. If there is more precise terminology I should use, please let me know. I'm striving to articulate my own thinking as clearly as possible.

Although I understand how they are intrinsically linked, I would distinguish between the following:

• the preparation procedure (using specific apparatus)
• the measurement procedure* (using other specific apparatus)
• the process between

*the measurement procedure results in specific measurement outcomes/observations which form the elements of the statistical sample.

If we were to say that the SEI describes ensemble, for which the statistical sample serves as a proxy, I would interpret that as meaning, it describes ratio of the different elements of the sample i.e. the ratio of one measurement outcome relative to another. For example: measurement outcomes where 50% spin up: 50% spin down.

If we were to say that the SEI describes the preparation procedure, I would interpret that as being distinct from a description of the statistical sample.

You have to define somehow the statistical sample in the lab. It's given by the preparation procedure. E.g., in a Bell experiment with photons it's given by a laser and a BBO crystal + some other optical equipment to get entangled photon pairs by spontaneous parametric down-conversion. Just shining long enough with your laser you get (in random temporal sequence) a sample of such prepared "Bell states" of photon pairs. Then you do measurements on this sample with outcomes that you can analyze statistically and compare it with the predictions of the model (QED).

To use a classical analogy, the preparation procedure might be the preparation of fair coins, while the ensemble (approximated by the statistical sample) would be the outcomes of N-trials of flipping those fair coins.

That sounds right.

The process in between then would be the process by which the coins are flipped.

The process in between is theoretically described by Newtonian mechanics of rigid bodies moving in the gravitational field of the Earth and subject to air resistance. The probabilistic description in this case comes just from the ignorance of the precise initial conditions. That's of course very different from the quantum probabilities of the above example with two photons. They are prepared (idealizing the real-world situation a bit) in a pure state, i.e., you have maximal possible knowledge of their state, but this doesn's imply that you know all observables. E.g., the polarization state of the single photons in this pair is maximally uncertain, i.e., the single photons are both simply unpolarized and described by a mixed state of maximum entropy.

Does that make sense?

Sounds good to me.

 

[Morbert]

To use a classical analogy, the preparation procedure might be the preparation of fair coins, while the ensemble (approximated by the statistical sample) would be the outcomes of N-trials of flipping those fair coins.

There is some ambiguity in the analogy here, insofar as whether or not you consider the flipping of the coin as part of the preparation or part of the measurement.

There is also some ambiguity in Ballentine's 1970s account. Ballentine, in his treatment of measurement (section 4.1), considers an initial state (equation 4.2) in the state space of the measured system, and a final state (equation 4.3) in the state space of the measured system + measurement apparatus. He then says the square of the amplitudes present in both expressions give the relative frequencies observed if the experiment is repeated "many times".

Home and Whitaker ( https://citeseerx.ist.psu.edu/viewd...E21C5C27?doi=10.1.1.675.655&rep=rep1&type=pdf ) review Ballentine's account. They say that these squared amplitudes are the relative frequencies "over the ensemble" of finding the measurement apparatus in the corresponding pointer states. Home and Whitaker invoke an infinite number of experimental runs, and our actually existing sample is more likely to reproduce the relative frequencies over the infinite ensemble, the larger our sample is. So using your analogy, it might be clearer to say.

i The preparation procedure is the flipping of a coin
ii The state represents an infinite ensemble of flipped coins
ii The measurement is the revealing of the coin face to some detection device
iv The possible outcomes are heads or tails
v The sample is actually existing outcome data compiled from a finite number of measurements

Statement ii is where a distinction between quantum and classical arises. If we are to quantise your analogy, instrumentalists like Asher Peres would replace "measurement" with "test". He would say something like

ii The test is the response of some detection device to the coin

Ballentine would be more agnostic, only committing himself to statistical statements about the data, whatever a datum might imply about a coin.

 

[Lynch101]

“The standard empirical interpretation of quantum mechanics is already statistical. However, a statistical (ensemble) interpretation can also be treated as a semantic interpretation which provides an understanding of empirical data. In contrast to the standard (Copenhagen) interpretation, the statistical interpretation does not refer to an individual object but it refers to a collective (ensemble) of similarly prepared ones.”

Alexander Pechenkin in “The Statistical (Ensemble) Interpretation of Quantum Mechanics” (Chapter 50 of “The Oxford Handbook of the History of Quantum Interpretations”, Oxford University Press (2022))

Just going back over the thread, and LJ's post helped to clarify a few things for me.

I'm not sure if this is the same for anyone else, but when it comes to the SEI, I think I have been guilty of trying to consider it in the context of the questions to which I, and others (I believe), are seeking answers.

Am I reading LJ's reference (and other posts in this thread) correctly when I say that, the SEI interprets the mathematics of quantum theory as a statistical description of the observed properties of an abstract, infinite set/ensemble of similarly prepared particles - against which observed statistical samples can be compared - and simply stops there? It is minimal in the sense that it states nothing more than what can definitively be implied by the mathematical formalism.

Where I have potentially been misinterpreting it, is in thinking that the SEI says there is nothing more to be explained, but (strictly speaking), does the SEI leave the door open for further explanation on issues such as:

• What are the properties of individual particles*?
• What is the process by which the statistical sample, in physical experiments, is populated by individual elements?
• Can anything further, beyond the minimal statistical description of an ensemble, be inferred from the mathematical formalism?

*Would "degrees of freedom" be a more accurate term here?

 

[Morbert]

From Home + Whitaker's review: "It is convenient to make an immediate comment concerning the relation between ensemble interpretations and hidden-variable theories. The existence of hidden variables would explain why one uses an ensemble interpretation; the ensembles would consist of systems with all possible distributions of values for the hidden variables. However use of an ensemble interpretation does not imply acceptance of hidden variables, to the possibility of which it remains neutral"

 

[Lord Jestocost]

Just going back over the thread, and LJ's post helped to clarify a few things for me.

I'm not sure if this is the same for anyone else, but when it comes to the SEI, I think I have been guilty of trying to consider it in the context of the questions to which I, and others (I believe), are seeking answers.

Am I reading LJ's reference (and other posts in this thread) correctly when I say that, the SEI interprets the mathematics of quantum theory as a statistical description of the observed properties of an abstract, infinite set/ensemble of similarly prepared particles - against which observed statistical samples can be compared - and simply stops there? It is minimal in the sense that it states nothing more than what can definitively be implied by the mathematical formalism.

Where I have potentially been misinterpreting it, is in thinking that the SEI says there is nothing more to be explained, but (strictly speaking), does the SEI leave the door open for further explanation on issues such as:

• What are the properties of individual particles*?
• What is the process by which the statistical sample, in physical experiments, is populated by individual elements?
• Can anything further, beyond the minimal statistical description of an ensemble, be inferred from the mathematical formalism?

*Would "degrees of freedom" be a more accurate term here?

The SEI leaves no new “door open” as compared to the orthodox interpretation of quantum mechanics.

To my mind, however, one should avoid the term "statistical ensemble interpretation" beccause a misleading reading of the term "ensemble" as a type of "Gibbs ensemble" can lead to enormous misunderstandings.

 

[Lynch101]

The SEI leaves no new “door open” as compared to the orthodox interpretation of quantum mechanics.

To my mind, however, one should avoid the term "statistical ensemble interpretation" beccause a misleading reading of the term "ensemble" as a type of "Gibbs ensemble" can lead to enormous misunderstandings.

No new door, no.

But, and this could well be due to my own biases with which I was approaching my attempts to understand the SEI, I sometimes interpret statements made in relation to the SEI as claiming there is nothing else to be explained beyond the statistical description of the ensemble; or, in terms of it's relation to the physical experimental set-up, that there is nothing to be explained beyond the correspondence of the statistical sample to the predictions of QT.

 

[PeterDonis]

I sometimes interpret statements made in relation to the SEI as claiming there is nothing else to be explained beyond the statistical description of the ensemble; or, in terms of it's relation to the physical experimental set-up, that there is nothing to be explained beyond the correspondence of the statistical sample to the predictions of QT.

The "minimal" statistical interpretation more or less says that; but discussion in this thread is not limited to that version of the statistical interpretation.

 

[vanhees71]

Just going back over the thread, and LJ's post helped to clarify a few things for me.

I'm not sure if this is the same for anyone else, but when it comes to the SEI, I think I have been guilty of trying to consider it in the context of the questions to which I, and others (I believe), are seeking answers.

Am I reading LJ's reference (and other posts in this thread) correctly when I say that, the SEI interprets the mathematics of quantum theory as a statistical description of the observed properties of an abstract, infinite set/ensemble of similarly prepared particles - against which observed statistical samples can be compared - and simply stops there? It is minimal in the sense that it states nothing more than what can definitively be implied by the mathematical formalism.

I think that's correct, at least that's how I also understand the minimal statistical interpretation.

Where I have potentially been misinterpreting it, is in thinking that the SEI says there is nothing more to be explained, but (strictly speaking), does the SEI leave the door open for further explanation on issues such as:

• What are the properties of individual particles*?

Before any empirical evidence indicates otherwise, QT in the minimal interpretation precisely describes the properties of individual particles. The inevitable consequence is that Nature is not deterministic, i.e., observables only take determined values if the particle is prepared in a corresponding state.

• What is the process by which the statistical sample, in physical experiments, is populated by individual elements?

The interaction between the measured system and the measurment device leads to an entanglement between the measured observable and the "pointer state" of the measurement apparatus. The outcome of the measurment is random with probabilities given by Born's rule.

• Can anything further, beyond the minimal statistical description of an ensemble, be inferred from the mathematical formalism?

I don't think so.

*Would "degrees of freedom" be a more accurate term here?

No. It's too unspecific. If you want to discuss particles, call them particles :-)). QT describes, however much more than single particles but rather everything except the gravitational interaction.

As with any theory this of course is subject to revision, as soon as empirical evidence provides new facts. That QT is incomplete is also clear, because it doesn't describe the gravitational interaction. It's not yet clear, how a more complete theory may look like and which revisions of all the issues discussed in conncection with the "foundations of QT" may be necessary.

 

[Lynch101]

The "minimal" statistical interpretation more or less says that; but discussion in this thread is not limited to that version of the statistical interpretation.

Just to clarify this a but further.

I would make the distinction between the following positions:

1. QT describes the statistical ensemble which is a complete description of "the physical reality" (EPR) of the experimental set-up, so there is nothing further to be described/explained.
2. QT describes the statistical ensemble but that is not a complete description of "the physical reality" (EPR) of the experimental set-up, so there is more to be described/explained.
3. QT describes the statistical ensemble but that is not a complete description of "the physical reality" (EPR) of the experimental set-up, so there is more to be described/explained. However, no further explanation is possible.

Which of those would you say corresponds to the SEI. Or would there be a more precise way of articulating it?

 

[Lynch101]

The interaction between the measured system and the measurment device leads to an entanglement between the measured observable and the "pointer state" of the measurement apparatus. The outcome of the measurment is random with probabilities given by Born's rule.

Ah, OK. This seems like a fairly straight forward picture, if I am interpreting it correctly.

To use a pretty crude classical analogy. If we imagine a microscopic sphere* prepared by some preparation procedure, which travels from the preparation device to the detector via a polarising filter or magnetic filed (as appropriate). Whether or not it passes the filter or is deflected up/down is simply random.

Would that be in the right direction in terms of understanding?

*Sphere is used here more as a placeholder.

 

[Morbert]

How are you modelling the "microscopic sphere"? With a hidden variable state $\lambda$?

[edit to add] - You have to be careful about asserting properties of a microscopic system independent of measurement.

 

[Lynch101]

How are you modelling the "microscopic sphere"? With a hidden variable state$\lambda$?

No, no hidden variables.

My understanding is that hidden variables (with a hidden variable state $\lambda$) would mean it is pre-determined whether or not the "sphere" will pass through the filter, or be deflected up/down as it passes through a magnetic field. Is that accurate?

However, without such hidden variables, whether or not the "sphere" will pass through the filter or be deflected up/down as it passes through a magnetic field, would simply be randomised.

 

[A. Neumaier]

Decoherent histories has been around for a good few decades at this stage, with one motivation for its development being the description of closed systems, and measurements as processes therein.
https://iopscience.iop.org/article/10.1088/1742-6596/2533/1/012011/pdf

It gives a clear account of what it means for a measurement to occur in a closed system.

No. It only gives an account of events ''that we can talk about at the breakfast table'' (according to the above paper) - not of dynamical processes that would qualify as measurement processes.

In particular, their discussion assumes measurement results that fall from heaven, given by a POM or POVM in addition to the untary dynamics of the system, rather than taking the state of the universe and deriving from it the distribution of the values read from a macroscopic detector that is part of the dynamics.

Thus everything is empty talk embellishing Born's rule.

 

[Morbert]

No, no hidden variables.

My understanding is that hidden variables (with a hidden variable state $\lambda$) would mean it is pre-determined whether or not the "sphere" will pass through the filter, or be deflected up/down as it passes through a magnetic field. Is that accurate?

However, without such hidden variables, whether or not the "sphere" will pass through the filter or be deflected up/down as it passes through a magnetic field, would simply be randomised.

The issue isn't whether the sphere evolves stochasitcally or deterministically. The issue is the conceptualisation of the sphere itself. For example, if the sphere has some real state $\lambda$ then orthodox quantum interpretation is bijectively incomplete, insofar as there isn't a one-to-one correspondence between the real state of the microscopic system and the quantum state.

While it is ok to say a given ensemble interpretation frames QM as a complete theory of the statistics of ensembles, a discussion about a complete theory of reality requires further exploration of priors.

 

[martinbn]

For example, if the sphere has some real state $\lambda$ then...

And what if it doesn't?

 

[Morbert]

And what if it doesn't?

Then an orthodox interpretation could imply quantum theory is a complete physical theory, even of individual systems, while an ensemble interpretation would at least be that quantum theory is a complete theory of the statistics of ensembles.

 

[vanhees71]

How are you modelling the "microscopic sphere"? With a hidden variable state $\lambda$?

[edit to add] - You have to be careful about asserting properties of a microscopic system independent of measurement.

No, simply quantum mechanically. Instead of some ominous sphere, take a silver atom. Then you have Stern's and Gerlach's experiment of 1922.