Determinism, realism, hidden variables

[greypilgrim]

Hi.

I'm still confused about those three concepts. They sound pretty much the same to me, but this does not comply with Wikipedia's table of comparison of QM interpretations, where pretty much all combinations seem to be present.

Actually, is the "Wavefunction real?" column even about "realism" in the EPR sense, or is this something different?

 

[Heinera]

Hi.

I'm still confused about those three concepts. They sound pretty much the same to me, but this does not comply with Wikipedia's table of comparison of QM interpretations, where pretty much all combinations seem to be present.

Actually, is the "Wavefunction real?" column even about "realism" in the EPR sense, or is this something different?

I would say that "realism" and "hidden variables" are basically the same thing. "Determinism" is even stronger, since it is generally interpreted as that everything in the universe is predetermined from the outset. A hidden variable on the other hand can be randomly generated at the source, and Bell's theorem will still hold (Bell's theorem tends to be the only context where these concepts are used).

 

[naima]

You can see in this table that everytime you have hidden variables the theory is deterministic. At the exception of stochastic interpretation (maybe the values of the hidden variables could have a random origin)
But in all the cases if you knew the values of the hidden variables an algorithm would give you the result of the measurement.
QM with no hidden variables gives the correct probabilities. If you add hidden variables that give the same correct probabilities what does it add if it does not give the outcome?

 

[greypilgrim]

I would say that "realism" and "hidden variables" are basically the same thing.

What about counterfactual definiteness, is this the same as well? Its definition sounds quite a bit different to me ("the ability to speak meaningfully of the definiteness of the results of measurements that have not been performed"), but on the correspondent Wikipedia page they seem use it as a replacement for what is usually called realism in the Bell/EPR context.

 

[Ilja]

I'm still confused about those three concepts. They sound pretty much the same to me, but this does not comply with Wikipedia's table of comparison of QM interpretations, where pretty much all combinations seem to be present.
Actually, is the "Wavefunction real?" column even about "realism" in the EPR sense, or is this something different?

No, there can be realistic theories which interpret the wave function as epistemic, thus, describing only our incomplete knowledge of reality. Once the reality may be described by something different, like the configuration itself, this interpretation may be realistic.

The problem to distinguish between a deterministic theory and a realistic one with some random process guiding the evolution of reality I do not understand.

Counterfactual definiteness is, instead, the assumption that some particular variables have predefined values. Say, two criminals want to event a cover story to hide what they have done. They are separated, and asked separately. They give consistent answers. So the police assumes that the answers where predetermined. And they may think that these answers describe reality even if they have no independent witness accounts which tell anything about these claimed facts - this would be counterfactual definiteness. They will probably stop to think so if they observe that these criminals have had a hidden communication channel. But believing in this case in counterfactual definiteness has not much to do with believing in the existence of some reality, or with beliefs about fatalism vs. freedom of choice.

 

[Ilja]

Its definition sounds quite a bit different to me ("the ability to speak meaningfully of the definiteness of the results of measurements that have not been performed"), but on the correspondent Wikipedia page they seem use it as a replacement for what is usually called realism in the Bell/EPR context.

This is a very common error in the presentation of Bell's theorem. Counterfactual definiteness of the spin values is not assumed in Bell's theorem, but derived using the EPR argument: We see that whenever we measure at A and B the same direction, we obtain identical results. That means, we can predict, with certainty, the result at B for a direction a if we measure this direction at A. The EPR criterion of reality assumes that in this case, if this measurement does not disturb in any way the other measurement (where we use Einstein causality to show this), then that means that the result at B for a direction a has to be predefined even if we do not measure it. This is the counterfactual definiteness. But it is derived, and nothing general, but a result about this particular situation, and depending on the very nontrivial assumption of Einstein causality.

 

[Ilja]

QM with no hidden variables gives the correct probabilities. If you add hidden variables that give the same correct probabilities what does it add if it does not give the outcome?

It clarifies that all the mysticism justified on the base of quantum theory is nonsense. Like the rejection of realism, or of basic principles of causality like Reichenbach's common cause principle.

 

[Markus Hanke]

It clarifies that all the mysticism justified on the base of quantum theory is nonsense. Like the rejection of realism, or of basic principles of causality like Reichenbach's common cause principle.

I do not see any "mysticism" in established quantum theory at all, unless one insists that the universe must satisfy some misguided notion of classicality based purely on subjective human experience, and hence on what may or may not appeal to us philosophically and aesthetically. The only thing "mystic" here is the concept of ethereal, undetectable, unmeasurable "hidden variables" - you might as well postulate invisible pink unicorns, for all the scientific value it has.

That's just my personal opinion, from an interested amateur outside of professional academia.

 

[Demystifier]

I do not see any "mysticism" in established quantum theory at all, unless one insists that the universe must satisfy some misguided notion of classicality based purely on subjective human experience, and hence on what may or may not appeal to us philosophically and aesthetically. The only thing "mystic" here is the concept of ethereal, undetectable, unmeasurable "hidden variables" - you might as well postulate invisible pink unicorns, for all the scientific value it has.

That's just my personal opinion, from an interested amateur outside of professional academia.

The idea of Bohmian hidden variables in QM is analogous to the idea of dark matter in astrophysics. Neither of them can be directly observed, yet both have a strong explanatory value. Let me explain the analogy in a few more entries copy-pasted from one of my conference presentations:

Bohmian interpretation: deterministic particle trajectories guided by ψ.
- If it s true, then why trajectories cannot be observed?

Analogous to dark matter (astrophysics):
- If dark matter exists, then why it cannot be observed?

Both questions have a similar answer.

Indirect detection:
- sufficient that exists influence on something else (“detector”)

Direct detection:
- humans tend not to be absolutely convinced that something exists,
until they are able to detect the exact place where it exists.
⇒ need to know where does the influence comes from!

Non-dark matter (stars):
- we observe light from the object
- light is a wave ⇒ it has direction of propagation
⇒ easy to determine where does it come from
⇒ observation is direct

Dark matter:
- does not produce (or interact with) light
- observed by static gravitational field produced by dark matter
- static gravitational field does not have direction of propagation
⇒ cannot easily determine where does the field come from
⇒ observation is indirect
⇒ Indirect detection of dark matter is considered
less convincing than direct detection of non-dark matter.

Analogy with Bohmian particles:
- there is evidence for Bohmian particles (observations can be explained
by it, but there are also other explanations)
- non-local quantum potential similar to gravitational static potential
(does not have direction of propagation)
⇒ cannot easily determine where does potential come from
⇒ cannot easily determine position of Bohmian particle
⇒ evidence for Bohmian particles is only indirect

 

[Markus Hanke]

The idea of Bohmian hidden variables in QM is analogous to the idea of dark matter in astrophysics. Neither of them can be directly observed, yet both have a strong explanatory value.

You are making a very good point here - perhaps it is my own opinion that is misguided.
Probably best to shut up and keep learning so

 

[Ilja]

I do not see any "mysticism" in established quantum theory at all, unless one insists that the universe must satisfy some misguided notion of classicality based purely on subjective human experience, and hence on what may or may not appeal to us philosophically and aesthetically. The only thing "mystic" here is the concept of ethereal, undetectable, unmeasurable "hidden variables" - you might as well postulate invisible pink unicorns, for all the scientific value it has.

That's just my personal opinion, from an interested amateur outside of professional academia.

I do not see anything mystical in quantum theory too - we do not understand QT sufficiently good, that's all. But a rejection of the existence of the existence of some objective reality certainly qualifies as mysticism. And to postulate the existence of correlations which do not have a causal explanation is what we know from astrology, and also clearly qualifies as mysticism.

You can, of course, justifiy any mysticism by rejecting non-mystical views as "misguided notion of classicality based on purely subjective human experience". But this is nothing but cheap polemics. The "hidden variables" of realistic QT interpretations are, by the way, not at all undetectable and unmeasurable, but they are all what we detect and observe - classical configurations. At least I have not yet seen any wave function, I have always only seen quite classical configurations - which are the "hidden variables" of the "hidden variable" interpretations of quantum theory.

 

[naima]

It is not difficult to see in the Young experiment how the DBB trajectories are related to the 2 slits.
It is harder with a Stern Gerlach to see how the possible trajectories are related to the inital values of the spin.
No intuitive thing comes to me from this interpretation

 

[Demystifier]

It is harder with a Stern Gerlach to see how the possible trajectories are related to the inital values of the spin. No intuitive thing comes to me from this interpretation

Perhaps this can help
http://arxiv.org/abs/1305.1280
To develop intuition, it may be sufficient to look at the figures.

 

[Markus Hanke]

I have always only seen quite classical configurations - which are the "hidden variables" of the "hidden variable" interpretations of quantum theory.

This is your interpretation of it, but surely you must see that the standard probabilistic interpretation without hidden variables explains these phenomena just as well. I grant you that the standard interpretation may not appeal to you, but that does not outright invalidate it.
I am prepared to concede that I must remain open to at least the possibility of some form of hidden variable interpretation as I continue to learn about QT, but I think this has to go both ways - people also need to be open to the idea that perhaps, just perhaps, the universe quite simply is not classical and deterministic at its core. Personally, I see no issue with this.

 

[Ilja]

This is your interpretation of it, but surely you must see that the standard probabilistic interpretation without hidden variables explains these phenomena just as well.

No. It gives rules how to compute probabilities, but explains nothing. I grant you that rules to compute probabilities are important, from an instrumental point of view, even without such explanations. But there is no reason to name something an explanation which explicitly refuses to give an explanation.

... but I think this has to go both ways - people also need to be open to the idea that perhaps, just perhaps, the universe quite simply is not classical and deterministic at its core. Personally, I see no issue with this.

It is, in principle, imaginable that this universe is only a wild dream and what happens does not have any explanations. But up to now it does not look like this, the idea that there is an external reality, which we can understand, that the correlations we observe have causal explanations, has been sufficiently successful during the last centuries of science. And actually there is not even a problem worth to be mentioned to cause doubt, we have nice realistic and causal interpretations.

 

[DrChinese]

Hi.

I'm still confused about those three concepts. They sound pretty much the same to me, but this does not comply with Wikipedia's table of comparison of QM interpretations, where pretty much all combinations seem to be present.

You will find both sides of the coin expressed: there are plenty of physicists who use these terms interchangeably in conjunction with Bell's Theorem. That is especially true of the terms "realism" and "hidden variables".

And there are others who tend to see distinctions. There are even papers that explore these differences. Most of those turn on semantics, and there is not a general consensus. So some of it comes down to personal choice. I personally consider both of the following to be fair restatements of Bell's Theorem:

"No physical theory of local hidden variables can ever reproduce all of the predictions of quantum mechanics."

"No local realistic theory can ever reproduce all of the predictions of quantum mechanics."

Further, I don't consider "counterfactual definiteness" to be different in any meaningful manner than the term "realism". Again, not everyone agrees on this. The important thing is for you to understand how Bell's Theorem works first. Then, later, look at the semantics.

 

[Ilja]

Further, I don't consider "counterfactual definiteness" to be different in any meaningful manner than the term "realism". Again, not everyone agrees on this.

I don't, and I would say it is clearly false. In particular, dBB theory is clearly realistic, and even deterministic. it specifies what really exists, with evolution equations, and all what described classical reality (the trajectory in configuration space) is part of reality too.

But does not have any counterfactual definiteness: except for position measurements and eigenstates of the measured operator the "measurement result" depends also on the state of the "measurement device". So, "unperformed measurements" do not have a predetermined result in dBB theory.

 

[Markus Hanke]

It gives rules how to compute probabilities, but explains nothing.

But what if those probabilistic rules are the explanation ? What if the universe actually does play dice ? Why rule out that possibility, if it fits experiment and observation so well ? You bring up Reichenbach's principle in this context, so, having had only superficial knowledge of it, I did a bit of reading about this - it seems that, firstly, the exact meaning and definition of the principle itself is subject to some debate ( i.e. there seem to be several versions of it ), and, secondly, that there is no general consensus as to whether or not the principle even applies to the case of QT, which exhibits certain differences to classical probability theory. I have found both authors arguing in favour of and against the applicability of the principle to QT. If anything, my impression is that the majority of authors lean more towards non-applicability, or leave the question open to further research. I have found this summary, and the examples given therein, quite interesting :

http://plato.stanford.edu/entries/physics-Rpcc/

This was also an interesting read, though I must admit that a lot of it is over my head :

http://arxiv.org/pdf/quant-ph/9805066v1.pdf

The impression I get from all this reading is that the applicability of Reichenbach's principle to QT is somewhat doubtful, but perhaps not quite ruled out as such just yet, depending on which exact definitions of the principle are used. What does seem clear though is that there is no fundamental law of nature that demands ( or even implies ) the existence of a common cause principle for the case of QT - this is of course not an argument to rule out the possibility, but still.

It is, in principle, imaginable that this universe is only a wild dream and what happens does not have any explanations.

I think that depends a lot on what one means by "explanation". I think it is perfectly conceivable that the universe functions such that there are not always common causes in the Reichenbachian sense present; but I don't think that this implies any kind of mysticism.

 

[Markus Hanke]

@Ilja - I would just like to say that ( while I can't speak for others ), your contributions here are appreciated by me. You are clearly quite knowledgeable in this area, and having the general consensus and opinion challenged in a positive and constructive way, is invaluable for someone like me who is still in the process of learning about all of this. What this did is make me go and do my own further research on certain subjects, and I have learned bits and pieces that I didn't know before. Also, it is always important to take a step back and critically evaluate one's own knowledge and understanding, every so often.

So for all it is worth, I thank you for bringing this up - I do not need to agree with all your assertions, in order to appreciate your contribution here

 

[Ilja]

But what if those probabilistic rules are the explanation ? What if the universe actually does play dice ?

Determinism vs. randomness is not the issue at all. In "Bertlmann's socks and the nature of reality" Bell gives some quotes which show that it was not an important issue for Einstein too, despite his side remark about God playing dice. Realism and causality are much more important. Personally I prefer hidden variable theories which have random trajectories, they are simply not that popular like deterministic dBB theory.

You bring up Reichenbach's principle in this context, so, having had only superficial knowledge of it, I did a bit of reading about this - it seems that, firstly, the exact meaning and definition of the principle itself is subject to some debate ( i.e. there seem to be several versions of it ), and, secondly, that there is no general consensus as to whether or not the principle even applies to the case of QT, which exhibits certain differences to classical probability theory.

Correct. There are, of course, the relativists who see that once there is a theorem which shows that Reichenbach's principle + Einstein causality => Bell's inequality, that means wrong, one has to reject Reichenbach's principle. And relativists are a clear majority.

My point of supporting Reichenbach's common cause is that without it we do not have a justification for the need of finding causal explanations of observable correlations. So I see it not that much as a claim about reality but as part of the justification of the scientific method itself.

I think that depends a lot on what one means by "explanation". I think it is perfectly conceivable that the universe functions such that there are not always common causes in the Reichenbachian sense present; but I don't think that this implies any kind of mysticism.

I would yet have to see an example of a correlation without any causal explanation which is not mystical. What I have seen as counterexamples were simply too restrictive interpretations of the meaning of "common cause", which excluded things like multiple common causes, microscopic or nonlocal common causes, or the flow of time as a common cause.

 

[stevendaryl]

But what if those probabilistic rules are the explanation ? What if the universe actually does play dice ? Why rule out that possibility, if it fits experiment and observation so well ?

My feeling of unease about the standard interpretation of quantum mechanics is not that it's probabilistic, but that it doesn't clearly separate what is real and physical from what is subjective. Is the wave function a physical quantity, or is it a subjective expression of what we know about a system? I don't think that either alternative is very satisfactory. If it is physical, then an observer's updating of the wave function upon performing a measurement seems to mean a nonlocal interaction. If it is subjective, and only reflects our knowledge, then it would seem to me that there ought to be a reality for the knowledge to be about. Some things are definite in QM---if I prepare an electron in a spin-up state, and there are no torques acting on the electron, and no interactions, then it will later be measured to be spin-up with 100% certainty. If you take the point of view that any 100% certain prediction must reflect something objective, then there is something objective about the wave function. But what?

 

[Markus Hanke]

Is the wave function a physical quantity, or is it a subjective expression of what we know about a system?

Perhaps the problem is that we make a distinction between these two options. Could we not somehow interpret the wave function to be an expression of the relationship between observer and quantum system ? Kind of like the reference frames in relativity ? That way it would be a physical quantity, but still subjective in the sense that it is observer-dependent. Everyone seems to assume that "the wave function" is an absolute quantity, but I am unaware of any principle of physics that dictates this to be so.

Simple example - a system of two entangled particles. Alice on her own just sees a free particle, and makes random measurements. Bob on his own just sees a free particle, and also makes random measurements ( but with a different outcome, unbeknownst to him of course ). However, if we have a third observer ( call him Charlie ) who can make measurements on both particles, and compares the results, he may notice correlations, so to him the system looks different than it does to Alice or Bob. Clearly, these observers will all argue that they are dealing with different wave functions, yet they perform measurements on the same system - in that sense, the wave function of the system depends intrinsically on the observer, and as such could be viewed as an expression of the relationship between observer and system.

Am I going completely off the rails here ?

But what ?

In light of the above, the answer would be that the objective bit is neither the properties of the system, nor the outcome of measurements performed by an observer, but rather the relationship between the two.

 

[stevendaryl]

Perhaps the problem is that we make a distinction between these two options. Could we not somehow interpret the wave function to be an expression of the relationship between observer and quantum system ? Kind of like the reference frames in relativity ? That way it would be a physical quantity, but still subjective in the sense that it is observer-dependent. Everyone seems to assume that "the wave function" is an absolute quantity, but I am unaware of any principle of physics that dictates this to be so.

I think it's possible that that idea, when fleshed out, would lead to something like Many-Worlds or Consistent Histories. Not that those are completely free of conceptual problems, either.

 

[Markus Hanke]

I think it's possible that that idea, when fleshed out, would lead to something like Many-Worlds or Consistent Histories. Not that those are completely free of conceptual problems, either.

Yes, you're probably right. I was just trying to look at this thing from different angles.

 

[naima]

Dr Chinese,
I do not know if you are aware of the teachings of Buddhism.
It says that nothing has an intrinsic existence. That all properties appear through relations.
Can we think that counterfactual definiteness is a product of our occidental culture?
Please forgive me for this digression!

 

[Paul Colby]

I do not see any "mysticism" in established quantum theory at all, unless one insists that the universe must satisfy some misguided notion of classicality based purely on subjective human experience, and hence on what may or may not appeal to us philosophically and aesthetically.

Amen. It's unclear to me in these types of discussions exactly what is meant by "value". If taken as "quantity as measured" then the EPR and hidden variable stuff seems at odds with QT from the very get go. An observed value of spin requires an observer device that of necessity defines a direction in space. In this case the observed value is forever a cooperative deal between spin and its measurement device.

 

[Derek Potter]

.

we have nice realistic and causal interpretations.

We do? Would that be that deeply suspect category of interpretation in which... I can hardly bring myself to say this... in which, gulp, there are no *proper* mixtures? :)

 

[Derek Potter]

I think it's possible that that idea, when fleshed out, would lead to something like Many-Worlds or Consistent Histories. Not that those are completely free of conceptual problems, either.

What conceptual problems are those?

I have an unfinished discussion with Ilja about the Factorization Problem - not the common-or-garden Preferred Basis - but I have not had time to think about it, it's only been about a year :) I also know it's quite difficult to justify the idea of a "typical" series of outcomes in order to use occurrence-frequencies as probability in MWI, but I'm not sure that QM is any worse than classical probability in this respect. Kastner seems to think it is.

 

[Ilja]

An observed value of spin requires an observer device that of necessity defines a direction in space. In this case the observed value is forever a cooperative deal between spin and its measurement device.

The problem with this is the EPR criterion of reality. The measurement in the same direction by Bob predicts, with certainty, the result of the measurement by Alice. In dBB theory, where faster than light causal influences exist, this is unproblematic, but in any Einstein-causal theory you have a problem explaining this with a "cooperative deal" with the measurement device of Alice.

 

[stevendaryl]

What conceptual problems are those?

There are a couple of problems. One is that it's not completely clear that you can make sense of probabilities in MWI without some kind of selection. Some people think that that has been solved, but others disagree. There is another problem, which is the sense in which anything at all happens in MWI. I can't remember the exact argument showing why this is problem, but it might be something like this: if you have the wave function of the whole universe then (at least nonrelativistically), you can write it as a superposition of energy eigenstates. Because wave function evolution is linear, you can treat each eigenstate as a separate universe that doesn't interact with any of the others. But an energy eigenstate is essentially a static world--the only time evolution is just an overall phase.

It seems to me that to get something like the real world out of MWI, you might need to impose some additional structure on top of it, instead of just wave functions evolving unitarily.

I have an unfinished discussion with Ilja about the Factorization Problem - not the common-or-garden Preferred Basis - but I have not had time to think about it, it's only been about a year :) I also know it's quite difficult to justify the idea of a "typical" series of outcomes in order to use occurrence-frequencies as probability in MWI, but I'm not sure that QM is any worse than classical probability in this respect. Kastner seems to think it is.

Yeah, I think that QM brings up problems, in the interpretation of probability, for example, that people just ignore in classical mechanics.

 

[Jilang]

I think it's possible that that idea, when fleshed out, would lead to something like Many-Worlds or Consistent Histories. Not that those are completely free of conceptual problems, either.

I haven't encountered any formulation that considers wavefunctions for different observers or how they transform from one to another. I would be very interested in any links that may shed light on this subject, as I agree that they are subjective animals.

 

[Paul Colby]

The problem with this is the EPR criterion of reality. The measurement in the same direction by Bob predicts, with certainty, the result of the measurement by Alice. In dBB theory, where faster than light causal influences exist, this is unproblematic, but in any Einstein-causal theory you have a problem explaining this with a "cooperative deal" with the measurement device of Alice.

I'm kind of a luddite. For example the fact that the speed of light is constant in every frame makes no sense "intuitively" but I accept it as actual fact for all the reasons we know and love and move on. Question, what's missing from QM that people don't feel comfortable with once the basic rules are accepted?

The entire EPR elements of reality stuff is clearly broken right from the get go by the basic rules. A given spin eigenstate may always be found in a different one simply by measuring along another axis. Clearly the whole "spin has a value" thing is pretty strongly depends on the measurement performed as a matter of basic principle. So I ask, what in QM need interpretation? Looks complete to me.

 

[eltodesukane]

<It gives rules how to compute probabilities, but explains nothing.>
<But what if those probabilistic rules are the explanation ? What if the universe actually does play dice ?>
-- In the blockworld view of reality, spacetime exits as a block, all of space and all of time within it.
To us, the transition from present time to future time may appear deterministic or probabilistic.
But from the blockworld point of view it doesn`t matter. The whole of spacetime is there, fixed, immutable, unchanging.

 

[Paul Colby]

<It gives rules how to compute probabilities, but explains nothing.>

Can you define "explain" in this context? The fact that a system may be assigned a state vector is fundamental, just like c being a constant. Can one be expected to explain constancy of c without falling back on observation or theory based on such?

 

[Derek Potter]

There is another problem, which is the sense in which anything at all happens in MWI. I can't remember the exact argument showing why this is problem, but it might be something like this: if you have the wave function of the whole universe then (at least nonrelativistically), you can write it as a superposition of energy eigenstates. Because wave function evolution is linear, you can treat each eigenstate as a separate universe that doesn't interact with any of the others. But an energy eigenstate is essentially a static world--the only time evolution is just an overall phase.

In the energy basis each world is stationary but in any other basis a world is a superposition of those energy states so they will interfere (or beat) and stuff will happen. I think this highlights the fact that in MWI, the basis is essentially arbitrary: you don't have just one set of worlds, you have many alternative sets. I don't see the problem with MWI saying that in some bases the worlds are static, in many bases the worlds are chaotic meaningless monsters, but in a few the worlds are quite like this one(s). What am I missing?