Implication of Hardy's ontological excess baggage theorem

[bohm2]

Given Hardy's "excess baggage" theorem showing that the size of the ontic state space must scale exponentially with the number of systems, does this necessarily pose any "threat" to the ontic approach to QM? Leifer writes:

If the quantum state truly exists in reality, it is puzzling that we cannot detect all of this extra information. Hardy has coined the term "ontological excess baggage" to refer to this phenomenon. It seems that ψ-ontologists are attributing a lot more information to the state of reality than required to explain our observations.

Is the quantum state real? A review of ψ-ontology theorems
http://mattleifer.info/wordpress/wp-content/uploads/2008/10/quanta-pbr.pdf

 

[atyy]

I think it is important to distinguish "ontic" versus "ψ-ontic". The Harrigan-Spekkens distinction between ψ-epistemic and ψ-ontic assumes the existence of hidden variables. In that sense ψ-epistemic and ψ-ontic theories are both ontic theories. Like Leifer and Spekkens, my own (increasingly untenable) preference is ψ-epistemic, yet I can agree with Leifer and Spekkens "Rather, the picture we have in mind is of the quantum state for a region representing beliefs about the physical state of the region, even though we do not yet have a model to propose for the underlying physical states." http://arxiv.org/abs/1107.5849 (footnote 9)

Also, the ontic approach to QM does not mean we should not use Copenhagen. It simply means that there is no measurement problem, and we can use Copenhagen as a superb effective theory. Strictly speaking, the ontic approach to QM is an approach that is beyond QM, and deciding which hidden variables to use will require guidance by observations that cannot be easily accommodated within QM. The exception is Many-Worlds, which if it works has no excess baggage.

 

[bohm2]

Also, the ontic approach to QM does not mean we should not use Copenhagen. It simply means that there is no measurement problem, and we can use Copenhagen as a superb effective theory. Strictly speaking, the ontic approach to QM is an approach that is beyond QM, and deciding which hidden variables to use will require guidance by observations that cannot be easily accommodated within QM. The exception is Many-Worlds, which if it works has no excess baggage.

Yes, I meant ψ-ontic models. But with respect to the MWI, would not MWI also have infinite excess baggage?

On the Bohmian schemes, I think there is a difference between the minimalist model of Durr/Goldstein/Zangi versus Valentini schemes. It is easier to conceptiualize the Valentini scheme as ψ-ontic versus the (DGZ) model. Interestingly Harrigan and Spekken even suggest this in their paper but it appears to make no difference with respect to the PBR theorem as all Bohmian models are untouched by PBR.

Inspired by this pattern, Valentini has wondered whether the pilot-wave (and hence ontic) nature of the wave function in the deBroglie-Bohm approach might be unavoidable. On the other hand, it has been suggested by Wiseman that there exists an unconventional reading of the deBroglie-Bohm approach which is not ψ-ontic. A distinction is made between the quantum state of the universe and the conditional quantum state of a subsystem, defined in Ref. [79]. The latter is argued to be epistemic while the former is deemed to be nomic, that is, law-like, following the lines of Ref. [80] (in which case it is presumably a category mistake to try to characterize the universal wave function as ontic or epistemic). We shall not provide a detailed analysis of this claim here, but highlight it as an interesting possibility that is deserving of further scrutiny.

Einstein, incompleteness, and the epistemic view of quantum states
http://arxiv.org/pdf/0706.2661v1.pdf

 

[atyy]

Ah, so maybe even if Bohmian Mechanics turns out to be true, one can be a ψ-epistemicist. Nice! I shall have to read those papers, I have been wondering if that was possible - in fact, in a way exactly along the lines that the wave function of the universe is ontic, while the conditional wave function is epistemic.

MWI has no excess baggage in the Hardy sense, because there are no additional variables.

 

[atyy]

Here is the DGZ paper mentioning the wave function of the universe and the conditional quantum state. http://arxiv.org/abs/quant-ph/0308039

Actually, I have read the paper and if I remember correctly, the idea of calling the conditional quantum state epistemic is not DGZ's. So it must be Wiseman's private unpublished view, that Harrigan and Spekkens cite in their reference 78 as a private communication. Anyway, I like it and it's what I was thinking about after reading DGZ's paper, so it's nice to have Wiseman's view published indirectly via Harrigan and Spekkens.

 

[Demystifier]

The exception is Many-Worlds, which if it works has no excess baggage.

MWI has no excess baggage in the Hardy sense, because there are no additional variables.

I don't think it's true. To understand why, it is important to distinguish the concept of ADDITIONAL variables from the concept of HIDDEN variables. Additional variables are any variables which are not defined by the wave function alone. Hidden variables are any variables which exist even when they are not measured. From these definitions it should be clear that they are not necessarily the same. In particular, MWI has no additional variables, but has hidden variables. The hidden variables of MWI are the wave function itself.

Now, the excess baggage theorem claims that most of the hidden variables cannot be measured in a single measurement. Applying this to MWI, this should mean that most of the wave function cannot be measured in a single measurement, which of course is true.

Therefore, the excess baggage theorem applies to MWI.

 

[bohm2]

I think it's interesting that in biology/cognitive science we have the opposite problem. There appears to be an "information short-fall":

The Naïve Nativist Model
The human brain is estimated to contain roughly 100 billion = 10¹¹cells, each of which has between 100 and 10,000 synapses, leading to at least 10¹⁴ synapses in the brain. To specify 1 of 10¹¹ cells exactly, you need 37 bits. Therefore, to specify simply the connecting cell corresponding to each synapse you would need 37 x 10¹⁴ bits (and to specify the synaptic weight you would need at least eight bits per synapse). There are about 3 billion (3 x 10⁹) base pairs in mammalian genome, so even if the genome was fully dedicated to specifying brain structure (which it is not) and had perfect coding in an information-theoretic sense, we would have a shortfall of at least 5 orders of magnitude to specify the connections in a human brain: We have 1/10,000th of the DNA we would need to code the detailed wiring of our brains. This ‘gene shortage’ has led scholars like Paul Ehrlich to conclude that little of our behavior could possibly be innate. Let us therefore similarly consider an exclusive role for the environment.

The Naïve Empiricist Model
Let us optimistically suppose that we learn something from our environments every second, waking or asleep, of our lives. There are 31 million seconds in a year (3.15 x 10⁷). If we live to 100, that’s just 3 x 10⁹ seconds (roughly the number of base pairs in the genome). The first five years of life, when most language learning is occurring, contain only 15 x 10⁹seconds. Even the most fortunate and well-stimulated baby has this paltry number of environmental inputs available to specify 10¹⁴ synapses. Although we can hope that many synapses are influenced by each environmental input, this doesn’t help unless each input event, is very highly structured, carrying a large amount of optimally coded information. This seems optimistic, to say the least. Thus the naïve empiricist faces the same vast information shortfall as the naïve nativist.

The Naïve Evolutionist Model
Finally, for completeness, consider the plight of a different type of nativist: An idealized ‘evolutionary empiricist’ who suggests that natural selection alone has programmed behavior. Vertebrate evolution has occupied about a billion (10⁹)years. If we optimistically hypothesize a few bits of information per generation to accumulate, that’s only a few billion bits again (and of course any particularities of the human brain have had far less time —roughly, 6 x 10⁶ years — to accumulate). Again a vast information shortfall exists, of roughly the same order: This one a shortage of evolutionary time. Are we to conclude from this little exercise that development is impossible? Or that the evolution of the brain could not have occurred? No, such basic considerations force us to reject overly simplistic models, and to conclude that both the naïve nativist (genome as blueprint) and naïve empiricist/evolutionist (environment as instructor) viewpoints are woefully inadequate models. Such considerations quickly lead all serious thinkers on these problems to realize that understanding any aspect of development and evolution requires understanding the interactions between DNA and the world beyond the cell nucleus. Despite its tiresome persistence, ‘nature versus nurture’ is a sterile conceptual dead-end, and any valid answer must consider ‘nature via nurture’ in some form or other.

Prolegomena to a Future Science of Biolinguistics
http://www.biolinguistics.eu/index.php/biolinguistics/article/view/133

 

[The_Duck]

It's off topic, I know, but:

to specify simply the connecting cell corresponding to each synapse you would need 37 x 10¹⁴ bits

Basing an argument on this is silly because it assumes that the brain's structure is encoded in DNA in the stupidest possible way. A more reasonable measure of the information needed would be something like the Kolmogorov complexity of the brain's structure (which would measure the size of the smartest possible way to encode the brain's structure).

 

[atyy]

It's off topic, I know, but:

Is it really off topic? http://www.rifters.com/real/articles/Science_No-Brain.pdf

If most of the brain is not necessary, then the exponential complexity in Hilbert space taken ontologically can specify the brain. Or is it that since most of the brain is not necessary, the needless complexity of the brain goes into specifying Hilbert space taken epistemically?

 

[atyy]

I don't think it's true. To understand why, it is important to distinguish the concept of ADDITIONAL variables from the concept of HIDDEN variables. Additional variables are any variables which are not defined by the wave function alone. Hidden variables are any variables which exist even when they are not measured. From these definitions it should be clear that they are not necessarily the same. In particular, MWI has no additional variables, but has hidden variables. The hidden variables of MWI are the wave function itself.

Now, the excess baggage theorem claims that most of the hidden variables cannot be measured in a single measurement. Applying this to MWI, this should mean that most of the wave function cannot be measured in a single measurement, which of course is true.

Therefore, the excess baggage theorem applies to MWI.

Here is a description of Hardy's work by Tung Ten Yong http://arxiv.org/abs/1005.3625. The Excess Baggage Factor is defined as $\gamma = \frac{K_{ontic}}{K_{inst}}$ (Eq 2 in Tung's paper). For an $N$-dimensional quantum state, $K_{inst}=N^{2}$, the number of measurements needed to pick out a particular mixed state from any other state. There is an analogous $K_{ontic}$ for the epistemic state, which is a probability distribution defined over the ontic states. In MWI, it seems to me that the ontic state is the quantum state, so the ratio of the K-values should be 1.

 

[bohm2]

Here is a description of Hardy's work by Tung Ten Yong http://arxiv.org/abs/1005.3625. The Excess Baggage Factor is defined as $\gamma = \frac{K_{ontic}}{K_{inst}}$ (Eq 2 in Tung's paper#. For an $N$-dimensional quantum state, $K_{inst}=N^{2}$, the number of measurements needed to pick out a particular mixed state from any other state. There is an analogous $K_{ontic}$ for the epistemic state, which is a probability distribution defined over the ontic states. In MWI, it seems to me that the ontic state is the quantum state, so the ratio of the K-values should be 1.

Maybe I'm not fully understanding this but in that paper note what the author writes:

We can view the issue in another way. Since the instrumental state of a physical system depends on observer’s knowledge and his ability to measure the system, there is not a unique value of K_inst for a physical object. What we should really have is K(_inst)_level , the subscript ‘level’ denoting the level of description or knowledge available to the observer about the object.

The author then goes on to argue the following:

This urges us to propose a theorem similar to the one in Hardy’s paper: The Quantum Relative Ontological Excess Baggage Theorem. Given that quantum states are the instrumental states, any interpretations/ theories of quantum theory in which the quantum states are also the ontic states are very uncomfortable. This is also true if quantum states themselves form parts or the whole of ontology.

And with respect to the MWI, he concludes:

Everettian many-worlds approach:
There are many versions of this but we restrict our concern with the most common version of it - there is only one wavefunction, that of the universe, and it is the only ontology. Then even if the dimension of its Hilbert space is finite, there will still be continually many possible wavefunctions, and thus infinite relative excess baggage.

 

[atyy]

I don't understand those arguments of the author either. I was only using his paper to reference Hardy's theorem.

 

[Demystifier]

Here is a description of Hardy's work by Tung Ten Yong http://arxiv.org/abs/1005.3625. The Excess Baggage Factor is defined as $\gamma = \frac{K_{ontic}}{K_{inst}}$ (Eq 2 in Tung's paper). For an $N$-dimensional quantum state, $K_{inst}=N^{2}$, the number of measurements needed to pick out a particular mixed state from any other state. There is an analogous $K_{ontic}$ for the epistemic state, which is a probability distribution defined over the ontic states. In MWI, it seems to me that the ontic state is the quantum state, so the ratio of the K-values should be 1.

I would say that, in MWI, K_ontic counts information in all possible worlds (branches of the wave function not realized in our world), while K_inst counts only information in our world. If so, then the ratio is much bigger than 1.

But to check if I am right, I would need to see the paper by Hardy. Do you perhaps have a link to a free version of it?

 

[bohm2]

I don't understand those arguments of the author either. I was only using his paper to reference Hardy's theorem.

I read that paper more closely and I sort of got the just of his argument, I think. Too bad there are no free on-line links to Hardy's paper. But with his new Relative Ontological Excess Baggage Theorem, he concludes that both MWI and some Bohmian models are not plausible. He seems to be okay with Valentini's model (e.g. "this does not concern
the pilot-wave approach as proposed by Valentini") but argues for the implausibility of the purely interpretational de Broglie-Bohm’s approach. Which Bohmian model is that? I have no clue.

 

[Demystifier]

I would say that, in MWI, K_ontic counts information in all possible worlds (branches of the wave function not realized in our world), while K_inst counts only information in our world. If so, then the ratio is much bigger than 1.

But to check if I am right, I would need to see the paper by Hardy. Do you perhaps have a link to a free version of it?

Now I have read the Hardy's paper so I can give a better answer.

Consider a spin-1/2 system in MWI. In this case K_ontic=infinite and K_inst=4 so the ratio is infinite, in agreement with the excess baggage theorem.

The key is to understand what exactly the numbers K_ontic and K_inst count. In MWI, K_ontic is simply the number of different wave functions, which of course is infinite. The tricky quantity is K_inst, so let me explain what is that for the spin-1/2 system.

First, K_inst does not depend on interpretation, so it is the same in MWI, Bohmian, Copenhagen, or whatever interpretation. The probability of any measurable (i.e. instrumental) quantity can be calculated from the density matrix rho. For spin-1/2 this matrix contains 2x2=4 complex numbers, i.e. 8 real numbers. Not all these numbers are independent, however, because rho must be hermitian so it contains only 4 real independent numbers (which corresponds to 4=N^2 for the N=2 dimensional Hilbert space.) These 4 independent numbers can be identified with 4 INDEPENDENT INSTRUMENTAL PROBABILITIES. For instance, one suitable choice of these 4 independent probabilities is p_z+, p_z-, p_x+, p_y+, where p_z+ is the probability of finding the particle in the state +1/2 in the z-direction, etc. So, if you know only these 4 independent instrumental probabilities, you can calculate the instrumental probability of ANYTHING. This is the content of the claim that K_inst=4.

Now what exactly is the excess baggage theorem telling to us?

To answer that question, let us first explain how one might argue if one were not aware of the theorem. Naively, one could reason as follows: The fundamental state is not a set of probabilities, but an actual ontic state. The probabilities can somehow be calculated from the ontic state. Since I can calculate anything from these 4 probabilities, this seems to suggest that the true ontic state must be one of these 4 states, i.e. that the particle is either in the state z+, z-, x+, or y+. If so, there are only 4 possible ontic states.

What the excess baggage theorem proves is that the naive reasoning above is wrong. The number of possible ontic states must be much larger than 4. In particular, for MWI the number of possible ontic states is infinite.

The above is more-or-less an overview of the Hardy's paper, but let me end with a personal comment. It seems to me that the excess baggage theorem is related to the well known property of general (including mixed) density matrices: A single density matrix (defining a single im-proper mixture) may correspond to an infinite number od different proper mixtures.

 

[stevendaryl]

Now I have read the Hardy's paper so I can give a better answer.

Consider a spin-1/2 system in MWI. In this case K_ontic=infinite and K_inst=4 so the ratio is infinite, in agreement with the excess baggage theorem.

I'm a little confused. There seems to be two different issues about "excess baggage" in the wave function.

One issue is that for a single system, there seems to be a lot more information in the wave function than can possibly be extracted by experiment. To give an example, a general spinor is described by a pair of complex numbers:

$\left( \begin{array}<br /> \\<br /> \alpha \\<br /> \beta \end{array} \right)$

There is an infinite amount of information in those two numbers. On the other hand, any measurement performed on a spin-1/2 particle reveals exactly 1 bit of information: spin-up or spin-down in whatever direction you measured. Afterward, there is nothing else you can learn about the original state. The information is gone forever.

The second issue is about statistical information obtained, not from a single system, but from many, many systems prepared identically. When you talk about "the probability of finding the particle in the state +1/2 in the z-direction", you're talking about this kind of statistical information. But the wave function doesn't seem to have a lot of excess baggage in this sense.

For a two state system: $|\psi\rangle = \alpha |\psi_1\rangle + \beta |\psi_2\rangle$, we can actually figure out the two numbers $\alpha$ and $\beta$ (up to an overall phase, which is undetectable). We have:

$|\alpha|^2$ = the probability of finding the system in state $|\psi_1\rangle$.
$|\beta|^2$ = the probability of finding the system in state $|\psi_2\rangle$.
$1/2 + Re(\alpha^* \beta)$ = the probability of finding the system in state $\frac{1}{\sqrt{2}} (|\psi_1\rangle + |\psi_2\rangle)$.
$1/2 - Re(\alpha^* \beta)$ = the probability of finding the system in state $\frac{1}{\sqrt{2}} (|\psi_1\rangle - |\psi_2\rangle)$.

It seems to me that since there are infinitely many different Hermitian operators to use as measurements, then we can get infinitely many different expectation values. So the amount of information available through repeated measurements is not finite.

 

[Demystifier]

Stevendaryl, what you say above is correct, but as I already said, the definition of K_inst is rather tricky. It has a very specific meaning, and one may feel that it is a rather artificial quantity.

 

[Demystifier]

I read that paper more closely and I sort of got the just of his argument, I think. Too bad there are no free on-line links to Hardy's paper. But with his new Relative Ontological Excess Baggage Theorem, he concludes that both MWI and some Bohmian models are not plausible. He seems to be okay with Valentini's model (e.g. "this does not concern
the pilot-wave approach as proposed by Valentini") but argues for the implausibility of the purely interpretational de Broglie-Bohm’s approach. Which Bohmian model is that? I have no clue.

I also read the Yong's paper which criticizes Hardy, and I think I understand it. So let me explain it in my own words.

In general, there are two motivations to study hidden variables (i.e. ontological models):
(i) To better understand the measurable predictions of quantum mechanics itself.
(ii) To make new measurable predictions, not given by quantum mechanics.

In the Hardy analysis it is implicit that only the motivation (i) is present. Yong criticizes Hardy for that and considers the subject from a wider perpective, by having both motivation (i) and motivation (ii) in mind. So Hardy is still right from his narrower point of view, but Yong is also right from his wider perspective.

In particular, when Yong speaks of "purely interpretational de Broglie Bohm", he means a version of de Broglie Bohm considered only with the motivation (i), which makes exactly the same measurable predictions as standard QM. But unlike Hardy, he also allows to speak of de Broglie Bohm from a wider (not "purely interpretational") perspective, in which one may also include the motivation (ii).

The main (and correct) point of Yong is this: The tricky quantity K_inst is even more tricky if one allows the possibility that a hidden variable theory (ontological model) makes new predictions. The K_inst with new predictions included may be larger than K_inst without new predictions. Therefore, from such a wider perspective, K_inst cannot be uniquelly determined from quantum mechanics alone. Consequently, the excess baggage theorem does not necessarily need to be valid in such a wider class of theories.

 

[atyy]

The above is more-or-less an overview of the Hardy's paper, but let me end with a personal comment. It seems to me that the excess baggage theorem is related to the well known property of general (including mixed) density matrices: A single density matrix (defining a single im-proper mixture) may correspond to an infinite number od different proper mixtures.

Thanks, I understand now. That's a good way of explaining why MWI also has infinite excess baggage!

 

[bohm2]

I also read the Yong's paper which criticizes Hardy, and I think I understand it. So let me explain it in my own words.

In general, there are two motivations to study hidden variables (i.e. ontological models):
(i) To better understand the measurable predictions of quantum mechanics itself.
(ii) To make new measurable predictions, not given by quantum mechanics.

Thanks, that explains a lot. So Yong is fine with Valentini's approach as it is not a purely interpretational de Broglie/Bohm model since Valentini does suggest that evidence for hidden variable might lead to a new physics:

A hidden-variables theory will agree with quantum mechanics only if the variables have a special quantum equilibrium distribution, analogous to thermal equilibrium. It is because of quantum equilibrium that superluminal influences cannot be used for superluminal signaling. When hidden variables have the equilibrium distribution, the signals average out to zero. If the hidden variables instead had a nonequilibrium distribution, the underlying superluminal signals would become observable and controllable. Relativity theory would be violated; time would be absolute rather than relative to each observer. The Heisenberg uncertainty principle would also be violated.

By this reasoning, there is no real conspiracy. We cannot use entangled systems to send superluminal signals simply because we happen to be stuck in a state of quantum equilibrium, just as hypothetical beings in a classical heat death would not be able to convert heat into work...

Even granting that all this could be true, one might conclude that no evidence for hidden variables will ever be found, since there would be little prospect of finding nonequilibrium today, nearly 14 billion years after the big bang. There may, however, be a way. Using the relic radiation known as the cosmic microwave background (CMB), it is possible to test quantum mechanics in the very early universe, potentially probing a time before the quantum heat death took place.

Cosmological Data Hint at a Level of Physics Underlying Quantum Mechanics
http://blogs.scientificamerican.com...sics-underlying-quantum-mechanics-guest-post/

Written in the skies: why quantum mechanics might be wrong
http://www.nature.com/news/2008/080515/full/news.2008.829.html