Dear Peter,
I’ve just picked up my head from doing the final corrections to my new book on realism in quantum foundations to find you asking, “where exactly does probability enter the theory?”
My understanding, after a lot of study, is that you have the following options:
1) Put the probabilities in at the beginning, as did Bohr, Heisenberg and von Neumann. This requires an operational approach which introduces measurement and probabilities as primitive concepts, ie through a “collapse” or “projection” postulate, which postulates Born’s rule and “eigenvalue realism”, or through a Hardy-style operational reconstruction. These are elegant but they do not answer your question as measurement and probability are primitive concepts.
2) You can attempt to derive probabilities from a formalism that has only unitary, Schrodinger evolution, which has no notion of probabilities to begin with. This is Everett’s MWI route.
This is by now a very long story. It took me a lot of time to sort out for the book, and I had help from Saunders and Wallace and others. At best, there is no consensus amongst experts that this can be done. (This agrees with Scott’s remark, above.) The rough outline is
i) the original version due to Everett fails, because you can show that with certainty there are branches of the wavefunction whose observers record measurements that disagree with Born’s rule. Because there is no primitive notion of probability you cannot say that these observers are improbable, in fact there are an infinite number of them, and also an infinite number whose observations agree with Born’s rule.
ii) There are recently several very sophisticated attempts to derive subjective probabilities and the Born rule. These are centred at Oxford, were initiated by David Deutsch and developed in different versions by Hillary Greaves, Wayne
Myrvold, Simon Saunders and David Wallace. These all use decoherence and also give up on recovering objective probabilities. Instead, they try, (in one version) from the axioms of decision theory, to show that it is rational for an observer to bet (ie choose subjective probabilities) as if Born’s rule were true. (Even though objectively Born’s Rule is false.)
If you read the literature you can only conclude, after some challenging technical arguments, that the experts disagree about whether this kind of approach succeeds or fails, and what the implications should be.
3) Invent a new physical theory which gives a complete description of individual processes from which the quantum probabilities are derived from ignorance about the initial state. This would then be a completion of QM rather than an interpretation. de Broglie-Bohm and collapse models are existence proofs that this is a possible route. There are also other approaches of this kind, such as Adler’s trace dynamics and my real ensemble formulation.
I have the impression you don’t find any of these 3 options satisfactory. The kind of answer to your question of where the probabilities come from would be one in which we start with QM without measurement, probabilities etc and derive them. But this was option 2 and a whole lot of very bright people have tried and failed to make it work (in a way that convinces all the experts).
My personal view is that option 3) is the only way forward for physics. But I wouldn’t try to do more here than argue that unless some notion of subjective probability can be made to work, as in option 2), you simply cannot get an answer to your question. You then either need to conclude with Bohr that the only kind of theory of atomic phenomena is operational, and has probabilities and measurement as primitive terms
or agree with Einstein, de Broglie, Schrodinger, Bohm, Bell ets that QM requires a completion that gives a complete description of individual experiments.
Thanks,
Lee