On the myth that probability depends on knowledge

[Physics Monkey]

Yes, if the gas is deterministic, and hence determined by the initial condition.

Should I read the subtext here to say that you don't believe classical gases are deterministic?

I am not that interested in money to accept your hypotheses. You may think of probabilities of single cases - these are very subjective, though. They have nothing to do with the probabilities used in physics.

Again, you're just making an assertion without any evidence. I claim the probabilities used in physics are highly subjective. They contain our prejudices about beauty and symmetry. They include our limited access to experimental data and our subjective assumptions about the relevant degrees of freedom, sources of error, etc. We even use them to help determine what are the interesting questions in physics. In short, they are always constrained and defined by our own limited experience and knowledge. I have no interest in forbidding you from talking about "objective probabilities" as some platonic notion, but real physics is done with subjective probabilities.

For example, the Boltzmann distribution is certainly subjective. It assigns non-zero weight to states that the system will never access, and indeed, many distributions will give precisely the same answers for macroscopic physical observables. Thus choosing Boltzmann is a subjective assignment.

In any case, since I don't know the properties of your 6-sided die, assigning probabilities is completely arbitrary. Unless I assume that the die is just like one of the many I have seen before, in which case I assign equal probabilities to each outcome, because I substitute ensemble probabilities for ignorance.
But if your die had painted 1 on each side, my choice of 2:6 based on my assumption would be 100% wrong.

Thus probabilities are based on _assumptions_, not on _knowledge_.

Assumptions are based on knowledge. You assign probabilities to the die rolls based on your knowledge and experience with other die. You want to make the best guess you can based on your limited knowledge. It's ok to be 100% wrong so long as you made a good guess. If you get to roll the die many times then you can improve your guess. Of course, it could really roll a classical many times in exactly the same then you would always get the same answer, thus the probabilities one assigns to die rolls are actually only even relevant because one has limited knowledge of the conditions of the throw. Another manifestation of subjectivity in physics.

 

[Physics Monkey]

Not every statement in a scientific discussion must be a scientific argument. And if you look at the context, you see that here ''applying probability theory'' meant ''deducing from a single case a probability'', which simply doesn't make sense.

Hardly. doing something that I consider foolish and being a fool are worlds apart.
I sometimes do foolish things, but don't think that this makes me a fool. And those who don't agree with me won't take my statement that ''Applying probability theory to single instances is foolish'' seriously anyway. Thus the statement is harmless.

I disagree, we were talking about assigning probability to a single event. Your phrase "deducing from a single case a probability" presupposes the notion that there is some abstract correct probability to be obtained.

I further disagree that your statement is harmless. It can discourage participation in the discussion and it can sway opinions based on rhetoric rather than sound scientific argument. I imagine you would agree those are both negative outcomes.

Finally, I agree there is a distinction between being a fool and acting foolish. I misquoted you. Nevertheless, I think you're totally missing the point. Telling someone they're doing something foolish still has no place in a scientific discussion.

 

[A. Neumaier]

I claim the probabilities used in physics are highly subjective. ?

Let us be specific. The probability of decay of a radium atom in the next 10 minutes is a constant independent of anyone's knowledge. It had that value even before there were physicists knowing about the existence of radium. No amount of subjectivity in the views about beauty and symmetry, relevant degrees of freedom, sources of error changes this fact.

Assumptions are based on knowledge. ?

They may be based on knowledge. They may also be based on ignorance or false information, unchecked belief, etc.. But all this is irrelevant for physics. Once your assumptions specified the ensemble in question, the probabilities are objectively determined. No matter whether you can calculate them, or whether you have any knowledge about the system so defined.

You assign probabilities to the die rolls based on your knowledge and experience with other die. You want to make the best guess you can based on your limited knowledge. It's ok to be 100% wrong so long as you made a good guess. If you get to roll the die many times then you can improve your guess. ?

This only implies that the guesses made depend on your knowledge. But the probabilities are not dependent on whether you guess them well or poorly. Nature doesn't care about our knowledge, it doesn't change its behavior when we get to know something new. And physics is about the properties of Nature, not about the psychology of human knowledge.

 

[A. Neumaier]

Telling someone they're doing something foolish still has no place in a scientific discussion.

I wasn't telling someone they're doing something foolish. I was telling something about my standards of judging, not meaning anyone in particular. If you felt offended, I apologize.

 

[Fra]

But the probabilities are not dependent on whether you guess them well or poorly. Nature doesn't care about our knowledge, it doesn't change its behavior when we get to know something new. And physics is about the properties of Nature, not about the psychology of human knowledge.

Umm... I'd say physics (and natural science in general) is ALL about us learning ABOUT nature, what we can say about nature.

So whatever nature is, or probabilities are, the PROBLEM is how to INFER it. THIS is the primary problem of the scientific method. The problem is not really what nature is or isn't. The problem, is how to, by means of experiments and interactions make rational inferences, that lead to rational and sound beliefs (scientific knowledge).

To me, physics is how to make rational inferences and produce rational expectations ABOUT nature, from past interaction history. And I even think that all physical interactions obey this structure, that two interacting atoms are in fact making inferences about each other. This is why I probably consider my self the complete opposite to your very strong structural realist position.

I agree it's not about psychology or human mind. But none that sees the inference perspective seriously makes that confusion. Observations, information states, expectations etc are thought to be encoded in any physical system. No brains are needed.

/Fredrik

 

[Studiot]

@A. Neumaier

I find discussion in this thread very difficult.

This is partly because I agree with much of what you say and partly because the thread appears to be a compartmentalised set of bilateral conversations, rather than a group discussion.

I am also inviting you to look a little further about probability.

Take for instance limit state design.
Or bridge strength assessment.
Or diversity as applied to electrical installation design
Or the error term as applied to many mathematical calculations.

You state that single event probability is either 1 or zero.
In the case of my bridge example this implies that a bridge either collapses or it doesn't.
In reality the bridge may suffer a partial collapse, indeed some bridges may suffer a small partial collapse (=degradation) on every use until finally that last straw walks over it.

 

[Stephen Tashi]

The objective probability of a single event is 1 if it happens and 0 otherwise.

"if it happens"? That's a nice conditional for a statement about a probability. You've made a Bayesian utterance.

By that reasoning, all "single events" have probabilities that are 0 or 1. So now we must look at non-single events. But what are "non-single" events? - collections of single events? Collections of events, each of which has probability 0 or 1 ? This sounds like the old Von Mises approach to probability theory using "collectives".


Are there any actual consequences to the theory of "objective probabilities"? Can it make any testable predictions that disagree with Bayesian predictions?

 

[SpectraCat]

Yes, it is meaningless. For either the particle will be observed, or it won't. Thus the probability must be one or zero, but |psi|^2 typically isn't.

|psi|^2 is the probability for observing the position in the ensemble of _all_ particles prepared in the same state psi, but says nothing about any particular such particle.

No, |psi|^2 defines the probability density .. it applies equally well to the probability of single measurements (before they are made obviously), as it does to ensembles of measurements. Of course *after* the measurement the particle position will be a delta function (for theoretically infinite precision), but that is not really a probability at all .. it is a result. Furthermore, if you consider the space of all possible results, the particle will always be observed somewhere, so the probability then is always 1. That seems a lot more meaningless than |psi|^2 to me ...

Please give a more precise context for this claim.

You are the one who started telling Varon (on the interpretations poll thread I think) about how the position of a particle does exist, but is not well-defined (you used the term fuzzy) until a measurement is made. What do you use to describe the existence of the particle position prior to the measurement if you don't use |psi|^2?

 

[A. Neumaier]

Take for instance limit state design.
Or bridge strength assessment.
Or diversity as applied to electrical installation design
Or the error term as applied to many mathematical calculations.

You state that single event probability is either 1 or zero.
In the case of my bridge example this implies that a bridge either collapses or it doesn't.
In reality the bridge may suffer a partial collapse, indeed some bridges may suffer a small partial collapse (=degradation) on every use until finally that last straw walks over it.

I have been doing a lot of practical work in uncertainty analysis (including FORM, SORM and various other engineering techniques). I even did research in advanced methods of uncertainty estimation in complex settings; see http://arnold-neumaier.at/clouds.html

Thus I make my assertions based on thorough and quite diverse experience.

Predicting a partial collapse is different from predicting a probability of collapse.
The correct modeling would try predict the expected amount of collapse or degradation, not a probability of collapse. Bringing this into play only confuses issues, and I'll disregard it in the following.

Saying that there is a 60% chance that it will rain tomorrow may sound like a probability statement about the single event tomorrow, but it isn't - this statement cannot be verified, whether or not it actually rains, and hence is empty. Instead it is a statement about the known preconditions of the weather tomorrow - namely that they belong to an ensemble described by a stochastic model in which the probability of raining is 60%.

Essentially the same holds for all other of the many engineering uses of probability I have met during my career.

A lot of knowledge (but also prejudice, or more or less justified assumptions) goes into the creation of an appropriate stochastic model for defining the ensemble. In this (and only this) sense, probabilities are knowledge-dependent. But this knowledge-dependence is of the same character as that of anything we say or believe, and hence is not something worth emphasizing.

On the other hand, once the ensemble is fixed, probabilites are objective. Of course, the language assigns probabilities to single events, but (as in the case of tomorrow's weather), these are not properties of these events but of an associated theoretical ensemble chosen
such that averaged over many actual events the predictions are maximally useful.

Thus if two people assign different probabilities to the same event, it means that they have different ensembles in mind for modeling the same situation.

Now suppose that we have a real ensemble, such as whether or not it rains at Vienna airport each day of the next two years, or whether or not some of the bridges in Europe crash in the next two years Then there are objective probabilities associated with them, namely the relative frequencies of the actual events. Again, these are completely independent of the knowledge of any observer or analyst. They are unknown now, but can be determiend in two years time, hence they are objective.

On the other hand, the probabilities we assign to them based on a particular model for predictions are approximations, whose quality depends on the knowledge (but also prejudice, or more or less justified assumptions) of the modeler.

But again, this is nothing surprising, and nothing special for probabilities - the quality of the _description_ of any property of anything depends on the describer's knowledge, although the properties themselves are objectively fixed (if they deserve the name ''property'').

Thus knowledge plays in probability no role different from that it plays everywhere - at least not in those aspects of probability that can be checked in reality.

Subjective probability are a different matter. They are not verifiable or falsifiable, hence do not fall under the above analysis. But because of that, they should have no place in science or engineering.

 

[Dale]

There may be also a subjective probability in the Bayesian sense, but such probabilities are physcally meaningless. And this is a discussion in a physics forum.

On what basis do you make the claim that Bayesian probabilities are physically meaningless? You can use them to make predictions, test hypotheses, and all of the other things that you would expect to be able to do with probabilities in physics. Your claim seems to represent simply a personal distaste for Bayesian reasoning rather than an informed understanding of how it can be used in science.

 

[A. Neumaier]

On what basis do you make the claim that Bayesian probabilities are physically meaningless

You didn't read correctly. I only stated that _subjective_ probability in the Bayesian sense, are physially meaningless.

But Bayesian analysis is a powerful body of theory, not restricted to a subjective interpretation. In fact I applied Bayesian techniques myself in very successful large-scale applications to animal breeding. http://arnold-neumaier.at/papers.html#reml
Nothing there is subjective.

You can use them to make predictions, test hypotheses, and all of the other things that you would expect to be able to do with probabilities in physics.

Nothing of this depends on a subjective interpretation of probability.

Your claim seems to represent simply a personal distaste for Bayesian reasoning rather than an informed understanding of how it can be used in science.

If someone in our conversation is not informed then it is you, exhibiting a lack of abilities to read correctly and a lack of knowledge of my background.

 

[harrylin]

The objective probability of a single event is 1 if it happens and 0 otherwise.

There may be also a subjective probability in the Bayesian sense, but such probabilities are physcally meaningless. And this is a discussion in a physics forum.

I agree with Dalespam and others: there are different uses of "probability" and more than one is physically meaningful. Predictive probability of single events ("betting") is very much used for such things as risk analysis. A simple example of predictive probability:

As a child I enjoyed a quiz, in the end of which the final contestant had to choose to stand in front of one of three doors. The prize was hidden behind one of them. Next the quiz master opens one of the two other doors (no prize behind it), and the contestant had the option to switch to the remaining closed door. I found it very funny that often the contestant switched doors. Later I was explained that it was the right thing to do: the probability that the prize was behind the other door was 2/3 and not 1/2 as I thought. The knowledge that the prize is not behind the one door affects the analysis of the other doors - in common language, it affects the "probabilities".

Now, the opening of a door to observe that no prize is behind it, is a physical measurement.
However, according to you the objective probability of a single event is 1 if it happens and 0 otherwise - thus the probability that the prize is behind a door is always 1 or 0. With that approach no calculation is possible, and no correct risk analysis can be made.

Harald

 

[Dale]

But Bayesian analysis is a powerful body of theory, not restricted to a subjective interpretation. In fact I applied Bayesian techniques myself in very successful large-scale applications to animal breeding.

OK, then I am not sure I know what you mean by _subjective_ probability. I understood that you are complaining either about the Bayesian definition of probability or about the subjectivity involved in selecting a prior. But if either of those are correct then I don't understand how you could have used Bayesian techniques in your own research.

Can you clarify your meaning of _subjective_ probability and why you think it is physically meaningless and how you reconcile that with your own use of Bayesian methods?

 

[Fra]

OK, then I am not sure I know what you mean by _subjective_ probability. I understood that you are complaining either about the Bayesian interpretation of probability or about the subjectivity involved in selecting a prior. But if either of those are correct then I don't understand how you could have used Bayesian techniques in your own research.

Can you clarify your meaning of _subjective_ probability and why you think it is physically meaningless?

There's usually a subdivision within bayesian views. Objective vs subjective bayesians. I suspect that's what he means.

Objective bayesians are more like a conditional probability where the conditional construct is somewhat objective.

Subjective bayesian views is similar but a litte bit more radical.

They are related but I think one difference is exemplified by how you view for example symmetry transformations in any relativity theory. Which RELATES objectively, the subjective views of each observer. One can say that relativity in that sense are objective since hte subjective views are related by an objective relation.

The subjectiv view may instead reject the existence of such forcing constraint, and instead the observer invariance is recovered by emergent agreements. It's not forcing hardcoded constraints.

I subscribe to the latter. I think Neumaier subscribes to the first. I'm sure he will correct me if I mischaracterized his views.

The difference is also analogus to HOW you UNDERSTAND the requirement of observer invariance of physical laws, that are one constructing principle of relativity. Is it a FORCING constraint (and then where does this come from??) or is it simply an emergent constraint in the sense of obsever invariance as observer DEMOCRACY?

The difference is subtle, but important.

/Fredrik

 

[A. Neumaier]

However, according to you, the probability that the prize is behind a door is always 1 or 0. With that approach no calculation is possible, and no correct risk analysis can be made.

If there is only a single event, it depends on what is actually the case whether switching is a better option, and no risk analysis will help you if your choice was wrong.

A risk analysis is based upon the assumption that the distribution of the prize is uniform, so that you gain something from the disclosed information. This assumes an ensemble of multiple repetitions of the situation.

 

[A. Neumaier]

OK, then I am not sure I know what you mean by _subjective_ probability. I understood that you are complaining either about the Bayesian definition of probability or about the subjectivity involved in selecting a prior. But if either of those are correct then I don't understand how you could have used Bayesian techniques in your own research.

Can you clarify your meaning of _subjective_ probability and why you think it is physically meaningless and how you reconcile that with your own use of Bayesian methods?

''The'' Bayesian definition does not exist. Wikiedia says:

Broadly speaking, there are two views on Bayesian probability that interpret the state of knowledge concept in different ways. According to the objectivist view, the rules of Bayesian statistics can be justified by requirements of rationality and consistency and interpreted as an extension of logic.[1][4] According to the subjectivist view, the state of knowledge measures a "personal belief"

http://en.wikipedia.org/wiki/Bayesian_probability


Bayesian probability can be either the same as that by Kolmogorov, and hence is objectively defined by the ensemble. Or it can be a personal belief based on knowledge or prejudice, then it is subjective.

All Bayesian statistics can be defined in the usual Kolmogorov setting, with a frequentist interpretation of probability, since it is nothing as a sophisticated use of conditional probability, which is independent of any interpretation of probability.

In situations alluded above where a prior can be correct or wrong, the wording shows already that there is something objective (knowledge independent) about the situation.

 

[Physics Monkey]

Let us be specific. The probability of decay of a radium atom in the next 10 minutes is a constant independent of anyone's knowledge. It had that value even before there were physicists knowing about the existence of radium. No amount of subjectivity in the views about beauty and symmetry, relevant degrees of freedom, sources of error changes this fact.

Presumably you want me to agree that the usual expression from nuclear physics is the correct objective probability? However, I don't think this point of view is consistent with what you said earlier. For example, based on your discussion in #48 (in a slightly different context) it seems to me you would have to claim that the probability in question for the radium atom is \sim \delta(t-t_{\mbox{actual}}). In other words, "it decays when it decays", but this expression is apparently totally unknowable and and has essentially nothing to do with the usual calculations in nuclear physics that give us what we usually call the decay probability. Perhaps you will dispute this?

And regarding the dice, I would say that probability is a tool for the description of physical systems, not necessarily some intrinsic element of reality. If I take sufficient note of the initial conditions and am careful to repeat them with every throw, then I obtain the same roll every time. Similarly, if I have knowledge of the initial conditions and a sufficiently detailed model, then I can predict the result of every throw. It is only without this knowledge in this case that I should describe the throw as random. The probability is subjective but it corresponds to physical reality, namely the fact that dice are excellent "randomizers" because of sensitivity to initial conditions.

 

[A. Neumaier]

Presumably you want me to agree that the usual expression from nuclear physics is the correct objective probability? However, I don't think this point of view is consistent with what you said earlier. For example, based on your discussion in #48 (in a slightly different context) it seems to me you would have to claim that the probability in question for the radium atom is \sim \delta(t-t_{\mbox{actual}}).

Note the indefinite article. ''a'' radium atom ia a member of an ensemble, whereas ''the radium atom prepared here'' is a specific instance.

If I take sufficient note of the initial conditions and am careful to repeat them with every throw, then I obtain the same roll every time.

How can you do this given that a real die must be described by quantum mechanics?

 

[skippy1729]

You didn't read correctly. I only stated that _subjective_ probability in the Bayesian sense, are physially meaningless.

But Bayesian analysis is a powerful body of theory, not restricted to a subjective interpretation.

On this point you are simply mistaken. The fundamental essence of modern Bayesian probability is that probabilities are degrees of belief or knowledge subject to the rules of logic. Different gamblers and different physicists may be privy to different information and apply different rules of inference in assigning their "book values" or "wave functions". They may rationally and consistently assign different probabilities to the same situation. If they are rational they will look at new information as it comes in and revise their probabilities. MANY theoretical and experimental physicists have used MANY "calculation schemes" and "lab configurations" and have, over years, arrived at amazing agreement to the lamb shift, electron g-factor &ct.
We may imagine that their results are converging in limit to THE OBJECTIVE VALUE. But this objective value does not exist except as a mathematical abstraction. It is all built on a pyramid of subjectivity.

Skippy

PS One of my instructors told me many decades ago that it is always best to read the original sources. One of the original papers which is the foundation modern Bayesian theory is "Truth and Probability" by Frank Ramsey which is available online:

http://www.fitelson.org/probability/ramsey.pdf

There is much material on ArXiv but

http://www.google.com/url?sa=t&sour...sg=AFQjCNFvLy41P5HErRRzDgX1k1PHD2yPcg&cad=rja

is a very good, light read, introduction to Bayesian ideas in physics. It also has a few pages of objections and replies.

PPS I would appreciate any reference to "objective" Bayesian probability theory.

 

[Dale]

Bayesian probability can be either the same as that by Kolmogorov, and hence is objectively defined by the ensemble. Or it can be a personal belief based on knowledge or prejudice, then it is subjective.

All Bayesian statistics can be defined in the usual Kolmogorov setting, with a frequentist interpretation of probability, since it is nothing as a sophisticated use of conditional probability, which is independent of any interpretation of probability

Yes, Bayesian statistics can be applied to an ensemble, but they can also be applied to other situations. It is more general. From the wikipedia link and your comments I still can't tell exactly what you are referring to specifically when you say _subjective_ probability and why you think it is not relevant in physics. Are you just concerned about making bad subjective assessments in the prior probability?

 

[A. Neumaier]

PPS I would appreciate any reference to "objective" Bayesian probability theory.

I had given a link to Wikipedia where both the subjective and the objective variant are mentioned.

 

[A. Neumaier]

Yes, Bayesian statistics can be applied to an ensemble, but they can also be applied to other situations. It is more general. From the wikipedia link and your comments I still can't tell exactly what you are referring to specifically when you say _subjective_ probability and why you think it is not relevant in physics. Are you just concerned about making bad subjective assessments in the prior probability?

Objective = independent of any particular observer, verifiable by anyone with the appropriate understanding and equipment.

Subjective = degree of belief, and such things, which cannot be checked objectively.

Bayesian statistics with an unspecified prior to be chosen by the user according to his knowledge is subjective statistics. It doesn't make user-independent predictions.

Bayesian statistics with a fully specified model, including the prior, is objective statistics.
One can check its predictions on any sufficiently large sample. Of this kind is the statistics in physics. The ensemble is always completely specified (apart from the parameters to be estimated).

 

[harrylin]

If there is only a single event, it depends on what is actually the case whether switching is a better option, and no risk analysis will help you if your choice was wrong.

A risk analysis is based upon the assumption that the distribution of the prize is uniform, so that you gain something from the disclosed information. This assumes an ensemble of multiple repetitions of the situation.

For a correct probability estimation beforehand, no "multiple" (infinite?!) repetitions of the situation are required. The subject can make an objective analysis based on the given information, even though for the quiz master the chance is 0 or 1 because he already knows the result.
As a matter of fact, the "probability" of what actually is, is always 1 - That's not really "probability".

 

[A. Neumaier]

For a correct probability estimation beforehand, no "multiple" (infinite?!) repetitions of the situation are required. The subject can make an objective analysis based on the given information, even though for the quiz master the chance is 0 or 1 because he already knows the result.

If the probabilities depend on the person it is a subjective probability.

For the person doing the analysis, though the interest may be in predicting a single case, the objective probability refers to the probability in the ensemble analyzed, and not to the single unknown case. For in the latter case, the probability of a future event would depend on the particular past data set used, which (a) is strange and (b) would make it again a subjective probability.

As a matter of fact, the "probability" of what actually is, is always 1 - That's not really "probability".

I disagree. The Kolmogorov axioms for a probability space are satisfied.

 

[harrylin]

If the probabilities depend on the person it is a subjective probability.

Any subjective estimations by that person don't play a role; only the available information. It's objective (although not "invariant") in the sense that the calculation is according to standard rules of probability calculus and everyone (except you?) agrees about that calculation.

For the person doing the analysis, though the interest may be in predicting a single case, the objective probability refers to the probability in the ensemble analyzed, and not to the single unknown case. For in the latter case, the probability of a future event would depend on the particular past data set used, which (a) is strange and (b) would make it again a subjective probability. [...]

I'm afraid that I can't follow that... this is like any other "take a marble without putting it back and then take another one" probability calculation. Future probabilities can depend on past actions, according to standard and objective rules of calculation.

Now, is that objective or subjective? That isn't the topic of this thread, but a quick sample from dictionary.com of the common meaning of words tells me that such calculations are definitely objective and not subjective:

- Objective: not influenced by personal feelings, interpretations, or prejudice; based on facts; unbiased: an objective opinion.

- Subjective: belonging to the thinking subject rather than to the object of thought; pertaining to or characteristic of an individual; personal; individual: a subjective evaluation.

I omitted "existing in the mind" as objective opinions and evaluations also exist in the mind - that isn't helpful.

Harald

PS I now see that you posted similar definitions; necessarily we cannot but agree on that point.

 

[Dale]

Objective = independent of any particular observer, verifiable by anyone with the appropriate understanding and equipment.

Subjective = degree of belief, and such things, which cannot be checked objectively.

Bayesian statistics with an unspecified prior to be chosen by the user according to his knowledge is subjective statistics. It doesn't make user-independent predictions.

Bayesian statistics with a fully specified model, including the prior, is objective statistics.

Thanks, now I clearly understand what you mean by subjective. You are correct that specifying a good prior can be a tricky business and that different users will often make different choices in priors which makes it subjective in your terminology.

Frequentist statistical tests often reduce to a Bayesian test with an ignorance prior. In your definition Bayesian statistics with an ignorance prior would be objective since it is user-independent.

However, what if we are not completely ignorant at the beginning? What if we have some knowledge that is not shared with other users? Why should the user-dependent (subjective) state of knowledge not lead to user-dependent priors and therefore user-dependent predictions about the outcome of some physical experiment?

One can check its predictions on any sufficiently large sample. Of this kind is the statistics in physics. The ensemble is always completely specified (apart from the parameters to be estimated).

On any sufficiently large sample the prior is irrelevant and only the data matters. So over an ensemble, even with subjective priors, the Bayesian approach gets user-independent (objective) posteriors.

 

[A. Neumaier]

Thanks, now I clearly understand what you mean by subjective. You are correct that specifying a good prior can be a tricky business and that different users will often make different choices in priors which makes it subjective in your terminology.

Frequentist statistical tests often reduce to a Bayesian test with an ignorance prior. In your definition Bayesian statistics with an ignorance prior would be objective since it is user-independent.

However, what if we are not completely ignorant at the beginning? What if we have some knowledge that is not shared with other users? Why should the user-dependent (subjective) state of knowledge not lead to user-dependent priors and therefore user-dependent predictions about the outcome of some physical experiment?

Specifying the prior defines the ensemble and hence makes the probabilities objective - no matter whether the prior is good or poor. The quality of the prior is a measure not of objectivity but of matching reality.

In most cases, one has two different ensembles: the model ensemble and the ensemble to which the model is supposed to apply. The second ensemble is usually unknown since part of it lies in the future, and often the future uses of a model are not even precisely known. Quality measures the gap between these two ensembles.

If the model is silent about the prior then the probabilities are subjective since different users may choose different priors and then get different predictions.

If the application is silent about precicely which events it should be applied to then the probabilities are subjective since different users may apply it to different scenarios and then get different results.

If the application is as single instance then the probabilities are 0 or 1, and only someone who knows the answer or guesses it correctly can have a correct model of the situation.

In physics (which is my concern in this thread), the physical description of a system completely specifies the ensemble, both of the model (the governing euations and boundary conditions) and of the application (the experimental setting). Thus both the predicted and the observable probabilities are objective. Whether one or both of therm may be unknowwn at particular times to particular people is completely irrelevant.

This objectivity is the strength of scientific practice in general, and of physics in particular. It allows anyone with access to the necessary information and equipment check the quality of any particular model with respect to the application it is supposed to describe.

On any sufficiently large sample the prior is irrelevant and only the data matters. So over an ensemble, even with subjective priors, the Bayesian approach gets user-independent (objective) posteriors.

But your ''sufficiently large'' may have to be far larger than mine.

 

[A. Neumaier]

Any subjective estimations by that person don't play a role; only the available information. It's objective (although not "invariant") in the sense that the calculation is according to standard rules of probability calculus and everyone (except you?) agrees about that calculation.

Bayesian techniques need both available information _and_ a prior. If the prior is not specified, it may depend on the persons subjective estimate, and calculations need not agree.

Thus if one gives strict rules for how to determine the prior from prior information (this is the case in the bayesian applications to animal breeding I had cited before), the calculated Bayesian estimates are objective.

In all other cases, the calculated Bayesian probabilities are subjective.

 

[Dale]

Specifying the prior defines the ensemble and hence makes the probabilities objective - no matter whether the prior is good or poor. ... This objectivity is the strength of scientific practice in general, and of physics in particular. It allows anyone with access to the necessary information and equipment check the quality of any particular model with respect to the application it is supposed to describe.

OK, I am fine with all of this. Your stance is even more acceptable to me than I had thought previously since you allow specified non-ignorance priors to encode available knowledge.

I don't see how it supports your claim that probability (in physics) does not depend on knowledge, but I agree with what you are saying.

 

[A. Neumaier]

O
I don't see how it supports your claim that probability (in physics) does not depend on knowledge, but I agree with what you are saying.

The model probabilities depend on the model, not on knowledge. Given the defintion of an ideal gas
(say) and specified values of P,, V, T, everything is determined - independent of the knowledge of anyone.

The application probabilities depend on the application, not on knowledge. Given the definition of the experimental arrangement specifying the application, everything is determined - independent of the knowledge of anyone.

So all probabilities encountered in physics are objective and knowledge independent.

What depends on knowledge is the assessment of how well a model fits an application, and hence the choice of a particular model to predict in a particular application. But this has nothing to do with probability, since it holds as well for deterministic models.

 

[Dale]

The model probabilities depend on the model, not on knowledge. Given the defintion of an ideal gas
(say) and specified values of P,, V, T, everything is determined - independent of the knowledge of anyone.

The application probabilities depend on the application, not on knowledge. Given the definition of the experimental arrangement specifying the application, everything is determined - independent of the knowledge of anyone.

So all probabilities encountered in physics are objective and knowledge independent.

What depends on knowledge is the assessment of how well a model fits an application, and hence the choice of a particular model to predict in a particular application. But this has nothing to do with probability, since it holds as well for deterministic models.

Sorry about this, I wasn't clear in my point above. My point is that the prior contains the knowledge, so if you are specifying the prior you are fixing the knowledge.

Suppose you have some quantity x and you want to determine if x depends on y or not. If you do not let y vary then you cannot claim that you have shown that x does not depend on y.

You claim that probability does not depend on knowledge, but knowledge is contained in the prior, and you require a specified prior. Similarly, when you said "anyone with access to the necessary information and equipment" you are fixing the knowledge. Since you are not allowing knowledge to vary you cannot make any conclusions about the dependence of probability on knowledge.

If you want to examine the dependence of physical probabilities on knowledge you must allow the priors and the information to vary across users.

 

[Studiot]

So all probabilities encountered in physics are objective and knowledge independent.

I have already said that I agree with much of what you posted.

However I maintain that your statements are too narrow.

Your response to my structural engineering examples clearly indicate you have no idea what a bridge assessment or limit state design theory involves.

Both are part of applied physics and properly represented in PF.

Since this is a Quantum section how about these questions

What is the probability that the Higgs will be discovered before the end of 2011?

Suppose I had asked a similar question in 1933

What is the probability that the positron will be discovered before the end of 1933?

 

[A. Neumaier]

Sorry about this, I wasn't clear in my point above. My point is that the prior contains the knowledge, so if you are specifying the prior you are fixing the knowledge.

You claim that probability does not depend on knowledge, but knowledge is contained in the prior, and you require a specified prior. Similarly, when you said "anyone with access to the necessary information and equipment" you are fixing the knowledge.

By the same argument, deterministic models would depend on knowledge. So if you insist on the correctness of your argument, why emphasize it in the probabilistic case but not in the determinstic case?

Moreover, a model may have a very unrealistic prior. In this case, probabilities depend - according to your view - on arbitrary assumptions or on misinformation rather than knowledge.

On the other hand, with my usage of the terms, everything is clear and unambiguous.

 

[A. Neumaier]

Your response to my structural engineering examples clearly indicate you have no idea what a bridge assessment or limit state design theory involves.

I have worked with structural engineers and am familiar with FORM and SORM techniques for limit state analysis, and with variations and alternatives for the assessment of reliability. This has no bearing on the theme.

Engineers calculate probabilities based on models applying to a large ensemble of cases parameterized by some parameters, and then specialize for a particular case by fitting the observed properties of a bridge to the model. The resulting parameter defines a subensemble of all conceivable bridges with characteristics matching the concrete bridge in question, and the safety probability refers to this ensemble, not to the specific bridge.

What is the probability that the Higgs will be discovered before the end of 2011?

Suppose I had asked a similar question in 1933

What is the probability that the positron will be discovered before the end of 1933?

In both cases, the answer is 0 or 1, and can be known only after the fact.

 

[Studiot]

and can be known only after the fact

This is the whole crux of my point.

You still have no idea what bridge assessment involves.

You are faced with the following scenario:-

You are presented with a specific bridge over a ravine. Not

a subensemble of all conceivable bridges with characteristics matching the concrete bridge in question,

As the Engineer you are asked

Will the bridge collapse if I drive my lorry over it?

This represents a one off unique situation and you have to make an assessment ie a subjective decision to allow for the fact that all the facts are not ( and probably cannot be ) known.

You did not read my post correctly either.

Studiot-
limit state design

A.Neumaier-
limit state analysis

Are you not familiar with the difference between analysis and the more difficult process of synthesis (or design)?

 

[Studiot]

In both cases, the answer is 0 or 1, and can be known only after the fact.

One of the direct consequences of this statement, if true, has deep philosophical implications because it implies determinism.
That is that any point in time the future is completely determined with a probability of either 1 or 0.

 

[SpectraCat]

One of the direct consequences of this statement, if true, has deep philosophical implications because it implies determinism.
That is that any point in time the future is completely determined with a probability of either 1 or 0.

I would go farther, and say that such statements *assume* determinism, in the sense that it is taken as a postulate, and thus cannot be proven or disproven.

 

[Dale]

By the same argument, deterministic models would depend on knowledge. So if you insist on the correctness of your argument, why emphasize it in the probabilistic case but not in the determinstic case?

No reason, except that the deterministic case is off topic and obvious.

Moreover, a model may have a very unrealistic prior. In this case, probabilities depend - according to your view - on arbitrary assumptions or on misinformation rather than knowledge.

Certainly, you could also make arithmetic errors or typographical errors, or you could misapply a formula, or you could use wrong formulas. Any time you use misinformation or misuse information in physics you will get nonsense. I don't think that is terribly interesting other than pedagogically.

On the other hand, with my usage of the terms, everything is clear and unambiguous.

Yes, but your definition is not the only valid and accepted definition of probability. Your claim is only true if you require probabilities to be defined only over ensembles. In that case I agree that the posterior probability does not depend on the prior so in that case you are indeed correct that probability does not depend on knowledge. Under the more general definition of probability the posterior can depend on the prior in any case where you do not have a sufficiently large number of observations.

 

[harrylin]

Bayesian techniques need both available information _and_ a prior. If the prior is not specified, it may depend on the persons subjective estimate, and calculations need not agree.

Thus if one gives strict rules for how to determine the prior from prior information (this is the case in the bayesian applications to animal breeding I had cited before), the calculated Bayesian estimates are objective.

In all other cases, the calculated Bayesian probabilities are subjective.

The case example I gave is objective since it has no subjective estimate as input. And what (nearly?) everyone calls "the probability" in that case depends on knowledge - take it or leave it.

 

[Dale]

Thus if one gives strict rules for how to determine the prior from prior information (this is the case in the bayesian applications to animal breeding I had cited before), the calculated Bayesian estimates are objective.

This is different from the fixed-prior case. Here, instead of having a fixed prior you have a family of priors with some hyper-parameters which are uniquely specified by available information. Note that in this case the probabilities are objective (user independent), but they do depend on knowledge.

 

[A. Neumaier]

You are the one who started telling Varon (on the interpretations poll thread I think) about how the position of a particle does exist, but is not well-defined (you used the term fuzzy) until a measurement is made. What do you use to describe the existence of the particle position prior to the measurement if you don't use |psi|^2?

You misunderstood what I said. Saying that a particle has a fuzzu position means that it actually _has_ this position independent of any measurement, but that its value is meaningful only up to an accuracy determined by the uncertainty relation. The position is given not by |psi|^2 but by xbar=psi^*x psi, with an absolute uncertainty of sqrt(psi^*(x-xbar)^2 psi).

Measuring the position gives a value statistically consistent with this and the measuring accuracy, but does not change the fact that the position remains fuzzy. You cannot read from your meter that the position is at exactly x.

 

[A. Neumaier]

You are presented with a specific bridge over a ravine. [...]
As the Engineer you are asked
Will the bridge collapse if I drive my lorry over it?

Whether you answer ''with 75% probability'' or ''with 10% probability'', nobody can verify whether your answer was correct when the bridge collapsed, or didin't collapse, upon driving the lorry over it.
And if you answer ''with 99% probability'' and you conclude that you better not drive, the answer can again not be checked.

This makes it clear that your answer is not about this bridge collapsing when you drive over it now,
but with the ensemble of all possible lorries and bridges matching the characteristics of your model as derived from your input data.

This represents a one off unique situation and you have to make an assessment ie a subjective decision to allow for the fact that all the facts are not ( and probably cannot be ) known.

As far as it is applied to a particular situation, you always have a subjective probability, which is not verifiable by checking against reality.

You did not read my post correctly either.
Are you not familiar with the difference between analysis and the more difficult process of synthesis (or design)?

I am familiar with it. But the bridge example is one of analysis, not of design. And though I know about limit state design, I was not directly involved in that. Thus I deliberately changed the wording. However, it is not _so_ different from limit state analysis, as it involves the latter as a constraining design condition. So it is part of the total optimization problem to be solved. I have been involved in the design of devices facing uncertainty by other methods; see, e.g., p.81ff of my slides http://arnold-neumaier.at/ms/robslides.pdf

 

[A. Neumaier]

One of the direct consequences of this statement, if true, has deep philosophical implications because it implies determinism.
That is that any point in time the future is completely determined with a probability of either 1 or 0.

It doesn't imply determinism, since no dynamical law is involved in it. It only implies (or assumes, depending on what you regard as given) that after something happened, it is a fact, independent of the future.

 

[A. Neumaier]

No reason, except that the deterministic case is off topic and obvious.

It is not off-topic since it serves to clarify the issue, and it is as obvious in the probabilisitc case as in the deterministc case, hence there is no reason to emphasize it in the latter case. It doesn't add any useful insight into the nature of probability.

Yes, but your definition is not the only valid and accepted definition of probability. Your claim is only true if you require probabilities to be defined only over ensembles. In that case I agree that the posterior probability does not depend on the prior so in that case you are indeed correct that probability does not depend on knowledge. Under the more general definition of probability the posterior can depend on the prior in any case where you do not have a sufficiently large number of observations.

But in that case, the probability is subjective, and not checkable by anyone.

Thus according to the customary criteria, it is not part of science.

 

[A. Neumaier]

This is different from the fixed-prior case. Here, instead of having a fixed prior you have a family of priors with some hyper-parameters which are uniquely specified by available information. Note that in this case the probabilities are objective (user independent), but they do depend on knowledge.

They do depend on the selected parameters, which is part of the specification of the ensemble.

Of course, the model reflects knowledge, prejudice, assumptions, the authorities trusted, assessment errors, and all that, but that's the same as in _all_ modeling. Hence it is not a special characteristics of probability.

 

[Studiot]

As far as it is applied to a particular situation, you always have a subjective probability, QUOTE]

Loud applause all round.

That is the point everyone has been trying to make to you. Subjective probability has a place in physical science.

Further there exist a range of probabilities, useful in science, between the values 0 and 1.

which is not verifiable by checking against reality.[/

You test your assessment by driving over the bridge.

My specific examples separately addressed two different points. (1) Uncertainty and (2)objective v subjective.

Limit State theory (analysis or design) is a real world example of applied science attempts to allow for inevitable uncertainty in an objective way. There is no subjectivism whatsoever in this theory. It has been highly successful in increasing design eficiency.

Bridge assessment contains a specific subjective component as a formal part of the process. An extra factor is introduced called the condition factor. This is a subjective derating factor, not present in normal limit state or other analysis methods. (Assessment does not necessarily use limit state theory.)

 

[A. Neumaier]

Subjective probability has a place in physical science.

No, since it is not testable.

You test your assessment by driving over the bridge.

Whether the assessment was ''with 75% probability'' or ''with 10% probability'', nobody can verify whether the statement was correct after you tried to drive over the bridge. Thus it cannot be regarded as a test.

 

[Studiot]

OK, so we have laid one ghost.

You have not disgreed that there is room, even a necessity, for a subjective component to probability in applied science.


Now for the second one.

You mentioned several times that a probability value exists for something whether the observer knows this value or not.

I agree.

Similarly a probability value exists whether the observer tests, or can test or not.

 

[Fra]

You test your assessment by driving over the bridge.

Yes, exactly.

This is also the gaming analogy. When driving over the bridge, you are placing best, you are taking risks. But this is how nature works. All you ever do, is place your bets and play the game. Along the game you shall then learn and revise your expectations as feedback is arrived.

However, sometimes fatal things happens. Driving over the bridge can be fatal. But this is also part of the game.

The predictions from this game is that only the players that are rational and good guessers and gamers, will survive. So the systems we observer in nature, are then likely to comply to these rationality constraints. But they are not FORCED to them. In fact evolution depends on mistakes and variation.

So subjective probabilites are not tested in the descriptive sense. But they don't need to. Their sole purpose are in evaluating the most rational action (think some action principle). But these "inference systems" that are somewhat subjective are subject to evolution and selection, and anywhere near equilibrium conditions this may yield predictions of expected behaviour (actions) of subsystems in nature; just assuming rationality in their way of placing bets based upon subjective probabilitis.

I think if you take the "rationality constraints" to be exact, and forcing, then the difference to this view and neumaiers "objective constraints" is almost nil.

But the problem is that even the effectively objective constraints are observer dependent and in particular scale dependent. So the only consistent stance as far as I am concerned, is to allow for evolution and selection here and understand that the subjective perspective is what is needed to understand how the effective objective has emerged. Without that, it just is what it is. An ad hoc choice for not particular reason.

The evolutionary picture has a power the deductive way hasn't - to provide a mechanism to understand effective objectivity from a democratic system of subjective views as they interact (equilibrate).

/Fredrik

 

[A. Neumaier]

You have not disgreed that there is room, even a necessity, for a subjective component to probability in applied science.

In the art of using science, not in science itself. Subjective probability is a guide to action in single instances, but not a scientific (testable) concept.

You mentioned several times that a probability value exists for something whether the observer knows this value or not.

Similarly a probability value exists whether the observer tests, or can test or not.

The latter sort of existence is meaningless. In the same sense, ghosts exist (subjectively) no matter whether it can be tested.