On the myth that probability depends on knowledge

[A. Neumaier]

On what basis do you make the claim that Bayesian probabilities are physically meaningless

You didn't read correctly. I only stated that _subjective_ probability in the Bayesian sense, are physially meaningless.

But Bayesian analysis is a powerful body of theory, not restricted to a subjective interpretation. In fact I applied Bayesian techniques myself in very successful large-scale applications to animal breeding. http://arnold-neumaier.at/papers.html#reml
Nothing there is subjective.

You can use them to make predictions, test hypotheses, and all of the other things that you would expect to be able to do with probabilities in physics.

Nothing of this depends on a subjective interpretation of probability.

Your claim seems to represent simply a personal distaste for Bayesian reasoning rather than an informed understanding of how it can be used in science.

If someone in our conversation is not informed then it is you, exhibiting a lack of abilities to read correctly and a lack of knowledge of my background.

 

[harrylin]

The objective probability of a single event is 1 if it happens and 0 otherwise.

There may be also a subjective probability in the Bayesian sense, but such probabilities are physcally meaningless. And this is a discussion in a physics forum.

I agree with Dalespam and others: there are different uses of "probability" and more than one is physically meaningful. Predictive probability of single events ("betting") is very much used for such things as risk analysis. A simple example of predictive probability:

As a child I enjoyed a quiz, in the end of which the final contestant had to choose to stand in front of one of three doors. The prize was hidden behind one of them. Next the quiz master opens one of the two other doors (no prize behind it), and the contestant had the option to switch to the remaining closed door. I found it very funny that often the contestant switched doors. Later I was explained that it was the right thing to do: the probability that the prize was behind the other door was 2/3 and not 1/2 as I thought. The knowledge that the prize is not behind the one door affects the analysis of the other doors - in common language, it affects the "probabilities".

Now, the opening of a door to observe that no prize is behind it, is a physical measurement.
However, according to you the objective probability of a single event is 1 if it happens and 0 otherwise - thus the probability that the prize is behind a door is always 1 or 0. With that approach no calculation is possible, and no correct risk analysis can be made.

Harald

 

[Dale]

But Bayesian analysis is a powerful body of theory, not restricted to a subjective interpretation. In fact I applied Bayesian techniques myself in very successful large-scale applications to animal breeding.

OK, then I am not sure I know what you mean by _subjective_ probability. I understood that you are complaining either about the Bayesian definition of probability or about the subjectivity involved in selecting a prior. But if either of those are correct then I don't understand how you could have used Bayesian techniques in your own research.

Can you clarify your meaning of _subjective_ probability and why you think it is physically meaningless and how you reconcile that with your own use of Bayesian methods?

 

[Fra]

OK, then I am not sure I know what you mean by _subjective_ probability. I understood that you are complaining either about the Bayesian interpretation of probability or about the subjectivity involved in selecting a prior. But if either of those are correct then I don't understand how you could have used Bayesian techniques in your own research.

Can you clarify your meaning of _subjective_ probability and why you think it is physically meaningless?

There's usually a subdivision within bayesian views. Objective vs subjective bayesians. I suspect that's what he means.

Objective bayesians are more like a conditional probability where the conditional construct is somewhat objective.

Subjective bayesian views is similar but a litte bit more radical.

They are related but I think one difference is exemplified by how you view for example symmetry transformations in any relativity theory. Which RELATES objectively, the subjective views of each observer. One can say that relativity in that sense are objective since hte subjective views are related by an objective relation.

The subjectiv view may instead reject the existence of such forcing constraint, and instead the observer invariance is recovered by emergent agreements. It's not forcing hardcoded constraints.

I subscribe to the latter. I think Neumaier subscribes to the first. I'm sure he will correct me if I mischaracterized his views.

The difference is also analogus to HOW you UNDERSTAND the requirement of observer invariance of physical laws, that are one constructing principle of relativity. Is it a FORCING constraint (and then where does this come from??) or is it simply an emergent constraint in the sense of obsever invariance as observer DEMOCRACY?

The difference is subtle, but important.

/Fredrik

 

[A. Neumaier]

However, according to you, the probability that the prize is behind a door is always 1 or 0. With that approach no calculation is possible, and no correct risk analysis can be made.

If there is only a single event, it depends on what is actually the case whether switching is a better option, and no risk analysis will help you if your choice was wrong.

A risk analysis is based upon the assumption that the distribution of the prize is uniform, so that you gain something from the disclosed information. This assumes an ensemble of multiple repetitions of the situation.

 

[A. Neumaier]

OK, then I am not sure I know what you mean by _subjective_ probability. I understood that you are complaining either about the Bayesian definition of probability or about the subjectivity involved in selecting a prior. But if either of those are correct then I don't understand how you could have used Bayesian techniques in your own research.

Can you clarify your meaning of _subjective_ probability and why you think it is physically meaningless and how you reconcile that with your own use of Bayesian methods?

''The'' Bayesian definition does not exist. Wikiedia says:

Broadly speaking, there are two views on Bayesian probability that interpret the state of knowledge concept in different ways. According to the objectivist view, the rules of Bayesian statistics can be justified by requirements of rationality and consistency and interpreted as an extension of logic.[1][4] According to the subjectivist view, the state of knowledge measures a "personal belief"

http://en.wikipedia.org/wiki/Bayesian_probability


Bayesian probability can be either the same as that by Kolmogorov, and hence is objectively defined by the ensemble. Or it can be a personal belief based on knowledge or prejudice, then it is subjective.

All Bayesian statistics can be defined in the usual Kolmogorov setting, with a frequentist interpretation of probability, since it is nothing as a sophisticated use of conditional probability, which is independent of any interpretation of probability.

In situations alluded above where a prior can be correct or wrong, the wording shows already that there is something objective (knowledge independent) about the situation.

 

[Physics Monkey]

Let us be specific. The probability of decay of a radium atom in the next 10 minutes is a constant independent of anyone's knowledge. It had that value even before there were physicists knowing about the existence of radium. No amount of subjectivity in the views about beauty and symmetry, relevant degrees of freedom, sources of error changes this fact.

Presumably you want me to agree that the usual expression from nuclear physics is the correct objective probability? However, I don't think this point of view is consistent with what you said earlier. For example, based on your discussion in #48 (in a slightly different context) it seems to me you would have to claim that the probability in question for the radium atom is \sim \delta(t-t_{\mbox{actual}}). In other words, "it decays when it decays", but this expression is apparently totally unknowable and and has essentially nothing to do with the usual calculations in nuclear physics that give us what we usually call the decay probability. Perhaps you will dispute this?

And regarding the dice, I would say that probability is a tool for the description of physical systems, not necessarily some intrinsic element of reality. If I take sufficient note of the initial conditions and am careful to repeat them with every throw, then I obtain the same roll every time. Similarly, if I have knowledge of the initial conditions and a sufficiently detailed model, then I can predict the result of every throw. It is only without this knowledge in this case that I should describe the throw as random. The probability is subjective but it corresponds to physical reality, namely the fact that dice are excellent "randomizers" because of sensitivity to initial conditions.

 

[A. Neumaier]

Presumably you want me to agree that the usual expression from nuclear physics is the correct objective probability? However, I don't think this point of view is consistent with what you said earlier. For example, based on your discussion in #48 (in a slightly different context) it seems to me you would have to claim that the probability in question for the radium atom is \sim \delta(t-t_{\mbox{actual}}).

Note the indefinite article. ''a'' radium atom ia a member of an ensemble, whereas ''the radium atom prepared here'' is a specific instance.

If I take sufficient note of the initial conditions and am careful to repeat them with every throw, then I obtain the same roll every time.

How can you do this given that a real die must be described by quantum mechanics?

 

[skippy1729]

You didn't read correctly. I only stated that _subjective_ probability in the Bayesian sense, are physially meaningless.

But Bayesian analysis is a powerful body of theory, not restricted to a subjective interpretation.

On this point you are simply mistaken. The fundamental essence of modern Bayesian probability is that probabilities are degrees of belief or knowledge subject to the rules of logic. Different gamblers and different physicists may be privy to different information and apply different rules of inference in assigning their "book values" or "wave functions". They may rationally and consistently assign different probabilities to the same situation. If they are rational they will look at new information as it comes in and revise their probabilities. MANY theoretical and experimental physicists have used MANY "calculation schemes" and "lab configurations" and have, over years, arrived at amazing agreement to the lamb shift, electron g-factor &ct.
We may imagine that their results are converging in limit to THE OBJECTIVE VALUE. But this objective value does not exist except as a mathematical abstraction. It is all built on a pyramid of subjectivity.

Skippy

PS One of my instructors told me many decades ago that it is always best to read the original sources. One of the original papers which is the foundation modern Bayesian theory is "Truth and Probability" by Frank Ramsey which is available online:

http://www.fitelson.org/probability/ramsey.pdf

There is much material on ArXiv but

http://www.google.com/url?sa=t&sour...sg=AFQjCNFvLy41P5HErRRzDgX1k1PHD2yPcg&cad=rja

is a very good, light read, introduction to Bayesian ideas in physics. It also has a few pages of objections and replies.

PPS I would appreciate any reference to "objective" Bayesian probability theory.

 

[Dale]

Bayesian probability can be either the same as that by Kolmogorov, and hence is objectively defined by the ensemble. Or it can be a personal belief based on knowledge or prejudice, then it is subjective.

All Bayesian statistics can be defined in the usual Kolmogorov setting, with a frequentist interpretation of probability, since it is nothing as a sophisticated use of conditional probability, which is independent of any interpretation of probability

Yes, Bayesian statistics can be applied to an ensemble, but they can also be applied to other situations. It is more general. From the wikipedia link and your comments I still can't tell exactly what you are referring to specifically when you say _subjective_ probability and why you think it is not relevant in physics. Are you just concerned about making bad subjective assessments in the prior probability?

 

[A. Neumaier]

PPS I would appreciate any reference to "objective" Bayesian probability theory.

I had given a link to Wikipedia where both the subjective and the objective variant are mentioned.

 

[A. Neumaier]

Yes, Bayesian statistics can be applied to an ensemble, but they can also be applied to other situations. It is more general. From the wikipedia link and your comments I still can't tell exactly what you are referring to specifically when you say _subjective_ probability and why you think it is not relevant in physics. Are you just concerned about making bad subjective assessments in the prior probability?

Objective = independent of any particular observer, verifiable by anyone with the appropriate understanding and equipment.

Subjective = degree of belief, and such things, which cannot be checked objectively.

Bayesian statistics with an unspecified prior to be chosen by the user according to his knowledge is subjective statistics. It doesn't make user-independent predictions.

Bayesian statistics with a fully specified model, including the prior, is objective statistics.
One can check its predictions on any sufficiently large sample. Of this kind is the statistics in physics. The ensemble is always completely specified (apart from the parameters to be estimated).

 

[harrylin]

If there is only a single event, it depends on what is actually the case whether switching is a better option, and no risk analysis will help you if your choice was wrong.

A risk analysis is based upon the assumption that the distribution of the prize is uniform, so that you gain something from the disclosed information. This assumes an ensemble of multiple repetitions of the situation.

For a correct probability estimation beforehand, no "multiple" (infinite?!) repetitions of the situation are required. The subject can make an objective analysis based on the given information, even though for the quiz master the chance is 0 or 1 because he already knows the result.
As a matter of fact, the "probability" of what actually is, is always 1 - That's not really "probability".

 

[A. Neumaier]

For a correct probability estimation beforehand, no "multiple" (infinite?!) repetitions of the situation are required. The subject can make an objective analysis based on the given information, even though for the quiz master the chance is 0 or 1 because he already knows the result.

If the probabilities depend on the person it is a subjective probability.

For the person doing the analysis, though the interest may be in predicting a single case, the objective probability refers to the probability in the ensemble analyzed, and not to the single unknown case. For in the latter case, the probability of a future event would depend on the particular past data set used, which (a) is strange and (b) would make it again a subjective probability.

As a matter of fact, the "probability" of what actually is, is always 1 - That's not really "probability".

I disagree. The Kolmogorov axioms for a probability space are satisfied.

 

[harrylin]

If the probabilities depend on the person it is a subjective probability.

Any subjective estimations by that person don't play a role; only the available information. It's objective (although not "invariant") in the sense that the calculation is according to standard rules of probability calculus and everyone (except you?) agrees about that calculation.

For the person doing the analysis, though the interest may be in predicting a single case, the objective probability refers to the probability in the ensemble analyzed, and not to the single unknown case. For in the latter case, the probability of a future event would depend on the particular past data set used, which (a) is strange and (b) would make it again a subjective probability. [...]

I'm afraid that I can't follow that... this is like any other "take a marble without putting it back and then take another one" probability calculation. Future probabilities can depend on past actions, according to standard and objective rules of calculation.

Now, is that objective or subjective? That isn't the topic of this thread, but a quick sample from dictionary.com of the common meaning of words tells me that such calculations are definitely objective and not subjective:

- Objective: not influenced by personal feelings, interpretations, or prejudice; based on facts; unbiased: an objective opinion.

- Subjective: belonging to the thinking subject rather than to the object of thought; pertaining to or characteristic of an individual; personal; individual: a subjective evaluation.

I omitted "existing in the mind" as objective opinions and evaluations also exist in the mind - that isn't helpful.

Harald

PS I now see that you posted similar definitions; necessarily we cannot but agree on that point.

 

[Dale]

Objective = independent of any particular observer, verifiable by anyone with the appropriate understanding and equipment.

Subjective = degree of belief, and such things, which cannot be checked objectively.

Bayesian statistics with an unspecified prior to be chosen by the user according to his knowledge is subjective statistics. It doesn't make user-independent predictions.

Bayesian statistics with a fully specified model, including the prior, is objective statistics.

Thanks, now I clearly understand what you mean by subjective. You are correct that specifying a good prior can be a tricky business and that different users will often make different choices in priors which makes it subjective in your terminology.

Frequentist statistical tests often reduce to a Bayesian test with an ignorance prior. In your definition Bayesian statistics with an ignorance prior would be objective since it is user-independent.

However, what if we are not completely ignorant at the beginning? What if we have some knowledge that is not shared with other users? Why should the user-dependent (subjective) state of knowledge not lead to user-dependent priors and therefore user-dependent predictions about the outcome of some physical experiment?

One can check its predictions on any sufficiently large sample. Of this kind is the statistics in physics. The ensemble is always completely specified (apart from the parameters to be estimated).

On any sufficiently large sample the prior is irrelevant and only the data matters. So over an ensemble, even with subjective priors, the Bayesian approach gets user-independent (objective) posteriors.

 

[A. Neumaier]

Thanks, now I clearly understand what you mean by subjective. You are correct that specifying a good prior can be a tricky business and that different users will often make different choices in priors which makes it subjective in your terminology.

Frequentist statistical tests often reduce to a Bayesian test with an ignorance prior. In your definition Bayesian statistics with an ignorance prior would be objective since it is user-independent.

However, what if we are not completely ignorant at the beginning? What if we have some knowledge that is not shared with other users? Why should the user-dependent (subjective) state of knowledge not lead to user-dependent priors and therefore user-dependent predictions about the outcome of some physical experiment?

Specifying the prior defines the ensemble and hence makes the probabilities objective - no matter whether the prior is good or poor. The quality of the prior is a measure not of objectivity but of matching reality.

In most cases, one has two different ensembles: the model ensemble and the ensemble to which the model is supposed to apply. The second ensemble is usually unknown since part of it lies in the future, and often the future uses of a model are not even precisely known. Quality measures the gap between these two ensembles.

If the model is silent about the prior then the probabilities are subjective since different users may choose different priors and then get different predictions.

If the application is silent about precicely which events it should be applied to then the probabilities are subjective since different users may apply it to different scenarios and then get different results.

If the application is as single instance then the probabilities are 0 or 1, and only someone who knows the answer or guesses it correctly can have a correct model of the situation.

In physics (which is my concern in this thread), the physical description of a system completely specifies the ensemble, both of the model (the governing euations and boundary conditions) and of the application (the experimental setting). Thus both the predicted and the observable probabilities are objective. Whether one or both of therm may be unknowwn at particular times to particular people is completely irrelevant.

This objectivity is the strength of scientific practice in general, and of physics in particular. It allows anyone with access to the necessary information and equipment check the quality of any particular model with respect to the application it is supposed to describe.

On any sufficiently large sample the prior is irrelevant and only the data matters. So over an ensemble, even with subjective priors, the Bayesian approach gets user-independent (objective) posteriors.

But your ''sufficiently large'' may have to be far larger than mine.

 

[A. Neumaier]

Any subjective estimations by that person don't play a role; only the available information. It's objective (although not "invariant") in the sense that the calculation is according to standard rules of probability calculus and everyone (except you?) agrees about that calculation.

Bayesian techniques need both available information _and_ a prior. If the prior is not specified, it may depend on the persons subjective estimate, and calculations need not agree.

Thus if one gives strict rules for how to determine the prior from prior information (this is the case in the bayesian applications to animal breeding I had cited before), the calculated Bayesian estimates are objective.

In all other cases, the calculated Bayesian probabilities are subjective.

 

[Dale]

Specifying the prior defines the ensemble and hence makes the probabilities objective - no matter whether the prior is good or poor. ... This objectivity is the strength of scientific practice in general, and of physics in particular. It allows anyone with access to the necessary information and equipment check the quality of any particular model with respect to the application it is supposed to describe.

OK, I am fine with all of this. Your stance is even more acceptable to me than I had thought previously since you allow specified non-ignorance priors to encode available knowledge.

I don't see how it supports your claim that probability (in physics) does not depend on knowledge, but I agree with what you are saying.

 

[A. Neumaier]

O
I don't see how it supports your claim that probability (in physics) does not depend on knowledge, but I agree with what you are saying.

The model probabilities depend on the model, not on knowledge. Given the defintion of an ideal gas
(say) and specified values of P,, V, T, everything is determined - independent of the knowledge of anyone.

The application probabilities depend on the application, not on knowledge. Given the definition of the experimental arrangement specifying the application, everything is determined - independent of the knowledge of anyone.

So all probabilities encountered in physics are objective and knowledge independent.

What depends on knowledge is the assessment of how well a model fits an application, and hence the choice of a particular model to predict in a particular application. But this has nothing to do with probability, since it holds as well for deterministic models.

 

[Dale]

The model probabilities depend on the model, not on knowledge. Given the defintion of an ideal gas
(say) and specified values of P,, V, T, everything is determined - independent of the knowledge of anyone.

The application probabilities depend on the application, not on knowledge. Given the definition of the experimental arrangement specifying the application, everything is determined - independent of the knowledge of anyone.

So all probabilities encountered in physics are objective and knowledge independent.

What depends on knowledge is the assessment of how well a model fits an application, and hence the choice of a particular model to predict in a particular application. But this has nothing to do with probability, since it holds as well for deterministic models.

Sorry about this, I wasn't clear in my point above. My point is that the prior contains the knowledge, so if you are specifying the prior you are fixing the knowledge.

Suppose you have some quantity x and you want to determine if x depends on y or not. If you do not let y vary then you cannot claim that you have shown that x does not depend on y.

You claim that probability does not depend on knowledge, but knowledge is contained in the prior, and you require a specified prior. Similarly, when you said "anyone with access to the necessary information and equipment" you are fixing the knowledge. Since you are not allowing knowledge to vary you cannot make any conclusions about the dependence of probability on knowledge.

If you want to examine the dependence of physical probabilities on knowledge you must allow the priors and the information to vary across users.

 

[Studiot]

So all probabilities encountered in physics are objective and knowledge independent.

I have already said that I agree with much of what you posted.

However I maintain that your statements are too narrow.

Your response to my structural engineering examples clearly indicate you have no idea what a bridge assessment or limit state design theory involves.

Both are part of applied physics and properly represented in PF.

Since this is a Quantum section how about these questions

What is the probability that the Higgs will be discovered before the end of 2011?

Suppose I had asked a similar question in 1933

What is the probability that the positron will be discovered before the end of 1933?

 

[A. Neumaier]

Sorry about this, I wasn't clear in my point above. My point is that the prior contains the knowledge, so if you are specifying the prior you are fixing the knowledge.

You claim that probability does not depend on knowledge, but knowledge is contained in the prior, and you require a specified prior. Similarly, when you said "anyone with access to the necessary information and equipment" you are fixing the knowledge.

By the same argument, deterministic models would depend on knowledge. So if you insist on the correctness of your argument, why emphasize it in the probabilistic case but not in the determinstic case?

Moreover, a model may have a very unrealistic prior. In this case, probabilities depend - according to your view - on arbitrary assumptions or on misinformation rather than knowledge.

On the other hand, with my usage of the terms, everything is clear and unambiguous.

 

[A. Neumaier]

Your response to my structural engineering examples clearly indicate you have no idea what a bridge assessment or limit state design theory involves.

I have worked with structural engineers and am familiar with FORM and SORM techniques for limit state analysis, and with variations and alternatives for the assessment of reliability. This has no bearing on the theme.

Engineers calculate probabilities based on models applying to a large ensemble of cases parameterized by some parameters, and then specialize for a particular case by fitting the observed properties of a bridge to the model. The resulting parameter defines a subensemble of all conceivable bridges with characteristics matching the concrete bridge in question, and the safety probability refers to this ensemble, not to the specific bridge.

What is the probability that the Higgs will be discovered before the end of 2011?

Suppose I had asked a similar question in 1933

What is the probability that the positron will be discovered before the end of 1933?

In both cases, the answer is 0 or 1, and can be known only after the fact.

 

[Studiot]

and can be known only after the fact

This is the whole crux of my point.

You still have no idea what bridge assessment involves.

You are faced with the following scenario:-

You are presented with a specific bridge over a ravine. Not

a subensemble of all conceivable bridges with characteristics matching the concrete bridge in question,

As the Engineer you are asked

Will the bridge collapse if I drive my lorry over it?

This represents a one off unique situation and you have to make an assessment ie a subjective decision to allow for the fact that all the facts are not ( and probably cannot be ) known.

You did not read my post correctly either.

Studiot-
limit state design

A.Neumaier-
limit state analysis

Are you not familiar with the difference between analysis and the more difficult process of synthesis (or design)?

 

[Studiot]

In both cases, the answer is 0 or 1, and can be known only after the fact.

One of the direct consequences of this statement, if true, has deep philosophical implications because it implies determinism.
That is that any point in time the future is completely determined with a probability of either 1 or 0.

 

[SpectraCat]

One of the direct consequences of this statement, if true, has deep philosophical implications because it implies determinism.
That is that any point in time the future is completely determined with a probability of either 1 or 0.

I would go farther, and say that such statements *assume* determinism, in the sense that it is taken as a postulate, and thus cannot be proven or disproven.

 

[Dale]

By the same argument, deterministic models would depend on knowledge. So if you insist on the correctness of your argument, why emphasize it in the probabilistic case but not in the determinstic case?

No reason, except that the deterministic case is off topic and obvious.

Moreover, a model may have a very unrealistic prior. In this case, probabilities depend - according to your view - on arbitrary assumptions or on misinformation rather than knowledge.

Certainly, you could also make arithmetic errors or typographical errors, or you could misapply a formula, or you could use wrong formulas. Any time you use misinformation or misuse information in physics you will get nonsense. I don't think that is terribly interesting other than pedagogically.

On the other hand, with my usage of the terms, everything is clear and unambiguous.

Yes, but your definition is not the only valid and accepted definition of probability. Your claim is only true if you require probabilities to be defined only over ensembles. In that case I agree that the posterior probability does not depend on the prior so in that case you are indeed correct that probability does not depend on knowledge. Under the more general definition of probability the posterior can depend on the prior in any case where you do not have a sufficiently large number of observations.

 

[harrylin]

Bayesian techniques need both available information _and_ a prior. If the prior is not specified, it may depend on the persons subjective estimate, and calculations need not agree.

Thus if one gives strict rules for how to determine the prior from prior information (this is the case in the bayesian applications to animal breeding I had cited before), the calculated Bayesian estimates are objective.

In all other cases, the calculated Bayesian probabilities are subjective.

The case example I gave is objective since it has no subjective estimate as input. And what (nearly?) everyone calls "the probability" in that case depends on knowledge - take it or leave it.

 

[Dale]

Thus if one gives strict rules for how to determine the prior from prior information (this is the case in the bayesian applications to animal breeding I had cited before), the calculated Bayesian estimates are objective.

This is different from the fixed-prior case. Here, instead of having a fixed prior you have a family of priors with some hyper-parameters which are uniquely specified by available information. Note that in this case the probabilities are objective (user independent), but they do depend on knowledge.