Two different types of probability

[disregardthat]

This is the introductory definitions of two different types of probability:

Frequentists talk about probabilities only when dealing with experiments that are random and well-defined. The probability of a random event denotes the relative frequency of occurrence of an experiment's outcome, when repeating the experiment. Frequentists consider probability to be the relative frequency "in the long run" of outcomes.[1]

Bayesians, however, assign probabilities to any statement whatsoever, even when no random process is involved. Probability, for a Bayesian, is a way to represent an individual's degree of belief in a statement, given the evidence.

I am only familiar with the first concept, probability based on the empirical evidence (for example a statiscial survey may help approximate the probability of a person holding a certain opinion), or determined with knowledge of all factors which can affect the outcome (for example a thought experiment by flipping a perfect coin, e.g. the unit disc thrown on a plane).

Bayesian probability states that there is a certain probability to every statement. I find this hard to justify, given that many statements is either true or false. For example, how can one determine the probability that there is 4 apples in a given basket? The statement becomes: "A: The basket b contains 4 apples". Do bayesian probability state that this true or non-true statement has a probability value attached to it, so that 0 < P(A) < 1 based on a degree of belief? P(A) should in my opinion be equal to either 1 or 0, determined by the truth of the statement. The probability that a basket chosen arbitrarily contains 4 apples is another thing. The statement becomes: "B: An arbitrarily chosen basket contains 4 apples". This, however does have a probability value 0 \leq P(B) \leq 1 attached to it. I can agree that bayesian probability makes sense accompanied with extensive statistical data, but to point to a certain case which is either true or false - isn't it formally wrong to assign a probability value to it?

The above may be hair-splitting (in a practical sense), but my problem comes when bayesian probability is expanded to statements regarding the truth of undeterminable events, such as the existence of ghosts, angles, etc. When one places a probability value to a statement that is by definition undeterminable, how can this be justified by the degree of one's belief?

 

[mgb_phys]

My understanding was that Bayesian is useful for looking at probablilty the other way around.
Classical stats says if you know there are 3 green apples in a basket and 1 red apple - what's the probability of pulling out a red one.
Whereas Bayesian is "if I pull out a red apple, what is the most likely proportion of red apples in the basket"
That's why it's useful in science, given this partial data - what's the likelyhood that my model is correct.

 

[disregardthat]

Yes, I agree with you on that. Statistical models is very useful in physics and mathematics. But formally, one should keep the two apart. Frequentism and Bayesian probability is in my opinion too different to be mixed with each other. I am speaking in definitions here, and not how they apply to theories and practical problems.

However, my main point was that I cannot understand how one can justify to make a claim about something that is by definition undeterminable based on a degree of belief, however that may be defined.

 

[Lord Ping]

According to Bayesians, Bayes' theorem can be used to calculate how a rational agent should update her degree of belief in some proposition A, given some new evidence B.

It says:

P(A|B) = P(B|A)*P(A)/P(B)

The probabilities here are usually interpreted as subjective "degrees of belief".

In plain English, this means that your degree of belief in A after obtaining evidence B depends on your degree of belief that B would obtain given A, your prior degree of belief in A, and your prior degree of belief in B.

In even plainer English: evidence that would be expected given the truth of your theory, but would be surprising otherwise, is great confirmation of your theory when it turns up.

Of course the theorem itself is an analytic truth. The novelty is in applying it as an account of how we rationally update our degrees of belief. I think it's a very good account indeed.

It would be wrong to suppose that someone who likes Bayesian confirmation theory also interprets all probabilities as degrees of belief. Maybe some people do, but I think there is plenty of room for a pluralist approach. If I say "It's probably about 11pm" I'm talking about a degree of belief. If I say "This nucleus has probability 0.1 of decaying in the next minute" I may well be talking about some kind of objective physical propensity.

To answer your original question, Bayes' theorem can't tell you what degree of belief to assign to any claim at face value. It can only tell you how to update your belief when evidence comes along.