Does a Probability of 1 Really mean a Dead Cert

[bhobba]

re the probability question I see that I should have the term "Ceterius Paribus" which means "use your brains" or "all things being equal". You don't need probability curve for tossing dice but I would have thought that everyone knows that you can't get a 7 if you only have one dice unless you have found some new type of dice.

See my post above.

I think it follows from the requirement to be able to apply the Kolmogorov axioms consistently. One can add any event with probability 0 or remove any event with probability 0, and the axioms still apply. I think the only reasonable conclusion, considering it can be modeled just as well without events of probability 0, is they can never occur.

Thanks
Bill

 

[Jano L.]

One can add any event with probability 0 or remove any event with probability 0, and the axioms still apply. I think the only reasonable conclusion, considering it can be modeled just as well without events of probability 0, is they can never occur.

Bill, you keep referring to Kolmogorov axioms. I do not see how they could possibly support your claim. Removal of an event with probability 0 may not change other probabilities, but this does not mean the event cannot happen. It just happens rarely enough so that its probability is zero.

Generally, I do not think you cannot use probability theory to decide what can and what cannot happen (what is the mathematical space of events one should consider). That kind of decision seems to be prior to the application of the probability considerations. In the probability calculations I know, the space of possible events is always given or assumed, and only then one thinks about how to assign or calculate probabilities.

 

[bhobba]

Bill, you keep referring to Kolmogorov axioms. I do not see how they could possibly support your claim. Removal of an event with probability 0 may not change other probabilities, but this does not mean the event cannot happen. It just happens rarely enough so that its probability is zero.

The reason I keep coming back to the Kolmogorov axioms is they are the foundational axioms of modern probability theory. The frequentest interpretation and Bayesian approach are either axiomatically equivalent to it or follow from the axioms.

When you apply the Kolmogorov axioms to an event space you assign some probability to each element of the event space. I can't see how that is in any doubt. Now suppose any element has probability zero. Remove it. The event space still obeys the Kolomogorov axioms. But that event now can never occur, be selected, or whatever, because it is no longer in the space. To me the only conclusion, if applying the axioms is to consistent, is that that event never occurs. The similar argument applies to probability 1 - you can remove all the other events leaving only the event with probability 1. It is the only event - it must occur.]

There are subtleties with continuous event spaces that you need things like a Dirac measure to overcome but I think the principles are still the same - you could probably come up with some kind of limit argument to justify it.

If you can't see it - well this sort of thing occurs every now and then - one person can't see what is obvious to another and basically you agree to disagree.

Thanks
Bill

 

[pbuk]

$P(x) = 0$ means that however many trials there are, $x$ will not happen. It doesn't say anything about whether $x$ cannot happen, simply that it does not happen.

But I don't see how you can equate this with the statement $x$ does happen, just very rarely (unless you mean infinitely rarely).

 

[bhobba]

$P(x) = 0$ means that however many trials there are, $x$ will not happen. It doesn't say anything about whether $x$ cannot happen, simply that it does not happen.

Every probability book I have ever read uses the Kolmogorov axioms to justify other appoaches like the frequentest approach through the law of large numbers derived from those axioms eg Feller's classic. So you really need to start from those axioms. And if you do that then you find, as I have pointed out, removing an event with probability 0 means the event space still obeys those axioms - but it can never occur. To me being able to apply those axioms consistently means it must never occur - otherwise you would not be able to remove it and the axioms still apply. I really can't see any way out of it.

Thanks
Bill

 

[HallsofIvy]

$P(x) = 0$ means that however many trials there are, $x$ will not happen. It doesn't say anything about whether $x$ cannot happen, simply that it does not happen.

But I don't see how you can equate this with the statement $x$ does happen, just very rarely (unless you mean infinitely rarely).

Neither of those statements is true. Take, for example, the experiment of choosing a number from the interval [0, 1] with uniform probability (probability that the number chosen is in sub-interval [a, b] is b- a). Then the probablity that any specific number is chosen is 0, but on every trial some number is chosen.

 

[atyy]

The reason I keep coming back to the Kolmogorov axioms is they are the foundational axioms of modern probability theory. The frequentest interpretation and Bayesian approach are either axiomatically equivalent to it or follow from the axioms.

When you apply the Kolmogorov axioms to an event space you assign some probability to each element of the event space. I can't see how that is in any doubt. Now suppose any element has probability zero. Remove it. The event space still obeys the Kolomogorov axioms. But that event now can never occur, be selected, or whatever, because it is no longer in the space. To me the only conclusion, if applying the axioms is to consistent, is that that event never occurs. The similar argument applies to probability 1 - you can remove all the other events leaving only the event with probability 1. It is the only event - it must occur.]

There are subtleties with continuous event spaces that you need things like a Dirac measure to overcome but I think the principles are still the same - you could probably come up with some kind of limit argument to justify it.

If you can't see it - well this sort of thing occurs every now and then - one person can't see what is obvious to another and basically you agree to disagree.

Thanks
Bill

But if the event with probability 0 occurred, then you could not remove it from the space. You only show that the axioms are consistent with the probability 0 event not occurring, but you don't show that it is inconsistent with the event occurring.

For example, in the Bayesian interpretation, the event can occur, it's just that your subjective belief is consistent and wrong.

 

[Jano L.]

I just found there is an idea similar to what Bill is claiming, namely that events with probability close to 0 should be treated as "morally impossible" (Bernoulli), which now seems to be advocated under the name "Cournot principle":

http://www.probabilityandfinance.com/articles/15.pdf

I think it does make sense as a practical rule of a thumb, especially if you consider that people do not really believe that a brick will fall on their head when they go along the street, or that they will ever throw six sixes in one roll.

Yet the small probability of such events (1/46656 for the dice) does not make them impossible event in the true meaning of the word "impossible". In fact, six sixes is as likely as any other physical result, because they are all equally probable.

 

[Stephen Tashi]

The q

Every probability book I have ever read uses the Kolmogorov axioms to justify other appoaches like the frequentest approach through the law of large numbers derived from those axioms eg Feller's classic. So you really need to start from those axioms. And if you do that then you find, as I have pointed out, removing an event with probability 0 means the event space still obeys those axioms - but it can never occur. To me being able to apply those axioms consistently means it must never occur - otherwise you would not be able to remove it and the axioms still apply. I really can't see any way out of it.

I'll harp repetitively on assertions in my previous posts.

There is nothing in the Kolmorogov axioms for proability (as they are given in modern measure theory texts) about events actually "happening" or "not happening". (And, as a technical matter you cannot abitrarily remove an event from a measure space because a measure has to be defined on a "sigma algebra of sets". If you remove an event, you must redefine the sigma algebra. If you are successful, this puts you in a different measure space.)

Certainly any probability book should mention applications of probability theory that use the concept of probabiliy in situations where the observed frequency of something happening or not happening is the thing of interest. However, there is nothing in the axioms of probability theory that asserts that you can or cannot make such observations. The use of the term "event" for a set in does not (in mathematics) imply the set is a measurement that can actually be made.

This thread is very much like discussions of such weighty topics as "Are dy and dx numbers?" or "Does .9999... = 1?". It involves a variety of personal opinions about mathematics, some presented in a non-mathematical way. I dont' object to such threads, but I do object to claims that personal opinions are backed by standard mathematical definitions. If we want to talk about the axioms of probability theory, we should talk about the axioms as they are presented formally, not about the presentations made in elementary texts that rely on studen't's inituitions about thinking of an "event" as something that actually happens or doesn't.

 

[bhobba]

There is nothing in the Kolmorogov axioms for proability (as they are given in modern measure theory texts) about events actually "happening" or "not happening".

I find I am forced to agree with that.

What I am talking about is when the axioms are presented in books like Feller's examples to cement their meaning are given, as well as the development presented in the text. Those invariably concern the event space having events that occur or something similar. Once that is done then my argument applies.

Although I have never seen an example of an event space that can't be interpreted this way I do not doubt they exist.

Out of curiosity though can anyone think of an example?

Thanks
Bill

 

[Stephen Tashi]

Once that is done then my argument applies.

According your agument,if we are drawing numbers from a uniform distribution on [0,1], I can remove the possibility of drawing \pi/4 since it has probability 0. If I proceed to remove all events that have zero probability, there won't be any events left.

I'm not saying this mathematically invalidates your conclusion in a practical sense because the axioms of probability theory don't say whether you can or cannot take random samples from a uniform distribution in the first place.

I think any practical implementation of sampling amounts to sampling from a discrete random variable with a finite number of values. I don't think it is controversial to say that events with probability 1 in such probability spaces always happen if the space correctly models the events. (For example, if a coin has a possibility of landing on its edge and your probability space only includes the events "heads" and "tails" then you have a wrong model. )

When we consider spaces with infinite outcomes (such as space of infinite sequences of coin tosses or random draws from a uniformly distributed random variable) then whether events with probability 1 always happen cannot be tested by practical methods. (It's an interesting question whether Nature herself can take samples from such distributions.)

There are certain questions in mathematics that are "undecided". But to be any kind of question, it must be precise. For example if a statement about all groups is "undecided" then the statement is precise enough that you can look at some particular group and see if the statement is true or false about it. The question of whether whether an event with probability 1 is a "dead certainty" is not an undecided question in probability theory. It isn't even a question at all! It is not precise enough, within the terminology of probability theory, to have a specific meaning.

There is a theory called "possiblity theory" that uses the terminology of "possible" and "necessary" events. I don't know if people have worked on combining it with probability theory.

 

[atyy]

I don't think it is controversial to say that events with probability 1 in such probability spaces always happen if the space correctly models the events. (For example, if a coin has a possibility of landing on its edge and your probability space only includes the events "heads" and "tails" then you have a wrong model.

I think Jano L's argument is that we should say it the other way: if an event is certain, it is correctly modeled as having probability 1. (But not that if an event has probability 1, then it is certain.)

For the case of an event that is modeled as having probability zero, but occurred once, maybe the practical way to decide whether to reject the model is some "traditional way" like the chi-squared test? I haven't tried it out yet.

 

[bhobba]

According your agument,if we are drawing numbers from a uniform distribution on [0,1], I can remove the possibility of drawing \pi/4 since it has probability 0. If I proceed to remove all events that have zero probability, there won't be any events left.

Indeed continuous event spaces are problematical - I think a rigorous development along the lines of my argument would need some kind of limit procedures and the introduction of distribution theory - here I mean distributions in the Schwartz sense ie the Dirac Delta Function etc.

Thanks
Bill

 

[atyy]

I attempted to run a chi-squared test with

Category A, observed = 1, expected = 0
Category B, observed =10000000000, expected = 10000000000

I got as the result:
The chi-square test is not possible when any of the expected values are zero.

Which supports the interpretation that if an event with probability 0 occurs, then the model is inadequate (but not because it fails the hypothesis testing).