Understanding the Concept of Almost Surely in Probability

[LegendLength]

"Almost Surely" vs "Surely"

I'm struggling to understand the concept of "Almost Surely" in probability. I've read the wiki article ( http://en.wikipedia.org/wiki/Almost_surely ) and looked at some old posts on this forum but there's something I still don't get:

Given a theoretical coin that lands on either heads or tails: If you flip it infinite times then you could say it will almost surely land on heads at least once. That is, the infinite sequence of T, T, T, T, T etc. is 'almost surely' not going to happen.

What I don't understand is if you have a theoretical coin that can be H or T, and it only lands on T, how can it ever be called "a coin that can be H or T". Isn't the definition wrong? Shouldn't it be called "a coin that can only ever be T"?

Sorry if that's not a very rigid argument as I don't have the skills to put it in structured mathematical form.

 

[Stephen Tashi]

What I don't understand is if you have a theoretical coin that can be H or T, and it only lands on T, how can it ever be called "a coin that can be H or T". Isn't the definition wrong? Shouldn't it be called "a coin that can only ever be T"?

How is that related to your example of the fact that a fair coin "almost surely" doesn't only land on T ? You haven't linked to any article on probability that did what you described.

In a probability problem, you can define a random variable, such as a coin. You can then ask questions about what is true if the random variable happens to take on certain values when you sample it. The fact that these questions assume it takes on certain outcomes need not contradict the fact that you defined a random variable. If the questions asserted that the variable always produced certain outcomes, that would be a different matter.

 

[D H]

Suppose you have a fair die. The probability that you will roll at least one 6 in N rolls is 1-(5/6)^N. This approaches 1 in the limit N→∞. That you will roll a 6 in an infinite number of rolls "almost surely" must happen.

What's the probability that you won't roll a 7? It's one, no matter how many times you roll the die. That you will never, ever roll a 7, even with an infinite number of rolls, "surely" must happen.

 

[HallsofIvy]

The term "almost surely" simply does not apply to situations that have a finite number of outcomes (discrete probability)- in which case a probability of 1 means "must happen". But in situations with an infinite number of outcomes, and a continuous probability distribution, a probability of 1 does not necessarily mean that.

For example, if we were to select a number from the set of all real numbers from 0 to 1, we can calculate a probability that the number is in a given interval, but the probability of choosing any specific number is 0- even though the number chosen must some number. That is, it is quite possible that the number we choose is, say, 1/2, but the probability of getting 1/2 is 0. The probability of getting any number other that 1/2 is 1. We would say that it is "almost sure" that the number will not be 1/2.

By the way, as long as we are talking about intervals, such as [a, b], we can use simple measure b- a. But if we want to be able to work with more complicated sets, such as the "set of all rational numbers between 0 and 1" or the "set of all irrational numbers between 0 and 1", we have to use the more general "Lebesque measure". It is really from the theory of "Lebesque measure" that the general concept of "sets of measure 0" and "almost" come. A theorem about real numbers is said to apply to "almost all" numbers if it applies to all except, possibly, a set of measure 0.

 

[Stephen Tashi]

There is a conflict between the title of the OP and the question that the post actually asks!

The term "almost surely" simply does not apply to situations that have a finite number of outcomes (discrete probability)- in which case a probability of 1 means "must happen".

I can agree with the spirit of that remark, but doesn't measure theory also apply to the probablity measures defined by discrete random variables? Don't all the definitions and theorem involving "almost surely" work for any probability space?

if we were to select a number from the set of all real numbers from 0 to 1

It's interesting to note that we can't do this - at least we can't sample from a non-discrete probability distribution such as the uniform distribution on [0,1]. All the methods we have (whether algorithms or physical measurements) have finite precision. One way to look at the nature of abstract probability theory is to say that it's the natural outcome of what happens when human beings deal with idealized concepts, things that are imagined as "limits" of processes that we can actually do.

 

[LegendLength]

Yes I apologize because I wasn't sure I was even understanding the concept correctly. So if it is just for non-discrete type problems could I try to ask my question in a different way?

For any coin that's flipped an infinite number of times, it must land on H or T at least once? In other words if you say something has the potential to happen, then it must happen over the course of an infinite number of tries. Otherwise that outcome never really did have the 'potential' to happen (i.e. you described the object incorrectly).

I realize this might be a more philosophical question. I'm just trying to understand it.

 

[D H]

The concept surely makes sense if you understand measure theory, and you almost surely need to learn measure theory before you can truly understand the concept.

Do you understand why almost all reals are irrational? Why f(x) = 1/(\Pi_i (x-a_i)) is defined almost everywhere? There's a close coupling between the terms "almost all", "almost everywhere", and "almost surely", and they are all related to measure theory.

 

[Stephen Tashi]

In other words if you say something has the potential to happen, then it must happen over the course of an infinite number of tries. Otherwise that outcome never really did have the 'potential' to happen (i.e. you described the object incorrectly).

if by "potential to happen", we mean that an evemt has non-zero probablity then n the case of the coin toss, we didn't say that an infinite number of tails had the potential to happen, The computations for any finite sequence of all-tails give a finite answer, but there is difference between something that is "true for any finite number" and something that is true "for an infinite number".

It's true that you could consider the "space of events" to be the set of infinite sequences of letters, consisting H's and T's. If you do that and assume they come from tossing a fair coin, you can't assign a non-zero probability to any individual sequence. All sequences would need to have the same probability and if you set that to be any nonzero number, the total of the probabilities would not be finite, much less 1.


A different example would be the scenario of drawing a random number U from the uniform distribution on [0,1]. You might draw any number in that interval and each individual number in the interval has zero probability of being drawn.

If you interpret "the potential to happen" only to mean that an given event is in the event space of probability scenario (and not as a claim that the event has non-zero probability) then you might suspect that a theoretical construct (like the probabiity density function of U) might be inherently paradoxical and lead to logical contradictions because it assigns zero probability to every individual number. (It's rather like how some people suspected that calculus would turn out to be inherently paradoxical since , as it was originally formulated, it involved using quantities that were treated as finite, but became zero.)

In my opinion, there isn't much deep philosophy involved in the idea of events with zero probability making up the space of events. As I mentioned before, you can't really draw a random number from a uniform distribution on [0,1]. You also can't toss a coin an infinite number of times. So both infinite sequences of coin tosses and the uniform distribution on [0,1] are concepts that we imagine. When we claim something is true about these concepts, the claim should pass two tests. First, there is the practical test of whether it makes sense as "limiting case" of things we can do (like toss a coin a large number of times or draw a 5 digit decimal at random from the numbers in [0,1]. Second, there is the question of whether the claim is logically consistent with the assumptions and known results of probability theory ( I mean the mathematical assumptions not the man-in-the-street assumptions.) As other posters have suggested, if you want to see the theory of probability developed in a rigorous fashion then have to study how it is developed with measure theory. ( I suppose that implies that if you are studying probability theory as it is developed in lower level probability courses, then you have every right to be suspicious!)

 

[HallsofIvy]

There is a conflict between the title of the OP and the question that the post actually asks!
I can agree with the spirit of that remark, but doesn't measure theory also apply to the probablity measures defined by discrete random variables? Don't all the definitions and theorem involving "almost surely" work for any probability space?

Well, yes, measure can be applied to any set. But in the discrete case, most of the power of "measure theory" is not needed. The measure reduces to counting. And "almost everywhere" reduces to "everywhere".

It's interesting to note that we can't do this - at least we can't sample from a non-discrete probability distribution such as the uniform distribution on [0,1]. All the methods we have (whether algorithms or physical measurements) have finite precision. One way to look at the nature of abstract probability theory is to say that it's the natural outcome of what happens when human beings deal with idealized concepts, things that are imagined as "limits" of processes that we can actually do.

If you are talking about some algorithm to actually pick a number, yes. But I was talking about the "pure mathematics" concept. We can certainly talk about choosing a number from 0 to 1 with all numbers being equally likely even if we cannot devise a method for actually doing so.