B Probability; what is "the long run"?

[Cliff Hanley]

Stephen Tashi, you said,

“Technically, probability theory gives you no guarantees about any event (or series of events) actually happening. Probability theory merely uses the given probabilities to assign probabilities to other events and series of events.”

Q. So those who say, for example, that we will see the expected value regards non-reds at roulette (19/37) in the long run, or, given sufficient spins, are in error? Can they only correctly say that we will probably see...’

You also said,

“When people assert that some event will happen in the long run, this is an assertion about the physics or other applied science involved in a problem, not a theorem of mathematical probability theory.”

Q. Would I be correct in presuming that the physics of a large number of spins of a roulette wheel in advance of said spins would be impossible to know?

“The best mathematical probability theory can do in such situations is to say the limit of the probability of an event approaches 1 as the "length" of the "long run" approaches infinity.”

Q. What does the ‘limit of the probability’ mean?

Q. By infinity do you mean an infinite number of spins?

Q. If so, how can an infinite number be approached? Isn’t a single spin just as close to an infinite number of spins as, say, a centillion (10^303) spins - in that after the former there are just as many spins ahead of us as in the case of the latter?

 

[phinds]

Stephen Tashi, you said,

“Technically, probability theory gives you no guarantees about any event (or series of events) actually happening. Probability theory merely uses the given probabilities to assign probabilities to other events and series of events.”

Q. So those who say, for example, that we will see the expected value regards non-reds at roulette (19/37) in the long run, or, given sufficient spins, are in error? Can they only correctly say that we will probably see...’

I think you have a good point there. When the probability can be shown to be 1 to within some large number of decimal places (and in this case it can be a VERY large number) we tend to talk in practical terms as though the outcome is a certainty, but I believe you are right that we should say will probably see since it is not mathematically guaranteed.

 

[mfb]

That's why I used "very likely". As in, a probability of more than 99.99999999999999999999999999999999999999999999999999999 % in the previous post (and I could add as many 9 as the forum would allow).

 

[Stephen Tashi]

Q. So those who say, for example, that we will see the expected value regards non-reds at roulette (19/37) in the long run, or, given sufficient spins, are in error? Can they only correctly say that we will probably see...’

Yes, if were talking about mathematical probability, we can only say "we will probably see...".

You also said,

“When people assert that some event will happen in the long run, this is an assertion about the physics or other applied science involved in a problem, not a theorem of mathematical probability theory.”Q. Would I be correct in presuming that the physics of a large number of spins of a roulette wheel in advance of said spins would be impossible to know?

It would take someone familiar with the construction of roulette wheels to answer that question.

“The best mathematical probability theory can do in such situations is to say the limit of the probability of an event approaches 1 as the "length" of the "long run" approaches infinity.”

Q. What does the ‘limit of the probability’ mean?

Q. By infinity do you mean an infinite number of spins?

Are you familiar with the mathematical definition of a "limit of a sequence"? The limit of a sequence of probabilities is just a special case of that concept.

Q. If so, how can an infinite number be approached? Isn’t a single spin just as close to an infinite number of spins as, say, a centillion (10^303) spins - in that after the former there are just as many spins ahead of us as in the case of the latter?

The mathematical definition of a "limit of sequence" is specific and technical. That's what we need to look at.

 

[zinq]

One fundamental fact about "the long run" is this:

Suppose we're interested in some event E with a positive probability Prob(E) = p > 0 of its occurring on anyone of many identical, independent trials.

And suppose we would like to conduct enough trials so that the probability of E occurring at least once is very high, say greater than Q = 1 - ε for some arbitrarily small quantity ε > 0.

Then, there always exists some positive integer N such that if N trials are conducted, the probability of E occurring at least once is at greater than the preassigned probability Q.

(((
Proof: The probability of E not occurring on one trial is 1-p. So the probability of E not occurring at all on N independent trials is (1-p)^N.

This means that the complementary probability, of E occurring at least once in N trials, is 1 - (1-p)^N.

If we try to solve for N in the equation

1 - (1-p)^N = Q

we get:

(1-p)^N = 1-Q

and so

N = log(1-Q) / log(1-p)

where the log is to any fixed base, so we may as well have it be the natural log (base e).

But, there is no reason that this expression for N need be an integer. Hence we must increase N to the next integer to have a sensible equality, while still ensuring that our ultimate probability of E occurring at least once is greater than Q, as desired:

N = floor[1+log(1-Q) / log(1-p)]

It follows that

(1-p)^N < 1-Q

and so the probability 1 - (1-p)^N of E occurring at least once in N trials satisfies

1 - (1-p)^N > Q

as desired.
)))

 

[BWV]

For more on large numbers go try to find a copy of Moby Dick here (it does exist!):
https://libraryofbabel.info/

 

[Cliff Hanley]

MrAnchovy, you said,

“You can google this [the record for successive non-reds in actual play ], although I'm not sure how reliable the answers would be.”

Thanks. I got this from allaboutbetting.co.uk; ‘In Monte Carlo in 1913 black came up 26 times in a row and in New York in 1943 red came up 32 times in a row.’

The odds for those, if I’ve got this correct, are (18/37)^26 and (18/37)^32 respectively. Which is 7.3087029 x 10^-9 (which is approx a 7 in a billion chance; and, 9.6886885 x 10^-11 (which is approx 1 in a trillion chance).

You also said,

“A wheel spun once a minute for 300 years will spin about 130 million times.”

I think 300 years at once a minute would be nearer 160 million years but we’ll go with 130 million for the sake of your example.

“The chance of 28 successive non-reds is about 1 in 127 million. However this doesn't mean that a run of 28 will happen, or that a run of more than 28 will not happen.”

Q. It’s just the mathematical probability, yes?

“...if everyone on Earth spent their whole lives playing roulette until the Earth's atmosphere is burned off by the Sun they are likely to see a hundred non-reds, but a thousand are unlikely before the universe reaches heat death (caution - I did these calculations rather carelessly).”

Q. I’ll heed your caution and not do any numbers on this one. Perhaps another poster can answer this one more accurately?

“We can't [know when the long run will give us the expected value] but we can say that the more trials we do the observed proportion is more likely to approximate the theoretical proportion more closely.”

Q. Only ‘more likely’?

Q. If it’s the case that in theory we could see black come up say, a million times in a row (or more), is it true that (given sufficient spins) it would be the case in practice?

“See the above comment on heat death [re the practice matching the theory re a million (or more) non-reds in a row].”

Again, I’ll heed your caution re you saying that you did your calculations rather carelessly.

“You would gain more understanding by learning about this section of probability (binomial probability/Bernouilli trials) and doing the calculations yourself.”

Yes, thanks, I’ve been checking out Bernoulli trials, and attempting many calculations myself, but I find that a bit of both (studying and asking questions) helps me to understand this better.

Thanks for the reply.

FactChecker, you said,

"The long run" depends on how close you want to get to 18/37. Even then, there is only a probability that it will get as close as you specify. So you have to frame the question this way: "How large of a sample size would it take so that the probability of the sample result being within xxx of its theoretical value is yyy?". The answer to that question would give you the sample size that you could call "the long run" for that case.”

Q. Is the sample size simply the number of trials (in this case, the number of spins)?

Q. And even if we carried out a centillion (10^303) trials (spins) we would still only probably see the expected value (or very close the expected value)? And that we might see a centillion successive non-reds instead (and if we did sufficient trials - spins - we would probably see a centillion successive non-reds)?

Q. And are you saying that there is no long run as such, that there’s only the long run for case x, the long run for case y, etc etc etc? If so, it seems to me that many people use this terms inaccurately; do you find this also?

You also said,

“Suppose you want to say that there is a probability of 95% that it is within 0.01 of 18/37.”

Q. Does ‘within 0.01 [1%] of 18/37’ mean between 18.17982/37 and 17.82018/37?

[I got those numbers by the following method;

18/37 x 0.01 = 0.00486...

To add 18/37 + 0.00486... we need to find a common denominator, which is 37, which gives us;

18/37 + 0.17982/37 which equals 18.17982/37

And;

18/37 – 0.17982/37 which equals 17.82018/37]

Q. Are the above calculations correct?

You added,

“Then there is an equation that tells you how many trials that would take. So it tells you what "the long run" would mean for that case.”

Q. What is the equation?

Q. Does this mean that if we carry out the number of trials that the equation tells us to there will be a 95% chance that we will get between 18.17982/37 reds and 17.82018 reds?

WWGD, you said,

“I think the long run here would be described by the LLN --Law of Large Numbers.”

Q. Do you think that term (LLN) is rather vague for mathematics given large is a relative term?

 

[micromass]

Q. Do you think that term (LLN) is rather vague for mathematics given large is a relative term?

No, of course not, since the law of large numbers doesn't just say "if we generate a lot of data, we will get closer to the true value". It has an actual mathematical meaning which is very precise.

 

[mfb]

9.6886885 x 10^-11 is about 97 in a trillion, or roughly 1 in 10 billion.

I get the impression the questions repeat.

 

[chiro]

For the OP:

I think the best way to understand this is through the Statistical Inference theorem and Cramer-Rao lower bound that relates the information matrix (known as Fischer information) to the statistical variance of an estimator.

The idea is simple - increase information content and reduce statistical variance.

If you understand how the information content grows then you understand how the estimate converges to the population value and how it does so consistently with variance also shrinking to zero.

Note that the information matrix makes no assumption about things like whether samples have completely independent data points, partially correlated ones and even completely correlated values. If you have collinearly correlated sample points then information density will not increase.

If you want to understand the nature of how things converge in more detail then you need to understand how the information density of that sample increases as you take more data points on to what already exists. This includes dealing with situations of correlated data and using some sort of inequality with your assumptions of how correlated your data points will be to others to estimate probabilistically when your variance will be in some interval range.

The above statistical inference theorem is general - but as others have mentioned, you will need to add further information constructs like distribution models to help make things more specific and also measurable.

 

[Cliff Hanley]

gill1109, you said,

“Probability theory tells us that if we play infinitely often we will certainly get to see, *infinitely* many times, a hundred non-reds in succession. And a thousand. And a million. And a billion, and a trillion.”

Q. I’ve heard that the concept of infinity is a complex one (and that there are different ideas about it in maths from those in physics); do you mean here simply a number of trials without end – a purely hypothetical, and impossible, situation?

You also said,

“You name it, you will get it ... with probability 1, infinitely many times.”

Q. So we would be guaranteed to see a centillion successive non-reds (10^303)? And see it an infinite number of times?

Q. What about an infinite number of non-reds in succession; what is the probability of that given an infinite number of trials?

“The strong law of large numbers.”

Thanks. I looked it up. But the maths is too advanced for me at the moment. I will go back to it (many times I expect) as I learn to deal with more and more complex maths.

 

[gill1109]

gill1109, you said,

“Probability theory tells us that if we play infinitely often we will certainly get to see, *infinitely* many times, a hundred non-reds in succession. And a thousand. And a million. And a billion, and a trillion.”

Q. I’ve heard that the concept of infinity is a complex one (and that there are different ideas about it in maths from those in physics); do you mean here simply a number of trials without end – a purely hypothetical, and impossible, situation?

You also said,

“You name it, you will get it ... with probability 1, infinitely many times.”

Q. So we would be guaranteed to see a centillion successive non-reds (10^303)? And see it an infinite number of times?

Q. What about an infinite number of non-reds in succession; what is the probability of that given an infinite number of trials?

“The strong law of large numbers.”

Thanks. I looked it up. But the maths is too advanced for me at the moment. I will go back to it (many times I expect) as I learn to deal with more and more complex maths.

Yes you are guaranteed to see a centillion successive non-reds (10^303), an infinite number of times. But not an infinite number of non-reds in succession. That has probability zero.

 

[Stephen Tashi]

Yes you are guaranteed to see a centillion successive non-reds (10^303), an infinite number of times. But not an infinite number of non-reds in succession. That has probability zero.

Any particular infinite sequence of outcomes has probability zero. So are we guaranteed not see any particular infinite sequence? Mathematically, probability theory doesn't define the "actual" occurrence of events, so it doesn't express any guarantees about that topic. If we are discussing a physical experiment, we can discuss how probability theory is interpreted in that experiment.

 

[Cliff Hanley]

phinds, you said,

“Yes, the probability of getting any string you can name approaches 1 as the number of trials approaches infinity, but since we can't actually do an infinite number of trials, we can't ever get an absolute certainty (probability = 1.0)”

But even if, as a thought experiment, we imagine doing an infinite number of trials, every single spin could be non-red (P[non-red] = 19/37). Or every spin could be red (P [red] = 18/37). Or every spin could be green (P [green] = 1/37). So even with an infinite number of trials there are no certainties. No?

 

[Cliff Hanley]

Stephen Tashi, you said,

“There is a further distinction between "actually" and "certainly". If we "actually" took a sample from a normal distribution and the value was 1.23 then an event with probability 1 ( namely the event "the value of the sample will not be 1.23) failed to "actually" happen.

Q. Would you explain what ‘normal distribution’ means in language suitable for a maths novice please?

Q. Likewise for ‘the value was 1.23’?

(I Googled it but the explanation was in language too advanced for me at the moment).

 

[Cliff Hanley]

FactChecker, you said,

“For any number, N, we can always continue long enough for N+1 occurrences.”

Q. Do you mean that if we substitute N for say, 1000, and carry out sufficient trials, we will (probably) see 1000 successive non-reds (or reds, or whatever); and if we continued with the trials for long enough we will see 1001 successive non-reds/whatever, and 1002 non-reds, and 1003, etc etc etc?

You added,

“The probability of infinitely many occurrences is 1 because the probability of finite occurrences is 0.”

Q. Do you mean the probability of infinitely many occurrences generally? Or the p of infinitely many occurrences of a specified sequence / sequences? I’m guessing the former given that you’ve also said that the p of finite occurences is 0?

Q. If we did do an infinite number of trials (assuming that we can somehow survive the ‘death’ of the Sun and whatever other life-threatening cosmological - or other - events will take place in the future, ie, assuming that we - us now and whatever we evolve to become - can survive eternally) what would we be likely to see in terms of sequences of non-reds / reds / etc etc?

 

[Cliff Hanley]

mfb, you said,

“As examples for the 49%/51% question:
After 1000 rolls, the chance to be within 1% of this result (so somewhere from 48/52 to 50/50) is roughly 50%.
After 10000 rolls, the chance to be within 1% is about 95%.
After 100,000 rolls, the chance to be within 1% is larger than 99.9999999%.”

Q. How do we work this out?

After 1 million rolls, the chance to be within 0.1% (between 48.9/51.1 and 49.1/50.9) is about 95% and the chance to be more than 1% away is completely negligible.
After 100 million rolls, the chance to be within 0.01% (between 48.99/51.01 and 49.01/50.99) is about 95%.

Q. And this?

You also said,

“Draw a random number from a uniform distribution over the real numbers in the interval [0,1].”

Q. I Googled ‘real numbers’ to discover that they are any number that we can find on a number line (including integers, fractions, decimals, irrational numbers such as pi, etc); but I was left wondering if these are real numbers what are non-real numbers; so, what are non-real numbers? And why is the distinction ‘real’ important when referring to real numbers?

You added,

"The number is not 0.5" has probability 1...”

Q. Is this because ‘uniform distribution’ (in this example) means the numbers 1,2,3 etc (with no fractions/decimals in between)?

“...but it is not certain.”

Q. Why is it not certain? If the pool of numbers from which we draw does not include 0.5 why is it not certain that the number will not be 0.5?

“It is almost certain.”

Q. Are you saying that ‘probability of 1’ ≠ ‘certainly will happen’?

 

[berkeman]

Thread closed temporarily for Moderation...

 

[Mark44]

@Cliff Hanley, the question you started with in this thread, "what is the long run?" has been asked and answered, so I am closing this thread.

But what is the LONG RUN?

You have asked a number of other questions as well, some of which can be answered by a web search, but others of which will require a fair amount of time studying the relevant mathematics subjects. This forum is not meant to take the place of academic studies.

Q. What does the ‘limit of the probability’ mean?

Based on other threads of yours that I have seen, your mathematical expertise is not yet at the stage where an answer would be meaningful to you.

Q. Would you explain what ‘normal distribution’ means in language suitable for a maths novice please?

Did you do a web search for this term? It is not the purpose of Physics Forums to be a tutorial for large swaths of probability theory.

Q. I Googled ‘real numbers’ to discover that they are any number that we can find on a number line (including integers, fractions, decimals, irrational numbers such as pi, etc); but I was left wondering if these are real numbers what are non-real numbers; so, what are non-real numbers? And why is the distinction ‘real’ important when referring to real numbers?

These are very basic questions. You should put in the effort at researching these questions rather than rely on PF as a tutorial service.

Thread closed.