"Strength" of the mean of the distribution curve

[icakeov]

My understanding of the distribution curves is very basic but I do have a couple of somewhat generic questions. I looked up a number of definitions but have had a hard time finding these specific answers:

- Is there an agreed on minimum number of samples that one needs to take to deem a result to be viable? Or does the result more depend on the data we get?

- Additionally, is there a certain "percentile" value of how far the standard deviation is from the mean (i.e. how "far" it is from it, or how low its value is), that would conclude that the mean value is "very likely" to happen?

What I am really aiming for with these questions is:

- If in science we are to declare something a "fact", how "precise" does the distribution curve have to be in order for us to declare it a fact? Does it have to occur 100% times? Or is there some "window" of probability that we take into account and what is that?

Apologies if the questions is sloppy or if it makes no sense.
Thanks for any feedback!

 

[FactChecker]

You have to decide how accurate you want your estimate of the mean to be and what probability you want that the true mean is within that range. Typically, you use the data to find a "confidence interval" where you can say that the mean is 95%, 97.5% or 99% like to be within that interval. The probabilities are assuming that you know the general form of the distribution of the thing you are measuring (like normal, poisson, etc.)

 

[icakeov]

Perfect! Thanks FactChecker!
I am guessing that 95% and over would be the value at which the mean value starts becoming a "fact"?
Not sure if this is the right forum for this question, but is this the step to conclude something is a fact? Or do facts have to have 100% confidence interval?

 

[FactChecker]

Perfect! Thanks FactChecker!
I am guessing that 95% and over would be the value at which the mean value starts becoming a "fact"?
Not sure if this is the right forum for this question, but is this the step to conclude something is a fact? Or do facts have to have 100% confidence interval?

For most things, you can never get 100% confidence. 95% is probably low because that will be wrong one out of every 20 times. There is a trade-off between the size of the confidence interval (the accuracy you want from your estimated mean) and the level of confidence that the interval contains the true mean. Different applications require different standards as far as the level of confidence and accuracy. You have to use your own judgment. An extremely conservative example (although they are not talking about a mean) is the level of confidence that a new particle has been found. For high-energy physics, they like 99.7% confidence before they say they have found "evidence of a particle" and 99.99997% confidence before they say that a new particle has been discovered.

 

[Dale]

Perfect! Thanks FactChecker!
I am guessing that 95% and over would be the value at which the mean value starts becoming a "fact"?
Not sure if this is the right forum for this question, but is this the step to conclude something is a fact? Or do facts have to have 100% confidence interval?

Each individual measured value is a fact. The mean of any collection of such values is also a fact. There is no lower limit, even the smallest piece of data is a fact.

 

[Stephen Tashi]

What I am really aiming for with these questions is:

- If in science we are to declare something a "fact", how "precise" does the distribution curve have to be in order for us to declare it a fact? Does it have to occur 100% times?

What do you mean by "it"? In order for something to be a fact, it would at least have to be expressed as a complete sentence.

If we consider a population of things, we can can make many different statements about the population. For example:

1) All the objects in the population have weight 102.3 and measurements that show a different value than 102.3 have errors in them.

or

2) The mean value of the objects in the population is 102.3, but there are objects in the population that have a weight different than the mean value.

 

[icakeov]

Thanks for all your responses, super helpful!
Yes, I was referring specifically to the mean value when talking about "it" being a fact.
I am understanding that every value on a graph can become a "fact" and have its "confidence interval" become very high with many measurements.

 

[icakeov]

For high-energy physics, they like 99.7% confidence before they say they have found "evidence of a particle" and 99.99997% confidence before they say that a new particle has been discovered.

Hi again, I was wondering if there is any published document that outlines the standards above that I can reference?
Many thanks again!

 

[FactChecker]

I don't know how authoritative this is. The standards of 3-sigma and 5-sigma correspond to "probabilities" of 99.7% and 99.99997%. These "probabilities" are the odds that a set of results that unusual or more unusual would not occur if there was not a new particle.

 

[gleem]

Here is another reference for 5 sigma in the context of the discovery of the Higg's Boson. You have to go down about half the article to get to the actual discussion of 5 sigma criteria.
https://www.science20.com/quantum_diaries_survivor/fundamental_glossary_higgs_broadcast-85365

 

[icakeov]

This is super helpful, thank you!

 

[WWGD]

The Central Limit Theorem relates what you obtain through sampling and the actual population parameters. Check out too the sampling distribution of the mean which gives you a confidence interval others mentioned here. Notice the larger the sample the narrower (i.e., "more confident") the estimate.

 

[Stephen Tashi]

- If in science we are to declare something a "fact", how "precise" does the distribution curve have to be in order for us to declare it a fact?

Measurements are only part of the facts of science. A paper giving a measurement should give a confidence interval for the measurement. Saying we have 95% interval doesn't tell us much until we also know the length of interval.

A paper publishing a measurement doesn't establish a scientific fact even if it announces 95% confidence. Usually, several other researchers must repeat the measurement and get consistent results before the measured value is accepted as a fact.

The famous and customary 95% confidence is a standard many journals require before they will consider publishing a result. However, the publication of a measurement in a journal does not guarantee the "scientific community" will regard the measurement as factual based on that single publication.

 

[icakeov]

That is SUPER helpful Stephen & WWGD, thank you so much!