- #1
- 3,796
- 1,853
- TL;DR Summary
- Term for unknown alternate causes in experiments.
You collect 200 data points, 100 with input ##A## and 100 with input ##A'##.
For all 100 ##A## you get output ##C## and for all 100 ##A'## you get ##C'##.
That's way more than 5 standard deviations, so you're ready to publish...
But not really.
Because what matters is not just that input ##A## is well-correlated to ##C## but that there is no ##B## that is also correlated to ##C##.
In the simplest (and probably worse) case, we may have collected all of the results from ##A## before any of the results from ##A'## - so there was a particular time when ##C## outputs ended and ##C'## outputs started. That would create many potential ##B##s: Room temperature or humidity were rising or falling; the experimenter was getting tired or more practiced; the apparatus broke or was wearing out.
So you are not ready to tout your 5 standard deviations until you run the experiment in a manner that reduces the likelihood that there could be such an unknown alternate cause. Is there a common (or otherwise better) term for such "unknown alternate causes"?
For all 100 ##A## you get output ##C## and for all 100 ##A'## you get ##C'##.
That's way more than 5 standard deviations, so you're ready to publish...
But not really.
Because what matters is not just that input ##A## is well-correlated to ##C## but that there is no ##B## that is also correlated to ##C##.
In the simplest (and probably worse) case, we may have collected all of the results from ##A## before any of the results from ##A'## - so there was a particular time when ##C## outputs ended and ##C'## outputs started. That would create many potential ##B##s: Room temperature or humidity were rising or falling; the experimenter was getting tired or more practiced; the apparatus broke or was wearing out.
So you are not ready to tout your 5 standard deviations until you run the experiment in a manner that reduces the likelihood that there could be such an unknown alternate cause. Is there a common (or otherwise better) term for such "unknown alternate causes"?
