# A Signal signficancies low expected events

1. Dec 24, 2016

### ChrisVer

Hi, I am a little "confused" of how to treat a problem stating: suppose you expect 1.25 background (SM) events from one measurement. During one particular measurement, you observe 5 events. What's the probability that you discovered new physics?
The standard/basic way to go about it is to calculate the significance of your "hypothetized new signal", that is $s = \frac{S}{\sqrt{B}}= \frac{5-1.25}{\sqrt{1.25}}=3.35$, which tells us that the new signal should be 3.35 sigma (standard deviations) away from the expected value to explain the observed events. In probabilities this translates to 99.958% signal exists/ 0.042% it's a statistical fluctuation, right?

My question is that the probabilities in those cases are taken from the standard normal distribution. Wouldn't (for so small expected events) the random variable (observed events) be taken out a Poisson distribution? In explicit wouldn't I have to calculate what's the probability of : $P( x \ge 5 | \lambda =1.25 )\approx 0.9\%$ (being statistical fluctuation) and the complementary for a signal? This would be relatively bad for the significance, as the standard deviation of a Poisson distributed variable is not equal to the square root of the events (for low expected events)...

Thanks.
Maybe this deserves to be moved in the statistics, but please do whatever you find fit.

Last edited: Dec 24, 2016
2. Dec 24, 2016

### Orodruin

Staff Emeritus
Ooops! Be careful here! Are you using frequentist or Bayesian statistics? In Bayesian statistics it depends on your prior. Frequentist statistics does not answer this question. It answers the question "what is the probability this (or something more extreme) would happen by chance?" which is not the same question.

And yes, for a low number of events you need to use the Poisson distribution.

3. Dec 24, 2016

### ChrisVer

I don't think I can use a prior for this? Except for saying that I expect (given the model) $B+ \mu S$ where $\mu$ is taken from a uniform prior, and calculating stuff like the 95-quantile for mu to claim a discovery or not (via credibility levels)... seeing eg how the likelihood works:
$L( x_{obs} | B + \mu S ) = Poi(x_{obs} | B+\mu S) Uni(\mu)$
again I don't think it's easy to determine by simple calculations the ranges for mu or the results of its quantiles (would have to input those in a statistical program).

I think this is done easier via calculations? I.e. it results in the way I calculated the Poisson probability (and also how partially I interpreted the result of it, being a stat fluctuation/happened by chance). Maybe I was wrong to move one step further and say that the complementary (99.1%) is the probability of having seen a signal.

yup, so the significance (as I defined it) for such cases wouldn't be a good indicator?

4. Dec 24, 2016

### Orodruin

Staff Emeritus
5. Dec 24, 2016

### Staff: Mentor

Correct. Stay away from such an interpretation, there is no meaningful way to define "the probability that you saw a signal". "The probability of a background fluctuation that large" is the only meaningful number.