Total significance Higgs discovery at 7+8TeV LHC

[Sleuth]

HI guys,

a quick question. After the announcement of the discovery of the higgs-like resonance in July 2012 with ~5sigma significance by both ATLAS and CMS, what is the current p-value distribution with the full data set taken into account? And therefore, what is the total significance reached with the full 7+8 TeV LHC data set? Does anyone know if the data has been completely analysed already?

thanks

 

[Vanadium 50]

The significance is so high - 13 sigma, 15 sigma - that it doesn't make sense to talk about it. The probability of a statistical fluctuation is so small, it is more likely that a cosmic ray passed through the CPU at the time you were doing the calculation than an actual fluctuation.

 

[Sleuth]

Thanks Vanadium. Could you point me to the CMS/ATLAS papers with the last updates on the significance?
I had and have no doubt about the smallness of the probability of a statistical fluctuation, and that was not the reason why I asked.
I still think it would be interesting to know how the distribution evolved, while after the discovery it was not so easy to find updates of the significance (at least for a non-expert like me). Increasing the statistics the significance should of course also increase, and I would simply be curious to see what the "Higgs-peak" looks like with the full data set.

Also, the fist 13TeV data showed actually quite a downward fluctuation in the region of 125GeV, compared to what was expected from the Higgs signal there... Not that I believe the Higgs could disappear at 13 TeV, but I guess keeping in mind how the peak evolves and "re-appears" at 13 TeV is also interesting (or maybe it's just me :D).

Thanks again and merry christmas!

 

[mfb]

while after the discovery it was not so easy to find updates of the significance (at least for a non-expert like me).

There were a few in 2012, but physicists quickly lost interest in combining significances - if you have 4 independent analyses with 5+ sigma each (and some with 3+), why bother combining a significance, if you can spend time on combining the central values and uncertainties?
This talk is relevant, for a total combined Higgs signal strength the experimental uncertainty is about 0.08 (slide 39) which would indicate ~13 sigma significance if the profile would be Gaussian. It is not, I guess the 13 sigma are an underestimate (4 leptons is great in producing significance due to its low background, but has large uncertainties in coupling strength).

You can also check slide 41: even the less frequent production or decay modes are in the 3 to 5 sigma range. Note that those numbers are correlated.

 

[Sleuth]

Thanks a lot mfb! That's precisely what I was looking for!

 

[Vanadium 50]

The problem with statistics out there is that small differences in technique make huge differences in significances. You can take μ/σ from Wolter's talk, and that gives you a p-value of 10^-38. Or you can combine the individual values on page 41 and you get χ² of 140 for 14 degrees of freedom. Calling this 10σ and you get a p-value of 10^-23. If you actually evaluate the χ², it's p-value is closer to 10^-22, or 9.8σ.

The point is that these probabilities vary by a factor of 100 billion. Classical statistics doesn't deal well with p-values this small. Tiny changes in how things are calculated make huge differences in significances. It doesn't make any sense to talk about the difference between about 7 or 8σ and anything higher.

 

[Vanadium 50]

Also, the fist 13TeV data showed actually quite a downward fluctuation in the region of 125GeV, compared to what was expected from the Higgs signal there...

That's not really true. CMS didn't look because the expected signal is so weak with that amount of data. ATLAS looked, and proved CMS right. Yes, there is a downward fluctuation relative to what you expect, but not a significant downward fluctuation.

 

[mfb]

@V50: ggf is missing in slide 41, it is not surprising that you get a lower result.
Also keep in mind that VH is WH+ZH.
How did you get 14 degrees of freedom by combining 4 numbers?

 

[ChrisVer]

how many are the dofs?

 

[Vanadium 50]

How did you get 14 degrees of freedom by combining 4 numbers?

Fisher's rule. If you have n variables with given p-values, the variable

$$X = -2 \sum_{i=1}^{n} \ln(p_i)$$

is distributed as a χ² with 2n degrees of freedom.

But this doesn't change my point - the p-values are so small, it's virtually impossible to accurately tell how small they are. No rational person would argue the Higgs signal is a statistical fluctuation. It could be some other kind of mistake, but not this kind.

 

[mfb]

I would expect 2*4=8 degrees of freedom.

But this doesn't change my point - the p-values are so small, it's virtually impossible to accurately tell how small they are. No rational person would argue the Higgs signal is a statistical fluctuation. It could be some other kind of mistake, but not this kind.

Yes, of course.
And it is really hard to imagine a mistake that pops up in several different production and decay modes in the same way, so a real particle (or particle-like object) is the only reasonable explanation.

 

[Vanadium 50]

I would expect 2*4=8 degrees of freedom.

I counted every row in the table, so I double counted. But the equations are all in the thread now, so anyone can redo it if they want. I don't think it will make a huge difference - tossing out two 2.something sigma points at this significance will leave the qualitative result intact.

I had a particle discovery once where the significance was anywhere between 7 and 14 standard deviations, depending on the assumptions of the calculation. That's 32 orders of magnitude in p-value. We tried to get the journal to write "greater than six standard deviations", spelling out "six" to make it clear that it was so significant the exact number didn't matter. The journal style guide wouldn't allow it.

 

[ChrisVer]

The journal style guide wouldn't allow it.

For what reason if I may ask? did they want the exact answer or the other way around (6 was already too high to mention)?

 

[Vanadium 50]

The journal style guide is what it is.