Effect of errors due to branching and acceptances

[CAF123]

Say I have say 10 measurements of an experimental observable e.g differential cross section with respect to rapidity , one measurement in each bin and corresponding statistical and systematic uncertainties given in % per bin.

Now, suppose I want the value of this dsigma/dy after correcting for e.g acceptance and certain branching fraction.

How are the statistical and systematic errors changed? The acceptance and branching typically come with an error so in what way will they affect the statistical and systematic errors?

 

[mfb]

Standard error propagation for the multiplication of two numbers.
It gets more interesting if you have significant migration between the bins or other sources of correlations.

 

[CAF123]

Standard error propagation for the multiplication of two numbers.

But does the acceptance and branching enter into the statistical or the systematic uncertainty? I think the acceptance is dependent on the rapidity so cannot be systematic (therefore statistical) and the branching would enter in the systematics?

 

[mfb]

I think the acceptance is dependent on the rapidity so cannot be systematic (therefore statistical)

What exactly is a statistic uncertainty about that? If you repeat the measurement, do you expect a different result?

In general both will come with a statistic and a systematic uncertainty. Treat them separately. Combine all sources of statistical uncertainty, combine all sources of systematic uncertainty.

 

[CAF123]

In general both will come with a statistic and a systematic uncertainty. Treat them separately. Combine all sources of statistical uncertainty, combine all sources of systematic uncertainty.

Thanks, in the article I am studying the acceptance and branching are given each in the form $A \pm \delta A$ only, so how should I interpret the $\delta A$? Added in quadrature of the statistical and systematic errors? If that might be the case, then it seems without further information on the decomposition of this error it is impossible for me to split into statistical and systematic error contributions?

 

[mfb]

I don't know, I would have to see the article. If it is both combined then you can't split it of course.
If you have external sources of uncertainty it is common to treat them as separate class. Something like that: $52 \pm 2 \text{(stat)} \pm 3 \text{(syst)} \pm 1 \text{(BF)}$.

 

[CAF123]

Indeed, if I had the errors for the branching and acceptance spelled out as $A \pm \delta A_{\text{stat}} \pm \delta A_{\text{sys}}$ for acceptance and $B\pm \delta B_{\text{stat}} \pm \delta B_{\text{sys}}$ for the branching then assuming independent sources of statistical and systematic errors, my net statistical on d(sigma)/dy would be $$\delta _{\text{stat}} = \sqrt{\delta A_{\text{stat}}^2 + \delta B_{\text{stat}}^2 + \dots}$$ and similarly for the systematic. I think that should be correct.

It's common to see experimental formulae for differential cross sections expressed in terms of parameters that are estimated by experimentalists, e.g the acceptance appears in the denominator. Could you also get the effect of the error on the differential cross section due to the acceptance through partial derivatives? And this would coincide with simply adding in quadrature?

 

[mfb]

I guess so - it works if I understand your description correctly.

 

[CAF123]

Can we do this explicitly? Given d(sigma)/dy_i = f(A_i,...), where A_i is the acceptance in bin i and dots denote other experimental quantities such as track efficiencies, purities etc..., we have in the notation used in previous posts $$\delta_{\text{stat}}^2 \left( \frac{d \sigma}{d y_i} \right) = \left( \frac{\delta f}{\delta A_i} \delta A_{i,\text{stat}} \right)^2 + \dots$$ so that $$\delta_{\text{stat}} \left( \frac{d \sigma}{d y_i} \right) = \sqrt{\left( \frac{\delta f}{\delta A_i} \delta A_{i,\text{stat}} \right)^2 + \dots}$$ But comparing this to the centered equation in #7, $$\delta_{\text{stat}} = \sqrt{\delta A_{\text{stat}}^2 + \dots}$$ there is an extra multiplicative factor $\left( \frac{\delta f}{\delta A}\right)^2$ in the case where I used partial derivatives. What's the reconciliation?

 

[mfb]

Ah, I thought you were talking about relative uncertainties in post 7. If not then you need the additional factor of course. Otherwise not even the units work out.

 

[CAF123]

Ah, I thought you were talking about relative uncertainties in post 7. If not then you need the additional factor of course. Otherwise not even the units work out.

Indeed, so do you mean to say it only makes sense to add percentage uncertainties in quadrature and if one is given absolute uncertainties then the formula in #9 (2nd equation centered) should be used?

Also, was wondering, it is common to add in quadrature errors due to independent systematic sources. What is the theoretical justification for that? Statistics have the Gaussian description for large number of measurements but the systematics are not described as such.

 

[mfb]

Indeed, so do you mean to say it only makes sense to add percentage uncertainties in quadrature and if one is given absolute uncertainties then the formula in #9 (2nd equation centered) should be used?

That, or convert it to relative uncertainties first.

Also, was wondering, it is common to add in quadrature errors due to independent systematic sources. What is the theoretical justification for that? Statistics have the Gaussian description for large number of measurements but the systematics are not described as such.

The variance still behaves just like it does for Gaussian errors. If a source of systematic uncertainty is not even close to a normal distribution (e.g. it is an interval with a minimum and maximum and everything in between is equally likely) and relevant it might make sense to treat it differently.

 

[dukwon]

Indeed, if I had the errors for the branching and acceptance spelled out as $A \pm \delta A_{\text{stat}} \pm \delta A_{\text{sys}}$ for acceptance and $B\pm \delta B_{\text{stat}} \pm \delta B_{\text{sys}}$ for the branching then assuming independent sources of statistical and systematic errors, my net statistical on d(sigma)/dy would be $$\delta _{\text{stat}} = \sqrt{\delta A_{\text{stat}}^2 + \delta B_{\text{stat}}^2 + \dots}$$ and similarly for the systematic. I think that should be correct.

The statistical uncertainty on your result should only depend on the size of the data sample with which you made the measurement. If you get acceptance from MC and a branching fraction from some other analysis then their uncertainties contribute only to the systematic uncertainties. If you can easily factorise external branching fractions from your result, then it is common to quote its uncertainty separately, so that your result can be improved if better external measurements are made.

 

[CAF123]

The statistical uncertainty on your result should only depend on the size of the data sample with which you made the measurement. If you get acceptance from MC and a branching fraction from some other analysis then their uncertainties contribute only to the systematic uncertainties. If you can easily factorise external branching fractions from your result, then it is common to quote its uncertainty separately, so that your result can be improved if better external measurements are made.

I see. So in what sense generally will the acceptance and the branching come with a statistical uncertainty and a systematic uncertainty? (as mfb mentioned in #4) That is, what experimental source would give rise to a statistical uncertainty upon the acceptance and branching?

 

[dukwon]

That is, what experimental source would give rise to a statistical uncertainty upon the acceptance and branching?

Say your acceptance is calculated from MC; the statistical uncertainty is just due to the sample size.

If you take branching fractions (please don't just say "branching", it's confusing) from an external measurement, then the statistical uncertainty is due to the size of the dataset used in that analysis.

However, the effect of these uncertainties on your measurement remains systematic: they don't depend on the size of the dataset you make your measurement with.

 

[CAF123]

Thanks. What you say makes sense for me regarding the branching fraction as this is a value found in e.g PDG and found separate from measurement under study.

But the acceptance is determined from the process at hand, so if I am studying a decay $A \rightarrow BC$, say, the amount of events selected as a signal enter into the acceptance and this is based on the actual measurement under study, not some prior measurement so would qualify as largely statistical. Or have I misunderstood here? Thanks

 

[dukwon]

I've never seen acceptance be taken from data (how do you count particles that you don't detect?) but if that's what you do, then yeah its statistical uncertainty should enter into the statistical uncertainty of the result.

 

[CAF123]

Ah sorry probably then I’m just misunderstanding how experimentalists determine the acceptance. Is it usually always done through simulations in Monte Carlo event generators and taking the figure from there?

 

[mfb]

It is often a mixture of both. You'll never get a number that is completely free of Monte Carlo input but you also don't want to rely exclusively on a good MC description. What is done depends on the individual analysis. Often you can find related datasets that share some features with the main signal sample, but where you can study some of its properties better.

 

[CAF123]

Thank you. Is there also an uncertainty which accounts for the robustness/reliability of the underlying MC method used in determining acceptances? I would say something like a 'model dependent uncertainty' but not sure if that's the correct terminology here.

 

[mfb]

You'll often see the results checked with a different MC generator, with a different detector description in the MC generator or similar things.
Have a look at some publications to see how they treat these things.

 

[CAF123]

@dukwon

Say your acceptance is calculated from MC; the statistical uncertainty is just due to the sample size.

If you take branching fractions (please don't just say "branching", it's confusing) from an external measurement, then the statistical uncertainty is due to the size of the dataset used in that analysis.

However, the effect of these uncertainties on your measurement remains systematic: they don't depend on the size of the dataset you make your measurement with.

So, just to check I understood what you said, If I consider a decay $A \rightarrow B + C$ then in its MC implementation, all measurements I make associated with the generated sample size are statistical but effect the actual measurement at the experiment systematically? (i.e we use an external MC to ascertain a quantity (e.g acceptance) needed in the experimental cross section)