I Monte Carlo for uncertainty estimation

[kelly0303]

Hello! This is tangentially also a follow up to this post. I have the following equation:

$$A = \frac{0.2\frac{W}{\Delta}}{\left(\frac{W}{\Delta}\right)^2+0.1^2}$$
where $\Delta$ is an experimental parameter, $A$ is obtained by some measurements and it depends on $\Delta$ and the statistics of the experiment, while $W$ is the parameter I want to extract from the experiment, which in the simulations described here was set to $4\pi$. I have some values for $\Delta$, which are: $2\pi\times [-500,-250,-200,-100,-50,50,100,200,250,500]$. For each $\Delta$ I ran some Monte Carlo (MC) simulations to extract A and the associated uncertainty and I obtained $A = [-0.07471803, -0.15907364, -0.20514187, -0.39216751, -0.696679, 0.70398886, 0.38746261, 0.20232256, 0.15935686, 0.0736096]$ and $dA =[0.10973486, 0.1076796, 0.10531444, 0.10150821, 0.07678416, 0.07809082, 0.10294303, 0.10685488, 0.10791492, 0.10993011]$. If I increase the statistics by a factor of 10, I get $A =[-0.07914394, -0.1585819, -0.19860262, -0.38868242, -0.70347071, 0.70340396, 0.38731616, 0.19894059, 0.15979929, 0.07932907]$ and $dA =[0.03594135, 0.03645251, 0.03466366, 0.03255766, 0.02302652, 0.022873, 0.03185962, 0.03428031, 0.03592418, 0.03339634]$ (I just dropped all the decimal places printed by Python, sorry about that), so almost the same values for A, but a factor of $\sqrt{10}$ lower uncertainty, as expected. I am not sure how to proceed from here in extracting W and its associated uncertainty. One way is to use the above equation and write W in terms of A and $\Delta$ (only one solution is physical), for each $\Delta$ sample A from the associated mean and standard deviation given above, then just perform a least square fit of W vs A. If I do that I am getting an error on W of ~$0.4\pi-0.5\pi$ (I am usually dividing everything by $2\pi$ in my calculations and just multiplying it back here). For the higher statistic case, the uncertainty is ~$0.04\pi-0.05\pi$ (for the second case, the central W value is actually not consistent with $4\pi$ given the uncertainty, at 1 $\sigma$ level). Another way to estimate the uncertainty on W is by sampling A for each delta a large number of time (say 1000), compute W for each one, and use the mean and standard deviation of the obtained W values. In this case I am getting an uncertainty of ~$3\pi$ and ~$1\pi$ for the low and high statistics case. Given the large values of uncertainty now I am consistent in both cases with the real W value, but the uncertainties seem too large. Can someone help me figure out which one is the right way and why the other one is wrong?

Also, in practice, in my experiment I will just have 10 points, corresponding to the 10 values of $\Delta$ and the associated W values (and it will take about a week to measure them). In that case I won't be able to sample A values a large number of time, so I would need to just use these 10 points to extract W. How would I proceed then (obviously in that case I don't know W, either)? Thank you and sorry for the long post!

 

[scottdave]

It sounds like this is for an actual physical process. You should consider uncertainty in your measurement of A. Depending on which section of the curve is the "physically impossible" then small changes in A measurement could have a big effect on W.

Even if you are in the region where small change in A does not move W very much, there is still uncertainty. So you need to consider how much could A be off when you measure it? How is it measured - by looking at a ruler or some gage? Is the same person performing the measurement each time? Just some thoughts.

 

[kelly0303]

It sounds like this is for an actual physical process. You should consider uncertainty in your measurement of A. Depending on which section of the curve is the "physically impossible" then small changes in A measurement could have a big effect on W.

Even if you are in the region where small change in A does not move W very much, there is still uncertainty. So you need to consider how much could A be off when you measure it? How is it measured - by looking at a ruler or some gage? Is the same person performing the measurement each time? Just some thoughts.

The values I provided are generated numerically not from the actual experiment. So I assume that all sources of uncertainty are accounted for (as they are used in the MC process).