Monte Carlo for uncertainty estimation

Click For Summary
SUMMARY

This discussion focuses on using Monte Carlo simulations for uncertainty estimation in the extraction of the parameter W from experimental measurements of A, defined by the equation A = (0.2 * (W/Δ)) / ((W/Δ)^2 + 0.1^2). The user conducted simulations with Δ values of 2π * [-500, -250, -200, -100, -50, 50, 100, 200, 250, 500] and reported results for A and its uncertainty dA. Two methods for estimating W and its uncertainty were proposed: a least squares fit of W vs A and sampling A multiple times to compute W. The discussion highlights the challenges of measuring A accurately and the implications of measurement uncertainty on the derived value of W.

PREREQUISITES
  • Understanding of Monte Carlo simulations for uncertainty estimation
  • Familiarity with least squares fitting techniques
  • Knowledge of statistical analysis, including mean and standard deviation calculations
  • Basic grasp of experimental physics and measurement uncertainty
NEXT STEPS
  • Explore advanced Monte Carlo simulation techniques for uncertainty analysis
  • Study least squares fitting methods in Python using libraries like NumPy or SciPy
  • Learn about measurement uncertainty and its impact on experimental results
  • Investigate statistical methods for analyzing small sample sizes in experimental data
USEFUL FOR

Researchers and experimental physicists involved in data analysis, particularly those working with Monte Carlo methods and uncertainty quantification in measurements.

kelly0303
Messages
573
Reaction score
33
Hello! This is tangentially also a follow up to this post. I have the following equation:

$$A = \frac{0.2\frac{W}{\Delta}}{\left(\frac{W}{\Delta}\right)^2+0.1^2}$$
where ##\Delta## is an experimental parameter, ##A## is obtained by some measurements and it depends on ##\Delta## and the statistics of the experiment, while ##W## is the parameter I want to extract from the experiment, which in the simulations described here was set to ##4\pi##. I have some values for ##\Delta##, which are: ##2\pi\times [-500,-250,-200,-100,-50,50,100,200,250,500]##. For each ##\Delta## I ran some Monte Carlo (MC) simulations to extract A and the associated uncertainty and I obtained ##A = [-0.07471803, -0.15907364, -0.20514187, -0.39216751, -0.696679, 0.70398886, 0.38746261, 0.20232256, 0.15935686, 0.0736096]## and ##dA =[0.10973486, 0.1076796, 0.10531444, 0.10150821, 0.07678416, 0.07809082, 0.10294303, 0.10685488, 0.10791492, 0.10993011]##. If I increase the statistics by a factor of 10, I get ##A =[-0.07914394, -0.1585819, -0.19860262, -0.38868242, -0.70347071, 0.70340396, 0.38731616, 0.19894059, 0.15979929, 0.07932907]## and ##dA =[0.03594135, 0.03645251, 0.03466366, 0.03255766, 0.02302652, 0.022873, 0.03185962, 0.03428031, 0.03592418, 0.03339634]## (I just dropped all the decimal places printed by Python, sorry about that), so almost the same values for A, but a factor of ##\sqrt{10}## lower uncertainty, as expected. I am not sure how to proceed from here in extracting W and its associated uncertainty. One way is to use the above equation and write W in terms of A and ##\Delta## (only one solution is physical), for each ##\Delta## sample A from the associated mean and standard deviation given above, then just perform a least square fit of W vs A. If I do that I am getting an error on W of ~##0.4\pi-0.5\pi## (I am usually dividing everything by ##2\pi## in my calculations and just multiplying it back here). For the higher statistic case, the uncertainty is ~##0.04\pi-0.05\pi## (for the second case, the central W value is actually not consistent with ##4\pi## given the uncertainty, at 1 ##\sigma## level). Another way to estimate the uncertainty on W is by sampling A for each delta a large number of time (say 1000), compute W for each one, and use the mean and standard deviation of the obtained W values. In this case I am getting an uncertainty of ~##3\pi## and ~##1\pi## for the low and high statistics case. Given the large values of uncertainty now I am consistent in both cases with the real W value, but the uncertainties seem too large. Can someone help me figure out which one is the right way and why the other one is wrong?

Also, in practice, in my experiment I will just have 10 points, corresponding to the 10 values of ##\Delta## and the associated W values (and it will take about a week to measure them). In that case I won't be able to sample A values a large number of time, so I would need to just use these 10 points to extract W. How would I proceed then (obviously in that case I don't know W, either)? Thank you and sorry for the long post!
 
Physics news on Phys.org
It sounds like this is for an actual physical process. You should consider uncertainty in your measurement of A. Depending on which section of the curve is the "physically impossible" then small changes in A measurement could have a big effect on W.

Even if you are in the region where small change in A does not move W very much, there is still uncertainty. So you need to consider how much could A be off when you measure it? How is it measured - by looking at a ruler or some gage? Is the same person performing the measurement each time? Just some thoughts.
 
scottdave said:
It sounds like this is for an actual physical process. You should consider uncertainty in your measurement of A. Depending on which section of the curve is the "physically impossible" then small changes in A measurement could have a big effect on W.

Even if you are in the region where small change in A does not move W very much, there is still uncertainty. So you need to consider how much could A be off when you measure it? How is it measured - by looking at a ruler or some gage? Is the same person performing the measurement each time? Just some thoughts.
The values I provided are generated numerically not from the actual experiment. So I assume that all sources of uncertainty are accounted for (as they are used in the MC process).
 

Similar threads

  • · Replies 12 ·
Replies
12
Views
3K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 18 ·
Replies
18
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 16 ·
Replies
16
Views
2K
  • · Replies 21 ·
Replies
21
Views
3K
  • · Replies 22 ·
Replies
22
Views
2K
  • · Replies 9 ·
Replies
9
Views
2K