Uncertainty on best fit gradient

Click For Summary

Discussion Overview

The discussion revolves around the calculation of the best fit gradient for a set of data points with associated error bars. Participants explore the implications of not considering these error bars in the standard error estimation provided by the software used for the analysis.

Discussion Character

  • Exploratory, Technical explanation, Debate/contested

Main Points Raised

  • One participant questions whether the software's method of calculating the best fit gradient adequately accounts for the error bars on the data points.
  • Another participant suggests that the software likely uses statistics from the data points to estimate uncertainty, which may not reflect the true error if the measurements are inaccurate.
  • A participant expresses a desire for a method that incorporates error bars into the gradient uncertainty estimation, suggesting that this would yield a better statistical representation.
  • There is a mention of the assumption behind least-squares fitting, which relies on the data coming from a normal distribution, affecting how uncertainty is estimated.
  • References to external resources are provided for further exploration of the topic, including a link to a discussion on estimating error on slope in linear regression.

Areas of Agreement / Disagreement

Participants express uncertainty about the adequacy of the software's approach to error estimation, indicating a lack of consensus on whether the current method is sufficient or if incorporating error bars would improve the results.

Contextual Notes

There is an assumption that the least-squares method is based on a normal distribution of data, which may not hold true in all cases, potentially affecting the validity of the uncertainty estimates.

kop442000
Messages
32
Reaction score
0
Hi everyone,

I have a plot of some data points that have error bars on the y axis.

A bit of software I am using gives me the best fit gradient and a "Standard Error", but it doesn't take the size of error bars into consideration. I'm assuming that it just looks at how well the gradient fits the data points to give the standard error on the gradient.

But doesn't one need to take into consideration the size of the error bars on the data points??

Thank you.
 
Physics news on Phys.org
It does help to know how your software arrives at it's conclusions.

Perhaps the program just uses the statistics from the data points to figure an uncertainty in it's estimations. You can imagine if you had inaccurate measurements which just happened to fall close to a line, then the program would provide a very small error. You can test by giving it made-up data.

http://www.learningmeasure.com/articles/mathematics/LeastSquaresLineFits.shtml

Example including error-bars, using mathematica:
http://mathematica.stackexchange.co...egression-given-data-with-associated-uncertai
 
Last edited:
Thanks Simon.

Yes it seems that the software (within Python) just does a least squares fit, compares the data points to the best fit gradient and let's you know how well they fit. I wanted something that took into account the error bars on the data too - for the reason that you mentioned above.

Statistically speaking, a gradient uncertainty that took into account the error bars on the data points would be better right?

I'll check out both the links - thank you!
 
Last edited:
Statistically speaking, a gradient uncertainty that took into account the error bars on the data points would be better right?
I'm not actually sure ... the assumption behind the least-squares is that the data comes from a normal distribution so the uncertainty estimate will be based on a distribution of means.

Sze Meng Tan's Inverse Problems lecture notes provide in in-depth into regression in one of the chapters... the notes are available here:
http://home.comcast.net/~szemengtan/
 
Last edited by a moderator:
Thank you for your help!
 

Similar threads

Undergrad Error bars in Excel (standard deviation, standard error, percentage)
  • · Replies 4 ·
Replies
4
Views
3K
Undergrad Take errors into account for a data fit
  • · Replies 3 ·
Replies
3
Views
2K
Undergrad Systematic uncertainties in linear fit
  • · Replies 28 ·
Replies
28
Views
3K
Undergrad How Should You Approach Fitting Multiple Gaussians to Data with Poisson Errors?
  • · Replies 28 ·
Replies
28
Views
4K
Undergrad Error on non-linearity in a linear fit
  • · Replies 8 ·
Replies
8
Views
2K
Undergrad Data Plotting Help: Calculating Error Bars for Gradients and Average Gradient
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Uncertainties from linear interpolation
  • · Replies 30 ·
2
Replies
30
Views
4K
Undergrad Right way to account for non-statistical uncertainties
  • · Replies 16 ·
Replies
16
Views
2K
Graduate Linear combination of data with uncertainty
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad Linear Regression, Population, Sample
  • · Replies 4 ·
Replies
4
Views
2K