Solving Uncertainty in Data Analysis with Spectrophotometry

  • Thread starter Thread starter LCSphysicist
  • Start date Start date
  • Tags Tags
    Data Uncertainty
Click For Summary
SUMMARY

This discussion focuses on estimating uncertainty in data analysis using spectrophotometry, specifically when the uncertainty is not provided. The user has approximately 2000 data points (x, y) and seeks to fit a curve using Ordinary Least Squares (OLS) but lacks uncertainty information. The user proposes adjusting the uncertainty to achieve a chi-squared to degrees of freedom ratio close to one, questioning the validity of this approach. The conversation highlights the need for statistical methods to estimate uncertainty when initial data lacks this information.

PREREQUISITES
  • Understanding of Ordinary Least Squares (OLS) regression
  • Familiarity with chi-squared statistics and degrees of freedom
  • Knowledge of spectrophotometry data collection methods
  • Basic statistical concepts related to uncertainty estimation
NEXT STEPS
  • Research techniques for estimating uncertainty in datasets without provided uncertainty values
  • Learn about the chi-squared goodness-of-fit test and its applications
  • Explore methods for fitting curves to data with known functional forms
  • Investigate statistical software options for data analysis, such as R or Python libraries
USEFUL FOR

Data analysts, researchers in the field of spectrophotometry, and anyone involved in statistical modeling and uncertainty estimation in experimental data.

LCSphysicist
Messages
644
Reaction score
162
Homework Statement
.
Relevant Equations
.
So i have a folder with a lot of data/information. Basically what i have is approximatelly 2k 2upla of x and y, because i need to find the function that describe the behavior of these data. Of course, i can use a program/software to fix/adjust the curve using the concept of OLS... BTW.

The problem is that i don't have the uncertainty! Basically, this data was achieved using a spectrophotometry, but it is not given at the relatory which model and other things.

So, i was asking myself, instead of guess the uncertainty, is it allowed to change it seeking for the best agreement between the value of chi2 and NGL? I mean, since i don't know the uncertainty, i have the free to guess it using the chi²/ngl \approx 1 concept?

If not, is there a way to, at least statiscally, guess the uncertainty?
 
Physics news on Phys.org
Are you saying you are looking for a continuous function to fit the data without any theoretical basis for what the function should look like? There are techniques for that. They work by having to justify each each addition of an arbitrary parameter by beating a threshold for the improvement in the fit.

Or do you already know the form of the function and are just trying to estimate its parameters?
 
haruspex said:
Are you saying you are looking for a continuous function to fit the data without any theoretical basis for what the function should look like? There are techniques for that. They work by having to justify each each addition of an arbitrary parameter by beating a threshold for the improvement in the fit.

Or do you already know the form of the function and are just trying to estimate its parameters?
I already have the function, already have the data. The problem is the uncertainty. The datas given does not provide any information about it.

The software that fix the points will give me the numerical valor of the parameters of the function, that i have. BUT, i will need to compare these parameters with real values, and see if they are compatible.

The Problem IS, the uncertainty provided by the software for each parameter is non sense because i don't even have the uncertainty for the datas, x and y, initially given to me.

SO, in order to make a good comparation using test T or Z, i need at first be able to estimate the uncertainty of the points.

Now, since i have no information about it, i am thinking if there is a statiscally way to "estimate" the uncertainty for the datas.

The conclusion i got was to guess it until the ratio chi to Degree of Freedom be approximatelly one.

The other problem is, we use the chi to degree ratio as a way to see if the adjust is good, and what i am doing is like "to force" the adjust to be good.

So, i would like to know if there is another way to estimate uncertainty of a lot of numbers.

OR, if there isn't, i would like to know if what i did with chi to df is "allowed"?
 
Not sure it's valid, but if you suppose the data points are normally distributed about the curve, all with the same variance ##\sigma^2##, then the likelihood of the data is maximised by ##\sigma^2=\frac 1n\Sigma_{i=1}^n(y_i-y(x_i))^2##.
 

Similar threads

  • · Replies 8 ·
Replies
8
Views
4K
Replies
7
Views
2K
Replies
1
Views
1K
Replies
1
Views
3K
Replies
3
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
Replies
24
Views
3K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 3 ·
Replies
3
Views
3K