Comparing Best-fits of different data sets that have different noise levels

Click For Summary
SUMMARY

This discussion focuses on comparing model fits for two data sets, day1 and day2, using the non-linear least squares method. The challenge arises from day1 containing significantly noisier data than day2, leading to an inherently larger RMS for day1. To achieve a fair comparison independent of noise levels, the discussion suggests employing a probability model that accounts for noise distribution. Additionally, utilizing the likelihood ratio test is recommended for a more robust comparison of the model fits.

PREREQUISITES
  • Understanding of non-linear least squares fitting
  • Familiarity with RMS (Root Mean Square) error measurement
  • Knowledge of probability models and noise characterization
  • Experience with likelihood ratio tests in statistical analysis
NEXT STEPS
  • Research methods for noise characterization in data sets
  • Learn about the implementation of likelihood ratio tests in model comparison
  • Explore advanced non-linear least squares fitting techniques
  • Investigate statistical methods for assessing model fit quality
USEFUL FOR

Data scientists, statisticians, and researchers involved in model fitting and analysis of noisy data sets will benefit from this discussion.

zachzach
Messages
257
Reaction score
1
Suppose I have 2 sets of data: day1 and day2. I want to fit a model to both data sets and then compare them to each other to see which one fits the model the best (the fit is done with a computer using non-linear least squares method). The RMS of the fit would be fine except that day1 has much noisier data than day2 and the noise level is unknown. This makes the RMS of the fit for day1 (the noisy data) intrinsically larger than the RMS for day2 simply because of the noise. Is there anyway to compare the fits that is independent of the noise level?
 
Physics news on Phys.org
I don't see how you can make a comparision unless you are willing to create a probability model for the data that includes the distribution of the noise. If you are willing to characterize the noise then you could consider a computation based on the "liklihood ratio test".
 

Similar threads

Undergrad Best way to quantify how well a model fits data
  • · Replies 20 ·
Replies
20
Views
3K
Graduate How to perform a fit with correlated variables
  • · Replies 16 ·
Replies
16
Views
3K
Undergrad Linear fit on the difference of data points
  • · Replies 26 ·
Replies
26
Views
3K
Undergrad Weighted average of a set of slopes with different goodness of fit
  • · Replies 7 ·
Replies
7
Views
8K
Graduate Comparing discrete data to a continuous model (1D)
  • · Replies 1 ·
Replies
1
Views
2K
Graduate The Blessings of Dimensionality and Hilbert's 6th Problem
  • · Replies 4 ·
Replies
4
Views
3K
Undergrad Determining the best fit regression for a set of data
  • · Replies 6 ·
Replies
6
Views
9K
Graduate What is the reasoning for this prediction?
  • · Replies 4 ·
Replies
4
Views
1K
Graduate AICc and parameter likelihood from repeated fits
  • · Replies 9 ·
Replies
9
Views
3K
Graduate Calculating errors and trendline - different correlation coefficients?
  • · Replies 7 ·
Replies
7
Views
3K