Comparing model fits to noisy datasets with unknown noise levels

[zachzach]

Suppose I have 2 sets of data: day1 and day2. I want to fit a model to both data sets and then compare them to each other to see which one fits the model the best (the fit is done with a computer using non-linear least squares method). The RMS of the fit would be fine except that day1 has much noisier data than day2 and the noise level is unknown. This makes the RMS of the fit for day1 (the noisy data) intrinsically larger than the RMS for day2 simply because of the noise. Is there anyway to compare the fits that is independent of the noise level?

 

[Stephen Tashi]

I don't see how you can make a comparision unless you are willing to create a probability model for the data that includes the distribution of the noise. If you are willing to characterize the noise then you could consider a computation based on the "liklihood ratio test".