# Comparing theory to data

## Main Question or Discussion Point

I need to quantify the statistical significance of how much a model deviates from a given set of data points, and I cannot do a fit.

Let's say the model is a one-parameter description of some time dependent quantity f_a(t). I have data points at n different times including error bars, so pi = {ti, fa(ti), sigmai}. The reason I cannot do a fit is that the data is actually the predicted errors for some fiducial value a_0, and the fit would obviously just find a = a0.

I want to know how far off a couple models with either a = a1 or a = a2 are. So in other words, how well does fa1(t) fit the data points pi? Can I rule it out on some confidence level?

It seems difficult to express my problem, I hope it became clear enough. All I am asking for is a hint in the right direction, so if you know some relevant reference, I can just get it in the library. I already calculated the likelihood L(a) but don't really know how to proceed.

Thanks.

Related Set Theory, Logic, Probability, Statistics News on Phys.org
EnumaElish
Homework Helper
If you have the predicted (projected) value and the error term for each t, why can't you calculate ("back out") the actual observed value corresponding to that t?

I don't understand. How am I supposed to do that? The actual observed value won't be available until several years from now. And "observed value" implies that it is observed, I don't see how I can calculate it.

EnumaElish
Homework Helper
So, the error bars you mentioned are forecast (future) errors? That wasn't clear to me.

Can you post your understanding of what these errors are, or what they represent?

Last edited:
Yes, the error bars are predicted. I am basically using the specifications of the experiment and the Fisher matrix formalism to assess the quality of the constraints that will be put on fiducial values of a set of parameters.

EnumaElish
Homework Helper
Let me make up an example: you are predicting the value of random variable Y for, say February 2012. You have a point estimate, $$\hat Y(Feb. 2012)$$, and an error "bar" "wrapped around" the point estimate. Have I got it?

Let me make up an example: you are predicting the value of random variable Y for, say February 2012. You have a point estimate, $$\hat Y(Feb. 2012)$$, and an error "bar" "wrapped around" the point estimate. Have I got it?
Not quite. We already kind of know the value of Y(Feb 2012). For the sake of clarity let's choose a date in the ast, say the average temperature at some place in the year 1500. Also I will adopt my notation from the original post.
We already kind of know how the temperature behaves over time, depending on a set of parameters, one of them being a=a0. According to earlier measurements, we expect something like f(1500)=20C, f(1700)=22C and f(1900)=25C.
But we want to constrain it as much as possible and plan an improved experiment. According to my calculations, we are expecting to get error bars of .1C, 0.15C and 0.2C on each of these values after that new experiment. Now we want to see if we are going to be able to test a competing theory (ie. the parameter a has a different value). I want to quantify how much the curve of the new theory with a=a1 would deviate from the data with the predicted error bars.

EnumaElish