A F-test regression test, when and how?

[RobosaurusRex]

I am aware that f-tests can be used to check the null hypothesis when comparing regression models if the models are nested.

What I am confused about is if I can apply an f-test to compare the following, (and if so what is the best way)

I have two regression laws
Y = a1*X1 + a2*X2 + b
Y = a3*X1 + a4*X2 + b

Is the best way to test these quantitatively to compare each in turn against the reduced model Y = b
or can I compare them against one another directly by using the f-test?

 

[Dale]

Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior

 

[RobosaurusRex]

Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior

I have been trying to follow the analysis of a paper where they claim that f-tests show model a is significantly better than model b.
Any ideas?

If not, can you link me to something regarding this bayesian analysis?

 

[Dale]

Well, comparing the two models you described seems strange since they are the same form. I don't know why you would have two models of the same form with different coefficients from the same data set.

However, there certainly is plenty of information about Bayesian statistics

I would start here for a basic intro
https://en.m.wikipedia.org/wiki/Bayesian_probability
https://en.m.wikipedia.org/wiki/Bayesian_inference

And here for a basic practical method
http://www.indiana.edu/~kruschke/BEST/BEST.pdf

 

[WWGD]

I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?

 

[RobosaurusRex]

I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?

Hi sorry for the lack on context, this may help.

So i have a bunch of model output which essentially boils down to a plot of X against Y, where X can either contain one or two variables

There are two derived 'scalings' based on the governing physics nd I want to test if the two are quantitatively different when describing the data.
So I want to be quite rigorous, I have done some 'relative misfit' calculations and now I am trying to do an f-test as it was performed in the literature but obviously they dedicate a sentence to the result and nothing about how it is done