F-test regression test, when and how?

In summary, the conversation discusses the use of f-tests in comparing regression models. The main question is whether an f-test can be used to compare two regression models with the same form but different coefficients. One person suggests using Bayesian analysis instead, which is sensitive to prior information. The conversation also mentions the use of software to input data and generate models, and the confusion about different models being produced. The setup being used involves plotting X against Y, with two derived "scalings" based on physics, and the goal is to test if the two are quantitatively different in describing the data. The person wants to be rigorous and is currently attempting to perform an f-test, but is unsure of the correct method.
  • #1
RobosaurusRex
29
1
I am aware that f-tests can be used to check the null hypothesis when comparing regression models if the models are nested.

What I am confused about is if I can apply an f-test to compare the following, (and if so what is the best way)

I have two regression laws
Y = a1*X1 + a2*X2 + b
Y = a3*X1 + a4*X2 + b

Is the best way to test these quantitatively to compare each in turn against the reduced model Y = b
or can I compare them against one another directly by using the f-test?
 
Physics news on Phys.org
  • #2
Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior
 
  • #3
Dale said:
Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior

I have been trying to follow the analysis of a paper where they claim that f-tests show model a is significantly better than model b.
Any ideas?

If not, can you link me to something regarding this bayesian analysis?
 
  • #4
Well, comparing the two models you described seems strange since they are the same form. I don't know why you would have two models of the same form with different coefficients from the same data set.

However, there certainly is plenty of information about Bayesian statistics

I would start here for a basic intro
https://en.m.wikipedia.org/wiki/Bayesian_probability
https://en.m.wikipedia.org/wiki/Bayesian_inference

And here for a basic practical method
http://www.indiana.edu/~kruschke/BEST/BEST.pdf
 
  • #5
I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?
 
  • #6
WWGD said:
I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?
Hi sorry for the lack on context, this may help.

So i have a bunch of model output which essentially boils down to a plot of X against Y, where X can either contain one or two variables

There are two derived 'scalings' based on the governing physics nd I want to test if the two are quantitatively different when describing the data.
So I want to be quite rigorous, I have done some 'relative misfit' calculations and now I am trying to do an f-test as it was performed in the literature but obviously they dedicate a sentence to the result and nothing about how it is done
 

1. What is an F-test regression test?

An F-test regression test is a statistical test used to determine the significance of the relationship between two or more variables in a regression model. It measures the ratio of the variance explained by the model to the variance not explained by the model.

2. When is an F-test regression test used?

An F-test regression test is typically used to compare two or more regression models and determine if there is a significant difference in the explanatory power of the models. It can also be used to test the overall significance of a regression model.

3. How is an F-test regression test performed?

To perform an F-test regression test, the first step is to define the null and alternative hypotheses. The null hypothesis states that there is no significant relationship between the variables, while the alternative hypothesis states that there is a significant relationship. Next, the test statistic is calculated by dividing the mean squared error of the model by the mean squared error of the residuals. This test statistic is then compared to the critical value from an F-distribution table to determine the p-value and significance of the test.

4. What is the interpretation of the results of an F-test regression test?

If the p-value from the F-test is less than the chosen significance level (usually 0.05), then we reject the null hypothesis and conclude that there is a significant relationship between the variables. On the other hand, if the p-value is greater than the significance level, we fail to reject the null hypothesis and conclude that there is not a significant relationship between the variables.

5. Are there any limitations to using an F-test regression test?

Yes, there are some limitations to using an F-test regression test. It assumes that the data is normally distributed and that there is a linear relationship between the variables. Additionally, the test may not be appropriate for models with a large number of variables or highly correlated variables. It is important to also consider the context and purpose of the regression model when interpreting the results of an F-test.

Similar threads

I Coefficient sign flip in linear regression with correlated predictors
    • Set Theory, Logic, Probability, Statistics
Replies
13
Views
2K
I Linear regression and random variables
    • Set Theory, Logic, Probability, Statistics
Replies
30
Views
2K
I Exploring Nonlinear Least Squares for Regression Analysis
    • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
A Follow-Up on F-Test in Multi-Linear Regression
    • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
I F-test longitudinal data
    • Set Theory, Logic, Probability, Statistics
Replies
2
Views
993
MHB Confidence Interval for Child's Weight Based on Television Watching
    • Set Theory, Logic, Probability, Statistics
Replies
26
Views
3K
I Regression: which parameters to use and how to plot the data
    • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
I Regression on extracted factors
    • Set Theory, Logic, Probability, Statistics
Replies
2
Views
894
Cleaning/Reordering towards regression
    • Computing and Technology
Replies
6
Views
853
I Desirable estimator properties...
    • Set Theory, Logic, Probability, Statistics
Replies
7
Views
438

Hot Threads

Recent Insights

Back
Top