F-test regression test, when and how?

Click For Summary
SUMMARY

The discussion centers on the application of F-tests for comparing nested regression models, specifically two models of the form Y = a1*X1 + a2*X2 + b and Y = a3*X1 + a4*X2 + b. Participants clarify that while F-tests can be used to compare each model against a reduced model (Y = b), direct comparison between the two models is not standard practice. Bayesian analysis is suggested as an alternative method for model comparison, particularly due to its sensitivity to prior distributions. Resources for Bayesian statistics are provided to assist in further understanding.

PREREQUISITES
  • Understanding of F-tests in regression analysis
  • Knowledge of nested regression models
  • Familiarity with Bayesian statistics and Bayes factors
  • Experience with statistical software for model fitting
NEXT STEPS
  • Research the application of F-tests in nested regression models
  • Learn about Bayesian model comparison techniques
  • Explore the use of statistical software for regression analysis
  • Read about the implications of prior distributions in Bayesian analysis
USEFUL FOR

Statisticians, data analysts, and researchers involved in regression modeling and hypothesis testing, particularly those interested in comparing models using F-tests and Bayesian methods.

RobosaurusRex
Messages
29
Reaction score
1
I am aware that f-tests can be used to check the null hypothesis when comparing regression models if the models are nested.

What I am confused about is if I can apply an f-test to compare the following, (and if so what is the best way)

I have two regression laws
Y = a1*X1 + a2*X2 + b
Y = a3*X1 + a4*X2 + b

Is the best way to test these quantitatively to compare each in turn against the reduced model Y = b
or can I compare them against one another directly by using the f-test?
 
Physics news on Phys.org
Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior
 
Dale said:
Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior

I have been trying to follow the analysis of a paper where they claim that f-tests show model a is significantly better than model b.
Any ideas?

If not, can you link me to something regarding this bayesian analysis?
 
Well, comparing the two models you described seems strange since they are the same form. I don't know why you would have two models of the same form with different coefficients from the same data set.

However, there certainly is plenty of information about Bayesian statistics

I would start here for a basic intro
https://en.m.wikipedia.org/wiki/Bayesian_probability
https://en.m.wikipedia.org/wiki/Bayesian_inference

And here for a basic practical method
http://www.indiana.edu/~kruschke/BEST/BEST.pdf
 
I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?
 
WWGD said:
I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?
Hi sorry for the lack on context, this may help.

So i have a bunch of model output which essentially boils down to a plot of X against Y, where X can either contain one or two variables

There are two derived 'scalings' based on the governing physics nd I want to test if the two are quantitatively different when describing the data.
So I want to be quite rigorous, I have done some 'relative misfit' calculations and now I am trying to do an f-test as it was performed in the literature but obviously they dedicate a sentence to the result and nothing about how it is done
 

Similar threads

Undergrad Linear regression and random variables
  • · Replies 30 ·
2
Replies
30
Views
4K
Undergrad Coefficient sign flip in linear regression with correlated predictors
  • · Replies 13 ·
Replies
13
Views
4K
Undergrad What is Nonlinear Least Squares in Regression Analysis?
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad What Are the Key Differences Between OLS and GLM Models in Statistical Analysis?
  • · Replies 3 ·
Replies
3
Views
3K
Undergrad How to propagate errors through a regression & non-linear model?
  • · Replies 8 ·
Replies
8
Views
3K
Graduate "Population-averaged"regression on panel data using Stata
  • · Replies 1 ·
Replies
1
Views
2K
Graduate Follow-Up on F-Test in Multi-Linear Regression
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Is the F-test Calculation Different for Longitudinal Data in Linear Regression?
  • · Replies 2 ·
Replies
2
Views
1K
Undergrad Regression Prediction with Time Series Data
  • · Replies 1 ·
Replies
1
Views
1K
Graduate Mathematical Connection between Cosmic Expansion and Exponential Growth
  • · Replies 24 ·
Replies
24
Views
2K