F-test regression test, when and how?

Click For Summary

Discussion Overview

The discussion revolves around the application of F-tests in comparing regression models, specifically when the models are nested. Participants explore the appropriateness of using F-tests for comparing two regression equations and consider alternative approaches, such as Bayesian analysis, to assess model performance.

Discussion Character

  • Debate/contested
  • Technical explanation
  • Mathematical reasoning

Main Points Raised

  • One participant questions whether an F-test can be applied to compare two regression models directly or if they should be compared against a reduced model.
  • Another participant suggests using Bayesian analysis instead of F-tests, noting its sensitivity to prior assumptions.
  • A participant expresses confusion about the rationale for having two models of the same form with different coefficients derived from the same dataset.
  • Some participants emphasize the role of software in generating models and goodness-of-fit information, questioning the necessity of having different models without clear justification.
  • A later reply provides additional context about the participant's setup, mentioning derived scalings based on governing physics and the desire for a rigorous quantitative comparison.

Areas of Agreement / Disagreement

Participants express differing views on the appropriateness of F-tests versus Bayesian analysis for model comparison. There is no consensus on the best approach, and the discussion remains unresolved regarding the application of F-tests in this context.

Contextual Notes

Participants highlight limitations in the original question, including a lack of context regarding the data setup and the nature of the models being compared. There are also unresolved assumptions about the models' derivation and the appropriateness of different statistical tests.

RobosaurusRex
Messages
29
Reaction score
1
I am aware that f-tests can be used to check the null hypothesis when comparing regression models if the models are nested.

What I am confused about is if I can apply an f-test to compare the following, (and if so what is the best way)

I have two regression laws
Y = a1*X1 + a2*X2 + b
Y = a3*X1 + a4*X2 + b

Is the best way to test these quantitatively to compare each in turn against the reduced model Y = b
or can I compare them against one another directly by using the f-test?
 
Physics news on Phys.org
Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior
 
Dale said:
Personally, I would use a Bayesian analysis for this. You can easily select models based on the Bayes factor. But it is sensitive to your prior

I have been trying to follow the analysis of a paper where they claim that f-tests show model a is significantly better than model b.
Any ideas?

If not, can you link me to something regarding this bayesian analysis?
 
Well, comparing the two models you described seems strange since they are the same form. I don't know why you would have two models of the same form with different coefficients from the same data set.

However, there certainly is plenty of information about Bayesian statistics

I would start here for a basic intro
https://en.m.wikipedia.org/wiki/Bayesian_probability
https://en.m.wikipedia.org/wiki/Bayesian_inference

And here for a basic practical method
http://www.indiana.edu/~kruschke/BEST/BEST.pdf
 
I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?
 
WWGD said:
I agree with Dale; usually you input data into some software and the software will spit out a model together with goodness of fit info related to the test at hand. Unless your software is doing iterations and gave you different ones, I don't see why you would end up with different models. Would you explain the setup you are using?
Hi sorry for the lack on context, this may help.

So i have a bunch of model output which essentially boils down to a plot of X against Y, where X can either contain one or two variables

There are two derived 'scalings' based on the governing physics nd I want to test if the two are quantitatively different when describing the data.
So I want to be quite rigorous, I have done some 'relative misfit' calculations and now I am trying to do an f-test as it was performed in the literature but obviously they dedicate a sentence to the result and nothing about how it is done
 

Similar threads

Undergrad Linear regression and random variables
  • · Replies 30 ·
2
Replies
30
Views
5K
Undergrad Coefficient sign flip in linear regression with correlated predictors
  • · Replies 13 ·
Replies
13
Views
5K
Undergrad What is Nonlinear Least Squares in Regression Analysis?
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad What Are the Key Differences Between OLS and GLM Models in Statistical Analysis?
  • · Replies 3 ·
Replies
3
Views
3K
Undergrad How to propagate errors through a regression & non-linear model?
  • · Replies 8 ·
Replies
8
Views
3K
Graduate "Population-averaged"regression on panel data using Stata
  • · Replies 1 ·
Replies
1
Views
2K
Graduate Follow-Up on F-Test in Multi-Linear Regression
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Is the F-test Calculation Different for Longitudinal Data in Linear Regression?
  • · Replies 2 ·
Replies
2
Views
1K
Undergrad Regression Prediction with Time Series Data
  • · Replies 1 ·
Replies
1
Views
2K
Graduate Mathematical Connection between Cosmic Expansion and Exponential Growth
  • · Replies 24 ·
Replies
24
Views
3K