What is a large change in deviance?

  • Context: Graduate 
  • Thread starter Thread starter FallenApple
  • Start date Start date
  • Tags Tags
    Change
Click For Summary
SUMMARY

A large change in deviance in Generalized Linear Models (GLMs) is assessed by comparing deviance values, such as a shift from 3500 to 3200. However, this change does not automatically indicate a better model; practical significance must be evaluated based on the application. Larger models with more terms may fit the data better but risk overfitting, which can lead to poor predictive performance. To address this, model selection criteria like Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) should be employed to penalize less parsimonious models.

PREREQUISITES
  • Understanding of Generalized Linear Models (GLMs)
  • Familiarity with deviance as a model evaluation metric
  • Knowledge of model selection criteria, specifically AIC and BIC
  • Concept of overfitting in statistical modeling
NEXT STEPS
  • Research the implications of deviance in GLMs and its practical applications
  • Learn about model selection techniques using AIC and BIC
  • Explore methods to detect and mitigate overfitting in statistical models
  • Study the impact of model complexity on predictive accuracy in GLMs
USEFUL FOR

Statisticians, data scientists, and researchers involved in model building and evaluation, particularly those working with Generalized Linear Models and seeking to optimize model performance.

FallenApple
Messages
564
Reaction score
61
So for GLMs, what is considered a large change in deviance?

If I go from 3500 in one model to 3200 in another, does that mean that the second model is better?
 
Physics news on Phys.org
That depends on the application. Does it make a practical difference in the intended purpose?

A larger model (more terms) will always fit the data better. In extreme cases this is just fitting to noise. You need to penalize the less parsimonious model. Two common ways of doing so are the AIC or the BIC.
 
Dale said:
That depends on the application. Does it make a practical difference in the intended purpose?

A larger model (more terms) will always fit the data better. In extreme cases this is just fitting to noise. You need to penalize the less parsimonious model. Two common ways of doing so are the AIC or the BIC.
Not sure if it would make a practical difference. Its just a small percentage loss for the deviance.

Is it because a model that is overfitted would not be able to capture the next point that is added to the model? Because the overfitted model would have the curve zigzag around the data to the point where the function is hardly even is a function.
 

Similar threads

Undergrad Expected coefficient change from simple to multiple linear regression
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Multicollinearity and Interactions
  • · Replies 5 ·
Replies
5
Views
2K
Graduate Second-order arithmetic
  • · Replies 34 ·
2
Replies
34
Views
4K
Graduate Analysing Sample's Statistical Fluctuation Probability
  • · Replies 3 ·
Replies
3
Views
2K
High School Central Limit Theorem: How does sample size affect the sampling distribution?
  • · Replies 31 ·
2
Replies
31
Views
5K
MHB Help Solve Variance Question with 80 Students
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Coefficient sign flip in linear regression with correlated predictors
  • · Replies 13 ·
Replies
13
Views
5K
Undergrad Best way to quantify how well a model fits data
  • · Replies 20 ·
Replies
20
Views
3K
Undergrad Collinearity between predictors: what happens under the hood
  • · Replies 13 ·
Replies
13
Views
2K
Undergrad What are the commonly used estimators in regression models?
  • · Replies 23 ·
Replies
23
Views
4K