Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

When is Linear Model not Good despite r^2 close to 1?

  1. Oct 3, 2011 #1
    Hi, All:
    I was reading of cases in which linear models in least-squares regression were found to be
    innefective, despite values of r, r^2 being close to 1 (obviously, both go together ).
    I think the issue has to see with the distribution of the residuals being distinctively non-linear (and, definitely, not being normal), e.g., having a histogram that looks like a parabola, or a cubic, etc.
    Just curious to see if someone knows of some examples and/or results in this respect, and of what other checks can be made to see if a linear model makes sense for a data set. Checks I know of are Lack-of-fit Sum of Squares F-test and inference for regression (with Ho:= Slope is zero.)

  2. jcsd
  3. Oct 5, 2011 #2
    Another way - suppose there is overfitting, or not enough data points for the number of dimensions. If you have 100 data points but are using a model with 100 different dimensions it doesn't matter how good your correlation is.
  4. Oct 6, 2011 #3


    User Avatar
    Homework Helper

    A high [itex] R^{2} [/itex] is not the only important statistic to check. I prefer adjusted [itex] R^{2} [/itex], because the more parameters you add to the former it'll tend to inflate it.
  5. Oct 6, 2011 #4
    Thanks, Pyrrhus:

    What do I then do if the adjusted R^2 is low ? Do I start considering linear models on two-or-more variables, or do I consider quadratic, cubic, etc. models?
  6. Oct 7, 2011 #5


    User Avatar
    Homework Helper

    You could try adding square terms, and interaction terms, but if the r-squared is still low it might just be that the regressors don't do a good job to explain the dependent variable.
  7. Oct 8, 2011 #6


    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    Last edited by a moderator: Apr 26, 2017
  8. Oct 8, 2011 #7
    Excellent, Thanks!.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook