1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Regression analysis

  1. Sep 10, 2008 #1
    What are the most sophisticated methods of performing regression analysis and how does least squares rank among them? Additionally which category would the least squares method fit into below (if any):
    Simple, Multiple, Non-linear, Robust, Ridge, Logistic


  2. jcsd
  3. Sep 15, 2008 #2


    User Avatar
    Homework Helper

    I'm not sure what you mean by "most sophisticated"? Many of the most robust methods require intensive calculations - is this what you mean? Least squares is probably the
    method we use most often for these (and possibly other) reasons:
    • It is the oldest method
    • It is based on the assumption of normality of errors, although this can be relaxed (asymptotically, as long as the errors are not to badly behaved, and as long as the design matrix satisfies certain conditions, Huber's condition being the most widely known)
    • People are familiar with it, and virtually every bit of software that performs regression implements least squares

    Linear regression can, depending on the person using the term, be the name given to Simple Linear Regression or Multiple Linear Regression, but the name does not automatically imply the calculations are performed with least squares. You can use methods based on ranks or M-estimation (both minimize some function of the residuals to obtain estimates, just not the sum of the squared residuals) and others are possible.
    There is no way that the least squares method could ever be considered robust so it doesn't fit in there. I refer to classical least squares here: some rank-based algorithms begin with a "scoring" for the residuals, then use least squares on what is, in essence, transformed data, to finish. The same is true for some M-estimation procedures)
    You could solve non-linear and logistic problems with the method of least squares - but more efficient ways exist.
  4. Sep 15, 2008 #3
    Well, I guess I mean most robust... but I am not quite sure what that means either :tongue:

    But seriously thanks for your reply. I was just looking for a little bit of an explanation and some info. And you provided me with both.
  5. Sep 15, 2008 #4


    User Avatar
    Homework Helper

    You are welcome - not many people offer thanks here.

    You can google robust regression - just stay away from Wikipedia - I have little regard for the mathematics that gets posted there (my grad degrees are in statistics and mathematics).
    Good luck with further investigations.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook