Regression Analysis: Most Sophisticated Methods & Least Squares

Click For Summary

Discussion Overview

The discussion revolves around the various methods of regression analysis, specifically focusing on the sophistication of these methods compared to least squares. Participants explore the categorization of least squares within different regression types and seek clarification on what constitutes "sophisticated" or "robust" methods.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested

Main Points Raised

  • Diffy questions the ranking of least squares among sophisticated regression methods and seeks to categorize it within various types.
  • One participant suggests that "most sophisticated" may refer to methods requiring intensive calculations and notes that least squares is commonly used due to its historical significance and familiarity.
  • This participant also mentions that least squares is based on the assumption of normality of errors but can be relaxed under certain conditions.
  • There is a clarification that linear regression can refer to both simple and multiple linear regression, but it does not necessarily imply the use of least squares.
  • Another participant asserts that least squares cannot be considered robust and discusses alternative methods like rank-based algorithms and M-estimation.
  • Participants express uncertainty about the definition of "most robust" and "sophisticated" methods, indicating a lack of clarity in these terms.
  • One participant advises against using Wikipedia for mathematical references, citing a lack of regard for its content.

Areas of Agreement / Disagreement

Participants do not reach a consensus on the definitions of "sophisticated" or "robust" methods, and there are competing views on the categorization of least squares and its robustness.

Contextual Notes

There are unresolved assumptions regarding the definitions of sophistication and robustness in regression methods, as well as the conditions under which least squares may be applied.

Diffy
Messages
441
Reaction score
0
What are the most sophisticated methods of performing regression analysis and how does least squares rank among them? Additionally which category would the least squares method fit into below (if any):
Simple, Multiple, Non-linear, Robust, Ridge, Logistic

Thanks,

-Diffy
 
Physics news on Phys.org
I'm not sure what you mean by "most sophisticated"? Many of the most robust methods require intensive calculations - is this what you mean? Least squares is probably the
method we use most often for these (and possibly other) reasons:
  • It is the oldest method
  • It is based on the assumption of normality of errors, although this can be relaxed (asymptotically, as long as the errors are not to badly behaved, and as long as the design matrix satisfies certain conditions, Huber's condition being the most widely known)
  • People are familiar with it, and virtually every bit of software that performs regression implements least squares


Linear regression can, depending on the person using the term, be the name given to Simple Linear Regression or Multiple Linear Regression, but the name does not automatically imply the calculations are performed with least squares. You can use methods based on ranks or M-estimation (both minimize some function of the residuals to obtain estimates, just not the sum of the squared residuals) and others are possible.
There is no way that the least squares method could ever be considered robust so it doesn't fit in there. I refer to classical least squares here: some rank-based algorithms begin with a "scoring" for the residuals, then use least squares on what is, in essence, transformed data, to finish. The same is true for some M-estimation procedures)
You could solve non-linear and logistic problems with the method of least squares - but more efficient ways exist.
 
statdad said:
I'm not sure what you mean by "most sophisticated"? Many of the most robust methods require intensive calculations - is this what you mean?

Well, I guess I mean most robust... but I am not quite sure what that means either :-p

But seriously thanks for your reply. I was just looking for a little bit of an explanation and some info. And you provided me with both.
 
But seriously thanks for your reply
You are welcome - not many people offer thanks here.

You can google robust regression - just stay away from Wikipedia - I have little regard for the mathematics that gets posted there (my grad degrees are in statistics and mathematics).
Good luck with further investigations.
 

Similar threads

  • · Replies 4 ·
Replies
4
Views
2K
Replies
3
Views
3K
  • · Replies 6 ·
Replies
6
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 21 ·
Replies
21
Views
3K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K