Regression Analysis: Most Sophisticated Methods & Least Squares

[Diffy]

What are the most sophisticated methods of performing regression analysis and how does least squares rank among them? Additionally which category would the least squares method fit into below (if any):
Simple, Multiple, Non-linear, Robust, Ridge, Logistic

Thanks,

-Diffy

 

[statdad]

I'm not sure what you mean by "most sophisticated"? Many of the most robust methods require intensive calculations - is this what you mean? Least squares is probably the
method we use most often for these (and possibly other) reasons:

• It is the oldest method
• It is based on the assumption of normality of errors, although this can be relaxed (asymptotically, as long as the errors are not to badly behaved, and as long as the design matrix satisfies certain conditions, Huber's condition being the most widely known)
• People are familiar with it, and virtually every bit of software that performs regression implements least squares

Linear regression can, depending on the person using the term, be the name given to Simple Linear Regression or Multiple Linear Regression, but the name does not automatically imply the calculations are performed with least squares. You can use methods based on ranks or M-estimation (both minimize some function of the residuals to obtain estimates, just not the sum of the squared residuals) and others are possible.
There is no way that the least squares method could ever be considered robust so it doesn't fit in there. I refer to classical least squares here: some rank-based algorithms begin with a "scoring" for the residuals, then use least squares on what is, in essence, transformed data, to finish. The same is true for some M-estimation procedures)
You could solve non-linear and logistic problems with the method of least squares - but more efficient ways exist.

 

[Diffy]

I'm not sure what you mean by "most sophisticated"? Many of the most robust methods require intensive calculations - is this what you mean?

Well, I guess I mean most robust... but I am not quite sure what that means either :tongue:

But seriously thanks for your reply. I was just looking for a little bit of an explanation and some info. And you provided me with both.

 

[statdad]

But seriously thanks for your reply

You are welcome - not many people offer thanks here.

You can google robust regression - just stay away from Wikipedia - I have little regard for the mathematics that gets posted there (my grad degrees are in statistics and mathematics).
Good luck with further investigations.