# Linear Regression and OLS

• I
fog37
TL;DR Summary
Understanding if linear regression can be done with other variants of least squares
Hello,

Simple linear regression aims at finding the slope and intercept of the best-fit line to for a pair of ##X## and ##Y## variables.
In general, the optimal intercept and slope are found using OLS. However, I learned that "O" means ordinary and there are other types of least square computations...

Question: is it possible to apply those variants of LS to the linear regression model, i.e., can we find the best-fit line parameters using something other than OLS (for example, I think there is "robust" LS, etc.)?

thank you!

Homework Helper
Gold Member
I have taken a course in regression and some courses that use regression techniques. From what I remember, the Ordinary in OLS refers to some assumptions we make, rather than the method
one assumption is: the residuals are randomly distributed.

Homework Helper
Gold Member
Simple linear regression aims at finding the slope and intercept of the best-fit line to for a pair of ##X## and ##Y## variables.
More precisely, it finds the line that uses the ##X## value to estimate the ##Y## values with the minimum sum-squared-errors for the ##Y## estimates. The phrase "best-fit line" can mean something different, referring to minimizing the sum-squared perpendicular distances from the data to the line.
In general, the optimal intercept and slope are found using OLS. However, I learned that "O" means ordinary and there are other types of least square computations...

Question: is it possible to apply those variants of LS to the linear regression model, i.e., can we find the best-fit line parameters using something other than OLS (for example, I think there is "robust" LS, etc.)?
This is an interesting question. I am not an expert in this, but I see ( https://en.wikipedia.org/wiki/Robust_regression ) that there are attempts to decrease the influence of outliers. Some methods have been implemented in R (see https://stat.ethz.ch/R-manual/R-patched/library/MASS/html/rlm.html ). I don't know if that implementation is publicly available. It is applied in an example in https://stats.oarc.ucla.edu/r/dae/robust-regression/

fog37
Staff Emeritus
Gold Member
Even least squares is not necessary. You can find a slope and intercept that minimize any penalty function you want.

fog37, scottdave and FactChecker
Homework Helper
Gold Member
Even least squares is not necessary. You can find a slope and intercept that minimize any penalty function you want.
Good point, although most penalty functions would require non-analytical iterative minimization algorithms that are less intuitive. Also, I do not know what the risk of introducing local minimums would be.

fog37