Regression: Regularization parameter 0

[SchroedingersLion]

TL;DR

ScikitLearn Elastic Net regression gives a hyperparameter of 0, implying that ordinary least squares is the best method.

Hi guys,

I am using ScikitLearn's Elastic Net implementation to perform regression on a data set where number of data points is larger than number of features. The routine uses crossvalidation to find the two hyperparameters: ElasticNetCV
The elastic net minimizes $\frac {1}{2N} ||y-Xw||^2 + \alpha c ||w||_1 + \frac 1 2 \alpha (1-c) ||w||_2^2$, where $\alpha$ and $c$ are the hyperparameters.

However, I obtain a hyperparameter of $\alpha=0$, which means the routine prefers no regularization at all. I was wondering what this means. The regularization is done in order to decrease overfitting on test data. What does a parameter of 0 imply? Does it mean I cannot have overfitting in this case?SL

 

[BWV]

Generally, if the number of features <<< data points and there is little correlation between them, then OLS is the best method

 

[SchroedingersLion]

So overfitting is not an issue when I have more data points than features?
Makes somewhat sense, because if I want to fit a line with n parameters, and I have N>>n data points, I can not hit each data point as good as I want, not even in the training data. So overfitting is suppressed.

 

[BWV]

The elastic net is a combination of lasso and ridge and will penalize collinear and low t-stat variables - so if you get the same results as an OLS your predictors are fine

 

[SchroedingersLion]

Thank you!

 

[BWV]

Should add that none of these methods deal well with autocorellation - need GLS or GMM for that