Decision Tree Regression: Avoiding Overfitting in Training Data

Click For Summary
SUMMARY

The discussion focuses on avoiding overfitting in Decision Tree Regression using the Scikit-learn library. A user identified that their model was overfitting by capturing noise in the training data, particularly with outliers. They adjusted parameters to reduce the number of outliers from 7 to 5 but sought confirmation on the effectiveness of their approach. The conversation emphasizes the importance of parameter tuning in achieving a more generalized model.

PREREQUISITES
  • Understanding of Decision Tree Regression
  • Familiarity with Scikit-learn version 0.24 or later
  • Knowledge of overfitting and its implications in machine learning
  • Basic skills in data visualization to interpret regression graphs
NEXT STEPS
  • Explore parameter tuning techniques in Scikit-learn Decision Trees
  • Learn about cross-validation methods to assess model performance
  • Investigate regularization techniques to prevent overfitting
  • Study the impact of outlier detection methods on regression models
USEFUL FOR

Data scientists, machine learning engineers, and anyone involved in developing predictive models using Decision Trees who seeks to enhance model accuracy and generalization.

falyusuf
Messages
35
Reaction score
3
Homework Statement
Remove the overfitting in the following example.
Relevant Equations
-
The decision tree in the following curve is too fine details of the training data and learn from the noise, (overfitting).
overfitting.png

Ref: https://scikit-learn.org/stable/aut...lr-auto-examples-tree-plot-tree-regression-py

I tried to remove the overfitting but not sure about the result, can someone confirm my answer? Here's what I got:
result.png
 
Physics news on Phys.org
From your graph, it appears that you trained to 5 outliers at max=5, vs the 7 outliers in the original graph. What parameter did you adjust?
 

Similar threads

Undergrad Ridge Regression Cross Validation
  • · Replies 4 ·
Replies
4
Views
2K
Current and Future AI Threats to Humanity and the Human Response
  • · Replies 10 ·
Replies
10
Views
5K
Requesting suggestions for languages, libraries, and architectures for parallel (and sometimes non parallel) numerical and scientific computations
  • · Replies 8 ·
Replies
8
Views
4K
The Scientist as Hero - or not
  • · Replies 1 ·
Replies
1
Views
3K
Why is the U-2 nicknamed the Dragon Lady?
  • · Replies 4 ·
Replies
4
Views
9K