Bias in Linear Regression (x-intercept) vs Statistics

Click For Summary
SUMMARY

The discussion centers on the concept of bias in linear regression, specifically the bias term 'b' in the equation Y=mx+b and its relation to the bias of estimators in statistics. The user questions the connection between the bias in machine learning models and the bias of an estimator for population parameters, highlighting that Y^=mx^+b^ serves as an unbiased estimator for Y. The conversation reveals a potential terminological confusion, suggesting that the usage of 'bias' in different contexts may not align, particularly when considering proportional relationships between X and Y.

PREREQUISITES
  • Understanding of linear regression models, specifically Y=mx+b
  • Familiarity with statistical bias in estimators
  • Knowledge of machine learning concepts and terminology
  • Basic grasp of population parameters and their estimators
NEXT STEPS
  • Research the implications of bias in linear regression models
  • Study the definitions and applications of bias in statistical estimators
  • Explore the relationship between machine learning models and statistical theory
  • Learn about the impact of terminology in different fields of study
USEFUL FOR

Data scientists, statisticians, machine learning practitioners, and anyone interested in the nuances of bias in regression analysis and statistical estimators.

WWGD
Science Advisor
Homework Helper
Messages
7,777
Reaction score
13,013
TL;DR
Trying to Reconcile two apparently/superficially different usages of the tern "Bias"
Hi,
In simple regression for machine learning , a model :

Y=mx +b ,

Is said AFAIK, to have bias equal to b. Is there a relation between the use of bias here and the use of bias in terms of estimators

for population parameters, i.e., the bias of an estimator P^ for a population parameter P is defined as the difference E[P^]- P?

The two do not seem to coincide as Y^= mx^ +b^ is an unbiased estimator of the population parameter Y . Can anyone explain the

disparity?
 
Physics news on Phys.org
Words have more than one meaning. I have never seen bias used with the first meaning, so that appears to be a specialized field of study just “hijacking” terminology from other fields of study. It happens often. I am afraid there is not much justification needed or provided for that type of thing.
 
  • Like
Likes   Reactions: FactChecker and WWGD
I think that the two uses are only logically similar in the context of a model where X and Y are known or assumed to be proportional (Y = mx). In that case, b would be a bias due to something.
 
  • Like
Likes   Reactions: Dale and WWGD

Similar threads

  • · Replies 23 ·
Replies
23
Views
4K
Replies
3
Views
3K
  • · Replies 30 ·
2
Replies
30
Views
4K
  • · Replies 6 ·
Replies
6
Views
3K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 64 ·
3
Replies
64
Views
5K
  • · Replies 19 ·
Replies
19
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 14 ·
Replies
14
Views
3K