Undergrad Bias in Linear Regression (x-intercept) vs Statistics

Click For Summary
In simple regression, the bias represented by the intercept (b) differs from the statistical definition of bias in estimators, which is the difference between the expected value and the true parameter. The model Y = mx + b can produce an unbiased estimator for the population parameter Y, leading to confusion about the term "bias." The terminology used in machine learning may borrow from statistics but does not align perfectly, creating a disparity in understanding. This discrepancy highlights the need for clarity in the use of terms across different fields. The discussion emphasizes that while the concepts may be related, they are not interchangeable.
WWGD
Science Advisor
Homework Helper
Messages
7,757
Reaction score
12,977
TL;DR
Trying to Reconcile two apparently/superficially different usages of the tern "Bias"
Hi,
In simple regression for machine learning , a model :

Y=mx +b ,

Is said AFAIK, to have bias equal to b. Is there a relation between the use of bias here and the use of bias in terms of estimators

for population parameters, i.e., the bias of an estimator P^ for a population parameter P is defined as the difference E[P^]- P?

The two do not seem to coincide as Y^= mx^ +b^ is an unbiased estimator of the population parameter Y . Can anyone explain the

disparity?
 
Physics news on Phys.org
Words have more than one meaning. I have never seen bias used with the first meaning, so that appears to be a specialized field of study just “hijacking” terminology from other fields of study. It happens often. I am afraid there is not much justification needed or provided for that type of thing.
 
  • Like
Likes FactChecker and WWGD
I think that the two uses are only logically similar in the context of a model where X and Y are known or assumed to be proportional (Y = mx). In that case, b would be a bias due to something.
 
  • Like
Likes Dale and WWGD
First trick I learned this one a long time ago and have used it to entertain and amuse young kids. Ask your friend to write down a three-digit number without showing it to you. Then ask him or her to rearrange the digits to form a new three-digit number. After that, write whichever is the larger number above the other number, and then subtract the smaller from the larger, making sure that you don't see any of the numbers. Then ask the young "victim" to tell you any two of the digits of the...

Similar threads

  • · Replies 23 ·
Replies
23
Views
4K
Replies
3
Views
3K
  • · Replies 30 ·
2
Replies
30
Views
4K
  • · Replies 6 ·
Replies
6
Views
3K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 64 ·
3
Replies
64
Views
5K
  • · Replies 19 ·
Replies
19
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 14 ·
Replies
14
Views
3K