What to do with biased estimators if we don't know the bias term?

[fog37]

TL;DR Summary

dealing with biased estimators

Hello,
I understand that we have a population of values. We don't know the parameters of this population. The parameters are numbers, each one describing the population in a collective sense. Examples of parameters are the mean, the median, the mode, the variance, skewness, kurtosis, etc.

We then take a single random sample and work with it to estimate the population parameters. For some parameters, the estimator we use to estimate the parameter itself is unbiased: it means that, on average, if we took many many samples, the average of the estimates, one from each sample, would end up being equal to the population parameter itself. That is great. The estimates, based on the CLM, will approximate a normal distribution centered at the population parameter....

• What if the estimator we choose use to estimate a specific population parameter is "biased"? We always prefer for an estimator to be unbiased but I guess that is not always possible....Why not?
• When an estimator is biased, the average of all the estimates (if we collected infinitely many) will not be equal to the parameter itself. The expectation value of estimate will be off by a fixed bias/constant term $b$ from the true population parameter. That would not be good! The sampling distribution of all the sample estimates will still tend to be normal. Conceptually, what do we if we don't know the bias term $b$? Are there situations in which we would be able to know the magnitude of $b$? Are there techniques we can use to figure $b$ out from the single random sample that we collected?

thank you!

 

[FactChecker]

What if the estimator we choose use to estimate a specific population parameter is "biased"? We always prefer for an estimator to be unbiased but I guess that is not always possible....Why not?

It is not true that the unbiased estimator is always best. It depends on what the distribution, parameter, and use of the parameter are. See this example of the parameter of the Poisson distribution.

• When an estimator is biased, the average of all the estimates (if we collected infinitely many) will not be equal to the parameter itself. The expectation value of estimate will be off by a fixed bias/constant term $b$ from the true population parameter. That would not be good! The sampling distribution of all the sample estimates will still tend to be normal. Conceptually, what do we if we don't know the bias term $b$? Are there situations in which we would be able to know the magnitude of $b$? Are there techniques we can use to figure $b$ out from the single random sample that we collected?

It is illustrative to consider the equation for the sample variance when the sample mean, $\bar X$, is used rather than the true population mean, $\mu$:
$\sum {(x_i - \bar X)}/(n-1)$
IMO, the natural first guess would be to divide by $n$ rather than by $(n-1)$. But that is a biased estimator. Using the estimated mean, $\bar X$, rather than the true population mean, $\mu$, gives a smaller summation because $\bar X$ tends to be closer to the majority of the sample than $\mu$ is. Luckily, dividing by $(n-1)$ gives an unbiased estimator.
In other situations, I think that the best thing to do about a bias depends on the distribution, the parameter, and your intended use of the parameter.