Simple regression: not including the intercept term

[939]

Homework Statement

The simple regression model is y = α + βx + u, where u is the error term. If you don't include α, when is β unbiased?

Homework Equations

y = α + βx + u

The Attempt at a Solution

Not including α doesn't affect whether β is unbiased because α is a constant.

 

[Ray Vickson]

If the true model is $y = \beta x + \epsilon$, you get an unbiased estimate of $\beta$ by using the least-squares method on the model $\hat{y} = a + b x$---including the intercept! The point is that $a, b$ are both unbiased for the true model $y = \alpha+\beta x + \epsilon$, and this is true even if it happens that $\alpha = 0$. Therefore, my guess would be that the estimated obtained from the no-intercept fit $\hat{y} = bx$ would be biased. After all, the two estimates of $\beta$ would be given by different formulas in the $(x_i, y_i)$ data points, and one of the formulas gives an unbiased result.

 

[statdad]

If \alpha \ne 0 but you fit the "no intercept" model then the estimate of the slope will be biased. To see this begin with
<br /> E(b) = E[\left( X&#039;X \right)^{-1}X&#039; y] = E[\left( X&#039;X \right)^{-1}X&#039; \left(\alpha + X \beta + \epsilon \right)]<br />

and work through the right side. You'll be able to see the only two conditions where the estimate of the slope won't be biased. Essentially - it's biased because you're fitting an incorrect model: fitting no intercept when one exists.

Regression without the intercept is rarely a good idea, for this reason AND for the fact that it means the traditional R^2 statistic is rendered useless (there are other issues as well).