Why Divide by n-2 in Least-Squares Error Variance Calculation?

[Cyrus]

In the textbook it says this:

http://img6.imageshack.us/img6/1896/imgcxv.jpg

Where does this hocus pokus 'it turns out that dividing by n-2 rather than n appropriately compensates for this' come from?

 

[John Creighto]

In the textbook it says this:

http://img6.imageshack.us/img6/1896/imgcxv.jpg

Where does this hocus pokus 'it turns out that dividing by n-2 rather than n appropriately compensates for this' come from?

You divide by n-2 because you only have n-2 degrees of freedom. Are you by chance doing a least mean fit on a line where you need two points to determine a line.

Anyway, the result is only valid if your errors are statistical independent. If there is low frequency noise then you have even less then n-2 degrees of freedom.

 

[Cyrus]

You divide by n-2 because you only have n-2 degrees of freedom. Are you by chance doing a least mean fit on a line where you need two points to determine a line.

Anyway, the result is only valid if your errors are statistical independent. If there is low frequency noise then you have even less then n-2 degrees of freedom.

Why do I have n-2 degrees of freedom?

 

[John Creighto]

Why do I have n-2 degrees of freedom?

The book says that the formula you are questioning is derived in (7.21) of section (7.2), maybe post that part if you are confused.

You are trying to fit a line to the data. Their are two points required to define a a line. You have n data points. The number degrees of freedom is equal to the number of data points minus the number points you need to fit a curve. It is therefore n-2.

 

[statdad]

In simple linear regression two things go on.
First, you are expressing the mean value of $$Y$$ as linear function; this essentially says you are splitting $$Y$$ itself into two sources, a deterministic piece (the linear term) and a probabilistic term (the random error)

$$Y = \underbrace{\beta_0 + \beta_1 x}_{\text{Deterministic}} +\overbrace{\varepsilon}^{\text{Random}}$$

When it comes to the ANOVA table, this also means that the total variability in $$Y$$ can be attributed to two sources: the deterministic portion and the random portion. It turns out that in this approach the variability in $$Y$$ can be broken into two sources. It is customary to do this with the sums of squares first. The basic notation used is

$$\begin{align*}<br /> SSE &= \sum (y-\widehat y)^2 \\<br /> SST & = \sum (y-\overline y)^2 \\<br /> SSR & = SST - SSE = \sum (\overline y - \widehat y)^2<br /> \end{align*}$$

here
SST is the numerator of the usual sample variance of $$Y$$ - think of it as measuring the variability around the sample mean
SSE is the sum of the squared residuals - think of this as measuring the variability around
the regression line (which is another way of modeling the mean value of $$Y$$, when you think of it)
SSR is measures the error between the sample mean and the linear-regression predicted values

Every time you measure variability with a sum of squares like these, you have to worry about the appropriate degrees of freedom. Mathematically, these also add - just like the sums of squares do.

The ordinary sample variance has $$n - 1$$ degrees of freedom. Perhaps think of this because, in order to calculate this, you must have done $$1$$ calculation, the sample mean. Thinking this way, $$SSE$$ must have $$n - 2$$ degrees of freedom, since its calculation requires two pieces of work - the slope and the intercept.
This leaves $$1$$ degree of freedom for $$SSR$$.