Least-Squares Error Variance

In summary, the formula for dividing by n-2 in simple linear regression is derived in section 7.2 of the textbook, and is based on the concept of degrees of freedom. In this type of regression, the total variability in the data can be attributed to two sources - the deterministic portion and the random portion. The formula for dividing by n-2 takes into account the number of data points and the number of points needed to fit a line (n-2), and is used because the calculation of SSE requires two pieces of work. This leaves 1 degree of freedom for SSR.
  • #1
Cyrus
3,238
16
In the textbook it says this:

http://img6.imageshack.us/img6/1896/imgcxv.jpg [Broken]

Where does this hocus pokus 'it turns out that dividing by n-2 rather than n appropriately compensates for this' come from?
 
Last edited by a moderator:
Physics news on Phys.org
  • #2
Cyrus said:
In the textbook it says this:

http://img6.imageshack.us/img6/1896/imgcxv.jpg [Broken]

Where does this hocus pokus 'it turns out that dividing by n-2 rather than n appropriately compensates for this' come from?

You divide by n-2 because you only have n-2 degrees of freedom. Are you by chance doing a least mean fit on a line where you need two points to determine a line.

Anyway, the result is only valid if your errors are statistical independent. If there is low frequency noise then you have even less then n-2 degrees of freedom.
 
Last edited by a moderator:
  • #3
John Creighto said:
You divide by n-2 because you only have n-2 degrees of freedom. Are you by chance doing a least mean fit on a line where you need two points to determine a line.

Anyway, the result is only valid if your errors are statistical independent. If there is low frequency noise then you have even less then n-2 degrees of freedom.

Why do I have n-2 degrees of freedom?
 
  • #4
Cyrus said:
Why do I have n-2 degrees of freedom?

The book says that the formula you are questioning is derived in (7.21) of section (7.2), maybe post that part if you are confused.

You are trying to fit a line to the data. Their are two points required to define a a line. You have n data points. The number degrees of freedom is equal to the number of data points minus the number points you need to fit a curve. It is therefore n-2.
 
  • #5
In simple linear regression two things go on.
First, you are expressing the mean value of [tex] Y [/tex] as linear function; this essentially says you are splitting [tex] Y [/tex] itself into two sources, a deterministic piece (the linear term) and a probabilistic term (the random error)

[tex]
Y = \underbrace{\beta_0 + \beta_1 x}_{\text{Deterministic}} +\overbrace{\varepsilon}^{\text{Random}}
[/tex]

When it comes to the ANOVA table, this also means that the total variability in [tex] Y [/tex] can be attributed to two sources: the deterministic portion and the random portion. It turns out that in this approach the variability in [tex] Y [/tex] can be broken into two sources. It is customary to do this with the sums of squares first. The basic notation used is

[tex]
\begin{align*}
SSE &= \sum (y-\widehat y)^2 \\
SST & = \sum (y-\overline y)^2 \\
SSR & = SST - SSE = \sum (\overline y - \widehat y)^2
\end{align*}
[/tex]

here
SST is the numerator of the usual sample variance of [tex] Y [/tex] - think of it as measuring the variability around the sample mean
SSE is the sum of the squared residuals - think of this as measuring the variability around
the regression line (which is another way of modeling the mean value of [tex] Y [/tex], when you think of it)
SSR is measures the error between the sample mean and the linear-regression predicted values

Every time you measure variability with a sum of squares like these, you have to worry about the appropriate degrees of freedom. Mathematically, these also add - just like the sums of squares do.

The ordinary sample variance has [tex] n - 1 [/tex] degrees of freedom. Perhaps think of this because, in order to calculate this, you must have done [tex] 1 [/tex] calculation, the sample mean. Thinking this way, [tex] SSE [/tex] must have [tex] n - 2 [/tex] degrees of freedom, since its calculation requires two pieces of work - the slope and the intercept.
This leaves [tex] 1 [/tex] degree of freedom for [tex] SSR [/tex].
 

What is Least-Squares Error Variance?

Least-Squares Error Variance is a statistical measure that calculates the average squared difference between the observed values of a variable and the predicted values from a regression model. It is used to assess the accuracy of a regression model and determine the amount of variability in the data that is not explained by the model.

How is Least-Squares Error Variance calculated?

The formula for calculating Least-Squares Error Variance involves summing the squared differences between the observed values and the predicted values, and then dividing by the number of data points minus the number of parameters in the regression model. This results in an estimate of the average squared error or variance.

What is the significance of Least-Squares Error Variance?

Least-Squares Error Variance is an important measure in regression analysis as it helps to evaluate the goodness of fit of a model. A lower error variance indicates a better fit and a higher level of accuracy in predicting the values of the dependent variable.

How does Least-Squares Error Variance relate to the concept of regression?

Regression analysis involves finding the best-fitting line or curve to a set of data points. Least-Squares Error Variance is used to determine the amount of error or variability in the data that is not accounted for by the regression model. It is a fundamental concept in regression analysis and is used to evaluate the performance of different regression models.

Can Least-Squares Error Variance be used to compare different regression models?

Yes, Least-Squares Error Variance can be used to compare the performance of different regression models. The model with the lowest error variance is considered to be the best fit for the data. However, it is important to note that other factors, such as the complexity of the model, should also be taken into consideration when comparing regression models.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
6
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
9
Views
1K
  • Introductory Physics Homework Help
Replies
1
Views
923
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • STEM Educators and Teaching
Replies
11
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
9
Views
979
Replies
1
Views
745
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
3K
Back
Top