- #1
Phoeniyx
- 16
- 1
Hey guys. I have some trouble understanding how the F-test is used for testing the viability of a regression model. Before I delve into the background/question, just wanted to post a link that discusses the topic briefly:
http://www.stat.yale.edu/Courses/1997-98/101/anovareg.htm
So, coming back to the question, let's say we have 77 data points (xi, yi) and we try to fit it to a regression model as:
\hat{y}_{i} = A + Bx_{i}
In the Yale example, A and B are calculated based on the 77 data points.
To check if the model is significant, we calculated the "explained sum of squares" (ESS) which is the squared difference between the model estimate and the mean: ESS = \Sigma{(\hat{y}_{i} - \bar{y})^{2}}
Then we calculate the "residual sum of squares" (RSS) which is the squared difference between the actual data point and the model: RSS = \Sigma{({y}_{i} - \hat{y}_{i})^{2}}
The degrees of freedom for RSS is: # of data points - estimated model parameters from data = 77 - 2 = 75. Perfectly fine with this.
BUT, apparently, the degrees of freedom for ESS is "1"... I do not get this. More on why I am confused later in the questions section.
The F-test value = \frac{ESS / DF_{ESS}}{RSS / DF_{RSS}}, where DF = degrees of freedom
In the Yale link above, this calculates to 8654.7/84.6 = 102.35
So, I have two questions:
1) Since the degrees of freedom of ESS is always "1", if I had 85 data points (instead of 77), the F-test value would be even larger - since ESS is not averaged and is simply the sum of squares between model value and mean. e.g. the 8654.7 above could be 9500 and the 84.6 (RSS/DF) above could be higher or lower - but will likely be still around 85 (larger sum of squares / 85). Wouldn't this imply that the significance is a function of the # of data points tested on the model (as opposed to real-world observed data points)? Related to this question, does the # of (x, y) real-world observed points used for RSS calculation (e.g. 77) must be the same as the # of points for the ESS calculation? e.g. can ESS be based on 95 model trials while RSS is only based on the 77 real world values?
2) I still don't understand (conceptually) why the F-test works to check whether the model parameters A and B are zero or not. Conceptually, how does this make sense?
Thanks very much everyone.
http://www.stat.yale.edu/Courses/1997-98/101/anovareg.htm
So, coming back to the question, let's say we have 77 data points (xi, yi) and we try to fit it to a regression model as:
\hat{y}_{i} = A + Bx_{i}
In the Yale example, A and B are calculated based on the 77 data points.
To check if the model is significant, we calculated the "explained sum of squares" (ESS) which is the squared difference between the model estimate and the mean: ESS = \Sigma{(\hat{y}_{i} - \bar{y})^{2}}
Then we calculate the "residual sum of squares" (RSS) which is the squared difference between the actual data point and the model: RSS = \Sigma{({y}_{i} - \hat{y}_{i})^{2}}
The degrees of freedom for RSS is: # of data points - estimated model parameters from data = 77 - 2 = 75. Perfectly fine with this.
BUT, apparently, the degrees of freedom for ESS is "1"... I do not get this. More on why I am confused later in the questions section.
The F-test value = \frac{ESS / DF_{ESS}}{RSS / DF_{RSS}}, where DF = degrees of freedom
In the Yale link above, this calculates to 8654.7/84.6 = 102.35
So, I have two questions:
1) Since the degrees of freedom of ESS is always "1", if I had 85 data points (instead of 77), the F-test value would be even larger - since ESS is not averaged and is simply the sum of squares between model value and mean. e.g. the 8654.7 above could be 9500 and the 84.6 (RSS/DF) above could be higher or lower - but will likely be still around 85 (larger sum of squares / 85). Wouldn't this imply that the significance is a function of the # of data points tested on the model (as opposed to real-world observed data points)? Related to this question, does the # of (x, y) real-world observed points used for RSS calculation (e.g. 77) must be the same as the # of points for the ESS calculation? e.g. can ESS be based on 95 model trials while RSS is only based on the 77 real world values?
2) I still don't understand (conceptually) why the F-test works to check whether the model parameters A and B are zero or not. Conceptually, how does this make sense?
Thanks very much everyone.