Testing regression model with F-test

AI Thread Summary
The discussion centers on the use of the F-test for assessing the significance of a regression model, specifically how the explained sum of squares (ESS) and residual sum of squares (RSS) are calculated and interpreted. The F-test value is derived from the ratio of ESS to its degrees of freedom and RSS to its degrees of freedom, with the participant questioning why the degrees of freedom for ESS is always "1." There is confusion regarding whether the number of data points used for ESS must match those used for RSS and how this affects the F-test value. Additionally, there is a conceptual inquiry into how the F-test determines the significance of the model parameters A and B being zero. The participant seeks further clarification on these statistical concepts.
Phoeniyx
Messages
16
Reaction score
1
Hey guys. I have some trouble understanding how the F-test is used for testing the viability of a regression model. Before I delve into the background/question, just wanted to post a link that discusses the topic briefly:
http://www.stat.yale.edu/Courses/1997-98/101/anovareg.htm

So, coming back to the question, let's say we have 77 data points (xi, yi) and we try to fit it to a regression model as:
\hat{y}_{i} = A + Bx_{i}

In the Yale example, A and B are calculated based on the 77 data points.

To check if the model is significant, we calculated the "explained sum of squares" (ESS) which is the squared difference between the model estimate and the mean: ESS = \Sigma{(\hat{y}_{i} - \bar{y})^{2}}

Then we calculate the "residual sum of squares" (RSS) which is the squared difference between the actual data point and the model: RSS = \Sigma{({y}_{i} - \hat{y}_{i})^{2}}

The degrees of freedom for RSS is: # of data points - estimated model parameters from data = 77 - 2 = 75. Perfectly fine with this.

BUT, apparently, the degrees of freedom for ESS is "1"... I do not get this. More on why I am confused later in the questions section.

The F-test value = \frac{ESS / DF_{ESS}}{RSS / DF_{RSS}}, where DF = degrees of freedom

In the Yale link above, this calculates to 8654.7/84.6 = 102.35

So, I have two questions:
1) Since the degrees of freedom of ESS is always "1", if I had 85 data points (instead of 77), the F-test value would be even larger - since ESS is not averaged and is simply the sum of squares between model value and mean. e.g. the 8654.7 above could be 9500 and the 84.6 (RSS/DF) above could be higher or lower - but will likely be still around 85 (larger sum of squares / 85). Wouldn't this imply that the significance is a function of the # of data points tested on the model (as opposed to real-world observed data points)? Related to this question, does the # of (x, y) real-world observed points used for RSS calculation (e.g. 77) must be the same as the # of points for the ESS calculation? e.g. can ESS be based on 95 model trials while RSS is only based on the 77 real world values?

2) I still don't understand (conceptually) why the F-test works to check whether the model parameters A and B are zero or not. Conceptually, how does this make sense?

Thanks very much everyone.
 
Physics news on Phys.org
Using the formulas for A and B, ESS is proportional to the variance of B, i.e. the variance of 1 degree of freedom.
 
Hi DrDu. I am sorry, but I am not understanding your response. Could you please elaborate a bit further? Thank you.
 
I was reading documentation about the soundness and completeness of logic formal systems. Consider the following $$\vdash_S \phi$$ where ##S## is the proof-system making part the formal system and ##\phi## is a wff (well formed formula) of the formal language. Note the blank on left of the turnstile symbol ##\vdash_S##, as far as I can tell it actually represents the empty set. So what does it mean ? I guess it actually means ##\phi## is a theorem of the formal system, i.e. there is a...

Similar threads

Replies
30
Views
4K
Replies
5
Views
2K
Replies
4
Views
2K
Replies
5
Views
2K
Back
Top