Multiple linear regression: partial F-test

[kingwinner]

"Suppose that in a MULTIPLE linear regression analysis, it is of interest to compare a model with 3 independent variables to a model with the same response varaible and these same 3 independent variables plus 2 additional independent variables.
As more predictors are added to the model, the coefficient of multiple determination (R^2) will increase, so the model with 5 predicator variables will have a higher R^2.
The partial F-test for the coefficients of the 2 additional predictor variables (H_o: β_4=β_5=0) is equivalent to testing that the increase in R^2 is statistically signifcant."


I don't understand the bolded sentence. Why are they equivalent?

Thanks for explaining!

 

[statdad]

Mathematically $$R^2$$ will increase whether or not the new variables contribute to the model. Because of this, the question in practice is whether the larger $$R^2$$ is due simply to the math (this corresponds to $$H_0 \colon \beta_4 = \beta_5 = 0$$) or whether the increase is due to at least one of the two coefficients is non-zero (this would be the alternative hypothesis that at least one of the two coefficients is non-zero). If $$H_0$$ is rejected, we know at least one coefficient is non-zero, and we also know that the increase in $$R^2$$ is due to something other than mere chance.

Does this help, or were you looking for a mathematical explanation?

 

[kingwinner]

Do you have a mathematical explanation for that?

The statement claims that the test of H_o: β_4 = β_5 = 0 is equivalent to testing that the increase in R^2 is statistically signifcant. What would be the equivalent null and alternative hypotheses in terms of R^2?

Thanks!

 

[statdad]

Suppose you have a total of five variables (since you reference $$\beta_4, \beta_5$$

We want to test

$$\begin{align*} H_0 \colon & \beta_4 = \beta_5 = 0 \\ H_a \colon & \text{At least one of } \beta_4, \beta_5 \ne 0 \end{align*}$$

The test begins with the fitting of a full and a reduced model:

$$Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \beta_4 x_4 + \beta_5 x_5 \tag{Full}$$

$$Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 \tag{Reduced}$$

Denote the sum of squares for error in the full model by $$SSE(F) = SSE(x_1, x_2, x_3, x_4, x_5)$$, and the sum of squares for error in the reduced model by $$SSE(R) = SSE(x_1, x_2, x_3)$$

Since we use more variables in the full model than in the reduced model, we will see $$SSE(F) < SSE(R)$$. The test statistic for the above hypotheses are

$$F = \frac{SSE(R) - SSE(F)}{(n-4) - (n-6)} \div \frac{SSE(F)}{n-6}$$

In the old days (to be read as "when statdad was in school") the numerator of this statistic was written as

$$SSE(R) - SSE(F) = SSE(X_1, X_2, X_3) - SSE(X_1, X_2, X_3, X_4, X_5) = SSR(X_4, X_5 \mid X_1, X_2, X_3)$$

Think of the last notation ("sum of squares R eduction") as denoting the reduction in variation from adding $$x_4, x_5$$ to a model that already contains the other three variables. The test is done by comparing F to the appropriate tables.

How is this related to $$R^2$$? It isn't, directly, it is related to something called a coefficient of partial determination . The first bit of notation is this:

$$r^2_{Y45.123}$$

In the subscript the numbers to the left of the "." are the dependent variable and the "number label" of the variables being added to the model, while the numbers to the right of the "." are the "number labels" of the variables originally in the model. The coefficient of partial determination is calculated as

$$r^2_{Y45.123} = \frac{SSR(X_4, X_5 \mid X_1, X_2, X_3)}{SSE(X_1, X_2, X_3)}$$

Technically, this measures the percentage reduction in error sum of squares that results when we move from the model with 3 variables to the model with all 5 variables.

When the F-test referred to above is significant) ($$H_0$$ is rejected), this coefficient of partial determination indicates a significant [/tex] change in $$R^2$$

Hope this helped.

 

[kingwinner]

Thanks!

R^2 = regression SS/total SS

F = [(R^2_full - R^2_reduced) / (5 -3)] / [(1 - R^_full) / (n - 5 - 1)] .
where R^2_full is the R^2 with 5 independent variables and R^2_reduced is the R^2 with 3 independent variables

Based on this form of the F statistic, can we say that the partial F-test for the coefficients of the 2 additional predictor variables (H_o: β_4=β_5=0) is equivalent to testing that the increase in R^2 is statistically signifcant?

 

[statdad]

Yes - good job.