Do Residuals Always Sum to Zero in Regression Analysis?

[FallenApple]

Ok so say I'm comparing two groups. I can do it this way $Y_{i}=b_{1}*I(G1)+b_{2}*I(G2)+e_{i}$ where I(G1) is 1 if in group 1 and 0 if not. I(G2) is 1 if in group 2 and 0 if not. In that case, my design matrix will not have a column of ones.

However, if I reparameterise to $Y_{i}=b_{0}+b_{2}*I(G2)+e_{i}$ since I know I(G2) and I(G1) has to sum to 1. I will get a design matrix with ones in the first column. I think there is a theorem that says that the residuals sum to 0 if this is the case.

Now, does this mean that the residuals sum to zero for the first parameterization as well? After all, the two models should be equivalent.

 

[andrewkirk]

The two models are equivalent, since $G1=1-G2$, $b_0=b_1$ and $b'_2=b_2-b_1$ where $b'_2$ is the coefficient of $I(G_2)$ in the second model.

Given the models are equivalent, I imagine that the parameter estimates will be equivalent. Conceivably that may differ by estimation method. The method for OLS is Maximum Likelihood and I'm pretty sure that would give identical estimates, but one would need to work through the equations for the estimates, substituting the equivalences in the preceding paragraph, to be sure.

If the parameter estimates are equivalent then the residuals will be identical since the linear estimators will be identical, so if the residuals sum to zero for the first model they will do that for the second as well.

 

[FactChecker]

If you are minimizing the sum-squared-errors for your parameter estimates, I don't think that the residuals have to sum to zero.

 

[andrewkirk]

I agree with FactChecker. I am not aware of any theorem about residuals summing to zero. If one is using Maximum-Likelihood estimation of the coefficients (ie the usual, simplest way) then the sum of the products of residuals $\varepsilon_i$ with regressors $I(G2_i)$ will be zero, that is, $\sum_i \varepsilon_i I(G2_i)=0$. Could that be the theorem you had in mind?

 

[FallenApple]

I agree with FactChecker. I am not aware of any theorem about residuals summing to zero. If one is using Maximum-Likelihood estimation of the coefficients (ie the usual, simplest way) then the sum of the products of residuals $\varepsilon_i$ with regressors $I(G2_i)$ will be zero, that is, $\sum_i \varepsilon_i I(G2_i)=0$. Could that be the theorem you had in mind?

I thought that if there is an intercept or that there could be a transformation to the intercept, the residuals sum to 0.