# A Residuals summing to zero?

1. May 15, 2017

### FallenApple

Ok so say I'm comparing two groups. I can do it this way $Y_{i}=b_{1}*I(G1)+b_{2}*I(G2)+e_{i}$ where I(G1) is 1 if in group 1 and 0 if not. I(G2) is 1 if in group 2 and 0 if not. In that case, my design matrix will not have a column of ones.

However, if I reparameterise to $Y_{i}=b_{0}+b_{2}*I(G2)+e_{i}$ since I know I(G2) and I(G1) has to sum to 1. I will get a design matrix with ones in the first column. I think there is a theorem that says that the residuals sum to 0 if this is the case.

Now, does this mean that the residuals sum to zero for the first parameterization as well? After all, the two models should be equivalent.

2. May 16, 2017

### andrewkirk

The two models are equivalent, since $G1=1-G2$, $b_0=b_1$ and $b'_2=b_2-b_1$ where $b'_2$ is the coefficient of $I(G_2)$ in the second model.

Given the models are equivalent, I imagine that the parameter estimates will be equivalent. Conceivably that may differ by estimation method. The method for OLS is Maximum Likelihood and I'm pretty sure that would give identical estimates, but one would need to work through the equations for the estimates, substituting the equivalences in the preceding paragraph, to be sure.

If the parameter estimates are equivalent then the residuals will be identical since the linear estimators will be identical, so if the residuals sum to zero for the first model they will do that for the second as well.

3. May 16, 2017

### FactChecker

If you are minimizing the sum-squared-errors for your parameter estimates, I don't think that the residuals have to sum to zero.

4. May 16, 2017

### andrewkirk

I agree with FactChecker. I am not aware of any theorem about residuals summing to zero. If one is using Maximum-Likelihood estimation of the coefficients (ie the usual, simplest way) then the sum of the products of residuals $\varepsilon_i$ with regressors $I(G2_i)$ will be zero, that is, $\sum_i \varepsilon_i I(G2_i)=0$. Could that be the theorem you had in mind?

5. May 24, 2017

### FallenApple

I thought that if there is an intercept or that there could be a transformation to the intercept, the residuals sum to 0.