Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

A Residuals summing to zero?

  1. May 15, 2017 #1
    Ok so say I'm comparing two groups. I can do it this way ##Y_{i}=b_{1}*I(G1)+b_{2}*I(G2)+e_{i}## where I(G1) is 1 if in group 1 and 0 if not. I(G2) is 1 if in group 2 and 0 if not. In that case, my design matrix will not have a column of ones.

    However, if I reparameterise to ##Y_{i}=b_{0}+b_{2}*I(G2)+e_{i}## since I know I(G2) and I(G1) has to sum to 1. I will get a design matrix with ones in the first column. I think there is a theorem that says that the residuals sum to 0 if this is the case.

    Now, does this mean that the residuals sum to zero for the first parameterization as well? After all, the two models should be equivalent.
     
  2. jcsd
  3. May 16, 2017 #2

    andrewkirk

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    The two models are equivalent, since ##G1=1-G2##, ##b_0=b_1## and ##b'_2=b_2-b_1## where ##b'_2## is the coefficient of ##I(G_2)## in the second model.

    Given the models are equivalent, I imagine that the parameter estimates will be equivalent. Conceivably that may differ by estimation method. The method for OLS is Maximum Likelihood and I'm pretty sure that would give identical estimates, but one would need to work through the equations for the estimates, substituting the equivalences in the preceding paragraph, to be sure.

    If the parameter estimates are equivalent then the residuals will be identical since the linear estimators will be identical, so if the residuals sum to zero for the first model they will do that for the second as well.
     
  4. May 16, 2017 #3

    FactChecker

    User Avatar
    Science Advisor
    Gold Member

    If you are minimizing the sum-squared-errors for your parameter estimates, I don't think that the residuals have to sum to zero.
     
  5. May 16, 2017 #4

    andrewkirk

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    I agree with FactChecker. I am not aware of any theorem about residuals summing to zero. If one is using Maximum-Likelihood estimation of the coefficients (ie the usual, simplest way) then the sum of the products of residuals ##\varepsilon_i## with regressors ##I(G2_i)## will be zero, that is, ##\sum_i \varepsilon_i I(G2_i)=0##. Could that be the theorem you had in mind?
     
  6. May 24, 2017 #5
    I thought that if there is an intercept or that there could be a transformation to the intercept, the residuals sum to 0.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Residuals summing to zero?
  1. PRESS Residuals (Replies: 1)

  2. Quadratic Residues (Replies: 3)

Loading...