Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Weighted least squares residuals

  1. Apr 18, 2014 #1

    When doing a weighted least squares fit of a model to data, I want to examine the residuals to see if their histogram matches the expected probability distribution. Since I am minimizing
    \chi^2 = \sum_i w_i \left[ y_i - Y(x_i) \right]^2
    would I define my (normalized/studentized) residuals as
    r_i = \frac{\sqrt{w_i} ( y_i - Y(x_i) )}{\sigma \sqrt{1 - h_i}}
    or in the usual way as
    r_i = \frac{y_i - Y(x_i)}{\sigma \sqrt{1 - h_i}}
    where [itex]h_i[/itex] are the statistical leverages and [itex]\sigma[/itex] is the residual standard deviation. I am using a robust (iterative) least squares procedure to determine the weights [itex]w_i[/itex] in order to detect and eliminate outliers, so the residual histogram should match the expected distribution of the M-estimator function I'm using (Huber in my case).
  2. jcsd
  3. Apr 20, 2014 #2

    Stephen Tashi

    User Avatar
    Science Advisor

    I don't know what a "statistical leverage" is.

    If the weights are just something used in the fitting process and are not parameters of the predictive model then the weights are not involved in the calculation of an individual "residual" of the model, by the usual definition of "residual". You'll get better advice if you state the problem precisely. If you are using an example of M-estimation as a guide, can you give a link to it?
  4. Apr 21, 2014 #3


    User Avatar
    Education Advisor

    It looks like to me that you're using a bisquare weight method. You will want to use the normal definition of the residual and with adjusted residual defined as [tex]r_{adj} = \frac{r_{i}}{\sqrt{1-h_i}}[/tex]. Where [tex] r_{i} [/tex] is your residual as defined normally and [tex]h_{i}[/tex] is your leverage. Matlab or SAS will produce this as your weighted least squares as the default.
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook