Weighted least squares residuals

[vibe3]

Hello,

When doing a weighted least squares fit of a model to data, I want to examine the residuals to see if their histogram matches the expected probability distribution. Since I am minimizing
$$\chi^2 = \sum_i w_i \left[ y_i - Y(x_i) \right]^2$$
would I define my (normalized/studentized) residuals as
$$r_i = \frac{\sqrt{w_i} ( y_i - Y(x_i) )}{\sigma \sqrt{1 - h_i}}$$
or in the usual way as
$$r_i = \frac{y_i - Y(x_i)}{\sigma \sqrt{1 - h_i}}$$
where $h_i$ are the statistical leverages and $\sigma$ is the residual standard deviation. I am using a robust (iterative) least squares procedure to determine the weights $w_i$ in order to detect and eliminate outliers, so the residual histogram should match the expected distribution of the M-estimator function I'm using (Huber in my case).

 

[Stephen Tashi]

I don't know what a "statistical leverage" is.

If the weights are just something used in the fitting process and are not parameters of the predictive model then the weights are not involved in the calculation of an individual "residual" of the model, by the usual definition of "residual". You'll get better advice if you state the problem precisely. If you are using an example of M-estimation as a guide, can you give a link to it?

 

[MarneMath]

It looks like to me that you're using a bisquare weight method. You will want to use the normal definition of the residual and with adjusted residual defined as $$r_{adj} = \frac{r_{i}}{\sqrt{1-h_i}}$$. Where $$r_{i}$$ is your residual as defined normally and $$h_{i}$$ is your leverage. Matlab or SAS will produce this as your weighted least squares as the default.