Confused about Weighted Least Squares

[tom8]

I am trying to use Weighted Least Squares with a linear model: Y = Xβ + ε, where Y are some observed measurements and β is a vector of estimates. For example, in this case β has two terms: intercept and slope.

The weighted least squares solution, as shown here, involves a weight matrix, W, defined as a diagonal matrix whose elements are inverse of the variance of the measurements Y (here we assume the measurements are uncorrelated so the matrix is diagonal). But this mean that, if I just keep W as an identity matrix, then I am assuming a measurement with errors / variances equal to 1 unit. So if my measurements are in meters, then I am assuming 1 m of variance. So it is unclear to me how this matrix W is used to represent measurement error in real life.

 

[olivermsun]

tom8,
The variances would be in squared units. But in any case the goal is to minimize the (weighted) sum of the squared residuals. If you use equal weighting (so that W is the identity matrix), then you minimize the sum of the squared residuals. It doesn't matter so much what the units/scaling used for the weights are; the important thing is that the weights have the correct relative values (all equal weights in this case). Then, the same set of inputs will produce the same residual.
To give a very simple example, suppose you have a set of measurements of some value $x$ that come out to be $(1.0, 1.1, 0.9) ~\mathrm{m}$. Then your least squares estimate for the true value of $x$ should minimize:
$w_1(x-1.0)^2 + w_2(x-1.1)^2 + w_3(x-0.9)^2$.
Assuming that all three measurements are equally good (they are all unbiased and have the same variance) so that you choose equal weighting, it shouldn't change the answer whether $w_1 = w_2 = w_3 = 1~\mathrm{m}^2$ or $w_1 = w_2 = w_3 = ~10 \mathrm{m}^2$.

 

[BWV]

Curious what actual problem would make weighted least squares necessary be OLS