Estimating the standard deviation

[Niles]

Hi

Say I have a linear problem Mp = d, where d is our data, p our parameters and M our "observation matrix" (see http://en.wikipedia.org/wiki/Inverse_problem#Linear_inverse_problems). So what we are dealing with is an overdetermined problem.

Now, I have read an example where we have a vector of data d whose standard deviation we don't know. Then we try and estimate it, and the estimate is given by

<br /> \sigma ^2 _{estimate} = \frac{1}{N}\sum\limits_i {\left( {d_i - \left( {Mp} \right)_i } \right)^2 } <br />

My question is: How can they estimate the standard deviation like this? Usually we would use the mean, but they use (Mp)_i, and I can't quite see why this yields an estimate.

 

[Stephen Tashi]

How can they estimate the standard deviation like this? Usually we would use the mean,

Suppose you are doing linear regression and the fitted equation is y = Ax + B. If you assume the random variable y_observed has mean Ax + B at each x value and also that the distribution of errors about the mean is the same at each x value, then you can estimate the standard deviation of y_observed by using the quantities ( y_observed - Ax)^2 since Ax is the mean value of y at that particular value of x.

The example may assume that the p vector is fitted well enough so that {(MP)}_i is the mean value of d_i and that the distribution of errors from the mean is identical for all i.

 

[Niles]

Ah, I see. Thanks for taking the time to help me!