Uncertainty in parameters -> Gauss-Newton

[Or Entity?]

Uncertainty in parameters --> Gauss-Newton

Hi guys!

I have a set of datapoints, and i´m about to use Gauss-Newton to fit a model function (Lorentzian) to these points. So we´re talking abut a nonlinear least squares regression.

How do I estimate error in function parameters given errors in data points?

Thanks.

 

[EnumaElish]

Simulation using pseudo-random number generation?

 

[Or Entity?]

Simulation using pseudo-random number generation?

Nothing I'm familiar with. Could you develop?

Isn´t there any method equivalent to that using the covariance matrix in weighted linear least squares?

 

[EnumaElish]

http://en.wikipedia.org/wiki/Resampling_(statistics)

Also see: http://en.wikipedia.org/wiki/Bootstrapping_(statistics)#Resampling_residuals . Instead of resampling, you could just generate fictitious residuals from an approximate distribution.

 

[Or Entity?]

Allright, have read it through. Thanks
One thing though, where does the errors from my original data come in?

 

[EnumaElish]

I will write vectors in bold, so for example y = {y(1), ..., y(i), ..., y(n)}.

In a proper bootstrap, you are just "mixing and matching" the errors you've already computed: y*(i) = y(i) + u(j), where j is "almost surely" different from i, u(.) are the computed errors, and y* is a convolution of y. This "remix" algorithm is repeated many times (say 100 times), so you have 100 different estimates of your model parameters coming from convoluted vectors y₁*, ..., y₁₀₀*.

In the approximate bootstrap (monte carlo?), you use the computed errors to derive the approximate "population distribution." Suppose the errors "look like" they are distributed normally with mean = 0 and standard deviation = s, i.e. u*(i) ~ N(0,s²). Then, you can use a pseudo-random generator to produce repeated draws from Normal(0,s²) and define y**(i) = y(i) + u*(i). Again, if you run this 100 times, you will have 100 parameter estimates from y₁**, ..., y₁₀₀**.

 

[Or Entity?]

Amazing.. I was experimenting with a a method just like that one when I saw your reply.
Interesting method that can be applied to any linear/nonlinear method.

By the way since gauss-Newton linearizes the problem in each iteration I should get pretty decent results by making a weighted regression and taking the covariance matrix in the last step. (Like in linear least squares)