- #1

- 1

- 0

## Main Question or Discussion Point

Hi

I'm having some big problems with some data! I will try to keep this as simple as possible...

I have a random variable that admits a probability distribution that I have a fit function for. With a large enough number of samples I can get good estimates of the fit function parameters via a least-squares optimizer (minpack from scipy.optimize.leastsq I believe). The optimizer gives me a covariance matrix from which I extract approximate errors on the parameters (square root of the diagonal).

The problem is that the errors obtained by this method are too small, because If I fit a different set of data from the same distribution I get some other estimate for the fit parameters with an error which is also very small and these do not overlap. As a test, I fit ~10,000 different sets of the data (with ~10,000 samples in each) and saw that I get a nicely shaped gaussian for the fit parameters. By eye, the standard deviation is about 10 times larger than the error I get from the covariance matrix.

I have manually verified that the errors calculated from the covariance matrix correspond to a change in ~1 of the chi-squared for the fit.

Am I doing anything obviously wrong? Please save me!

Thank you

Alex

I'm having some big problems with some data! I will try to keep this as simple as possible...

I have a random variable that admits a probability distribution that I have a fit function for. With a large enough number of samples I can get good estimates of the fit function parameters via a least-squares optimizer (minpack from scipy.optimize.leastsq I believe). The optimizer gives me a covariance matrix from which I extract approximate errors on the parameters (square root of the diagonal).

The problem is that the errors obtained by this method are too small, because If I fit a different set of data from the same distribution I get some other estimate for the fit parameters with an error which is also very small and these do not overlap. As a test, I fit ~10,000 different sets of the data (with ~10,000 samples in each) and saw that I get a nicely shaped gaussian for the fit parameters. By eye, the standard deviation is about 10 times larger than the error I get from the covariance matrix.

I have manually verified that the errors calculated from the covariance matrix correspond to a change in ~1 of the chi-squared for the fit.

Am I doing anything obviously wrong? Please save me!

Thank you

Alex