Calculating standard deviation from a confidence interval?

[pergradus]

Hello, I'm doing some nonlinear regression fitting, and I have a set of data with n data points and I am fitting the data to a model which contains 3 parameters.

I can get a confidence interval from the program I'm using for each parameter, but how can I calculate the standard deviation for each parameter using these confidence intervals?

Thanks!

 

[Stephen Tashi]

Whatever your program is doing is not something I recognize as a standard statistical method. If you have documentation for the program, see what assumptions the program is making and how it defines a "confidence interval".

If you can't figure that out, name the specific program you are using or post a link to its documentation.

 

[pergradus]

Thanks for the reply! I'm using MatLab's fit function:

http://www.mathworks.com/help/toolbox/curvefit/fit.html

to fit the data, and then using this function:

http://www.mathworks.com/help/toolbox/curvefit/confint.html

to produce the confidence intervals. Does that help?

 

[Stephen Tashi]

Those links somewhat help if I read between the lines of the documentation. I think what confit does is calculate "asymptotic linearized confidence intervals", not ordinary confidence intervals. ( I'm not an expert on asymptotic linearized confidence intervals and I only know about them because of the concurrent thread: https://www.physicsforums.com/showthread.php?t=565490)

When you called confit, you specified some "confidence" value, such as 0.95. If the numerical endpoints of your confidence interval are [A,B] then the program probably takes (A+B)/2 to be the mean value of the parameter and plus or minus (B-A)/2 to be plus or minus a certain number of standard deviations of the parameter.

By looking at a table of the standard (i.e. mean= zero, standard deviation= 1) normal distribution, you can find the value r such that there is a 95% chance that a realization of the random variable is within plus or minus r standard deviations of its mean.

Let r \sigma = (B-A)/2 and solve for \sigma.

 

[pergradus]

Thank you for your help so far! I think I'm starting to understand this stuff a little more, but it's still very abstract to me.

So, a table like the one near the bottom of the page is what you're referring to?

http://mathworld.wolfram.com/ConfidenceInterval.html

 

[Stephen Tashi]

Yes, use a table like that. It says a "confidence" of 0.95 applies to an interval centered at the mean and spanning plus or minus 1.95996 sigma

 

[pergradus]

Hi Stephen, so I tried using that table and here's the results I got.

StarName P_max P_err l_max l_err K/B K/B_err
80% Confidence
HD141657 1.828 0.045 0.584 0.021 0.888 0.122
HD150193 5.117 0.149 0.658 0.028 0.944 0.109
HD60325 1.149 0.037 0.583 0.022 1.141 0.133
90% Confidence
HD141657 1.828 0.049 0.584 0.023 0.888 0.132
HD150193 5.117 0.159 0.658 0.030 0.944 0.116
HD60325 1.149 0.040 0.583 0.023 1.141 0.141
95% Confidence
HD141657 1.828 0.054 0.584 0.025 0.888 0.144
HD150193 5.117 0.170 0.658 0.032 0.944 0.124
HD60325 1.149 0.043 0.583 0.025 1.141 0.151

P_max, l_max and K/B are the parameters I'm calculating, and P_err, l_err and K/B_err are the std I found using that table and the method you described. It seems as I increase the confidence level, I get larger std, but shouldn't the std be independent of the CI?

Do you make anything of this?

 

[Stephen Tashi]

The confidence interval produced by the program that you mention in the original post should change in size as you ask it for different levels of confidence.

What your figures in the previous post show are keeping the size of the confidence interval the same while changing the level of confidence that it supposedly gives. If you do that sort of calculation, the larger confidence we associate with a fixed interval, the more standard deviations we assume are in it, so the smaller the implied standard deviation must be.

 

[pergradus]

I see, sorry to keep bugging you and I understand if you don't wish to reply.

One thing that's a little confusing to me is the meaning of "standard deviation" in this context (I know that's strange considering that's what I'm trying to calculate...). std is normally a measure of how much spread there is in the data, correct? In this case however, I have one set of data, and one set of calculated parameters - so what exactly is the meaning of a standard deviation of these parameters in this context?

 

[Stephen Tashi]

That's a good question and I can't give you an authoritative answer!

Thinking about this clearly requires using certain terminology precisely. My thoughts on the subject:

When we have a random variable with a normal probability distribution then the standard deviation of that distribution is indeed a parameter that measures the spread of the distribution and you can calculate (or use tables) to find the probability that a random draw from that distribution is within plus or minus so many standard deviations of the mean value.

When you have a different distribution you can usually do the same sort of thing, but you must use different calculations and tables. In other words, the tables for the normal distribution are not "universal"; they don't apply to all probability distributions. However, the normal distribution is the most commonly encountered and many not-quite-normal distributions can be well approximated by the normal distribution.

When we have a sample of data, this is not a probability distribution. We usually assume it came from a probability distribution and we use some calculation of the sample values to estimate the parameters of the distribution that we assumed. So, in your situation, the errors of the residuals are involved in the calculations.

As to terminology: In your problem, the quantity e^2/(n-1) is an "estimator" of the standard deviation of the distribution of errors. Some people define "the standard deviation of the sample" to be the quantity e^2/(n-1) and some people define it to be e^2/n. It's a rather arbitrary decision. However, the "estimator of the population standard deviation" is not such an arbitrary choice. It is almost always taken to be e^2/(n-1) since one can prove that this formula has desirable properties.

In summary, we have at least 3 different sorts of things involved in statistics:
A) Probability distributions and their parameters, such as their standard deviations and means
B) Samples and their parameters such as their standard deviations and means
C) Estimators , which are formulas or algorithms that estimate the parameters of distributions as functions of the data in samples.

So when you say "standard deviation", it is an ambiguous phrase until you specify whether you mean a parameter of a probability distribution, a parameter of a sample, or an estimate of the parameter of the distribution based on the data in a sample.

As I understand the method in the paper you linked, it doesn't worry much about the distinction between the value of the estimated standard deviation and the actual standard deviation of the probability distribution. It assumes that the estimated value is close enough to the actual value.

----

Your (and my ) big conceptual problem is that in the curve fitting scenario the data is assumed to be randomly generated ( to the extent that it has random deviations from a curve that gives the mean value of the data), but the output of the program talks about the parameters of the curve as if they were somehow random. It does calculations that are based on the parameters of the curve having a standard deviation, in the sense of the standard deviation of a probability distribution.

I haven't studied the details of what is going on. My impression is that the scenario is as a follows. We use the data to estimate the parameters of a probability distribution for the data. We assume this is the true distribution of the data. We imagine that many different samples (each consisting of n data points) are drawn from this distribution and that the curve fitting process produces a slightly different curve fit on different samples. So there is a distribution of different parameters for the curves. This way, the parameters of the curves do have random variation.

If a computer program carried out the above calculation exactly it would:
1) Use the data to estimate a probability disbribution for the data
2) Use the estimated probability distribution to compute the distribution of parameters that would occur when we fit curves to random batches of n data points.
3) Use the distributions of the parameters to state confidence intervals for the parameters.

As typical nonlinear curve fitting problems work, I think they make some simplifying assumptions. In spite of the fact that they are "nonlinear" curve fits, I think that they do calculations that assume the curve is approximated by a linear function. I don't know the technical details of that yet. They also assume that the distributions involved are normal distribiutions (which may be true "asymptotically" as we use larger and larger sample sizes). This is why I think the confidence intervals for the parameters are "asymptotic linear confidence intervals" rather than ordinary confidence intervals.

From browsing the web, it appears that the use of nonlinear curve fitting software is becoming more and more common, so the question of what these programs produce is very topical. I intend to study it more.

 

[pergradus]

Thanks for the help! I think what I'll do is just use a confidence level corresponding to 1sigma and call that my error, with the warning that it's only an approximation.

By the way, I came across this paper while I was trying to research this myself, don't know if you'd find it interesting or not.

http://adsabs.harvard.edu/abs/1990AmJPh..58..160B