How to calculate confidence interval for a CDF curve

Click For Summary

Discussion Overview

The discussion revolves around calculating the 95% confidence interval for a cumulative distribution function (CDF) curve. Participants explore various methods and algorithms for deriving confidence intervals, particularly in the context of non-linear regression models.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • One participant describes the CDF in terms of the error function and seeks guidance on calculating confidence intervals for the curve.
  • Another participant suggests using polynomial regression and analyzing residuals to determine confidence bounds, mentioning the possibility of piecewise confidence intervals on the Y-axis.
  • Some participants express confusion regarding the algorithms used in statistical software like MATLAB and Minitab, specifically questioning the applicability of the formula C=b±t*sqrt(S) for their needs.
  • A later reply introduces a formula for confidence intervals based on maximum likelihood estimates, including a correction to a previously stated equation.
  • Participants share links to external resources for further reading, indicating a desire for more comprehensive examples and explanations.

Areas of Agreement / Disagreement

There is no consensus on a single method for calculating confidence intervals for the CDF curve, with multiple competing views and approaches presented. Participants express varying levels of understanding and seek clarification on specific algorithms.

Contextual Notes

Participants note limitations in their understanding of proprietary algorithms used in statistical software, as well as unresolved mathematical steps in their discussions.

ILEVEN
Messages
5
Reaction score
0
I got a question which has been confused me for a long time.

The question is to calculate the 95% confidence interval for a curve. I have already learned how to calculate for a straight line.

For example, the cumulative distribution function (CDF) could be expressed as below:
Y = 1/2 * {1 + erf [(X-mean) / (sd * 2^0.5)]}
where ‘erf ’ is called error function, ‘mean’ and ‘sd’ are the mean value and standard deviation of X, respectively. Y is distributed normally from 0 to 1.

If ‘mean’ and ‘sd’ are known, by varying the value of X we could obtain a series values of Y. Then you could plot a typical CDF graph.

Then I need to calculate the 95% confidence intervals of this plotted curve. Could someone tell me how to do it?

I know it could be completed using MATLAB, Minitab, etc. But I want to know the algorithm.

Thank you.
 
Physics news on Phys.org
Someone please help:cry:
 
ILEVEN said:
Someone please help:cry:

The most general non-linear regression model is the polynomial. If you find a reasonable fit you can use an analysis of residuals to determine confidence bounds. Of course you can also simply do piecewise point by point CIs on the Y axis and simply connect the dots of upper and lower "curves" if your data allows it. This gives you some idea of the consistency of data quality.

http://www.mathworks.com/help/toolbox/curvefit/bq_5ka6-1_1.html

If you are just curve fitting for CDFs or PDFs, most stat packages contain programs for this.
 
Last edited by a moderator:
SW VandeCarr said:
The most general non-linear regression model is the polynomial. If you find a reasonable fit you can use an analysis of residuals to determine confidence bounds. Of course you can also simply do piecewise point by point CIs on the Y axis and simply connect the dots of upper and lower "curves" if your data allows it. This gives you some idea of the consistency of data quality.

http://www.mathworks.com/help/toolbox/curvefit/bq_5ka6-1_1.html

If you are just curve fitting for CDFs or PDFs, most stat packages contain programs for this.

thank you.

I have read the information on mathworks but it seems I still can not figure out what algorithm they used. I think only use C=b+-t*sqrt(S) can not solve the problem. Or I might not fully understand this.

I know I can simplely use MATLAB or minitap, etc to analyze such statistics problem, but I need to understand how it works?

Could you give me a example of it, please?

Thank you!
 
Last edited by a moderator:
ILEVEN said:
thank you.

I have read the information on mathworks but it seems I still can not figure out what algorithm they used. I think only use C=b+-t*sqrt(S) can not solve the problem. Or I might not fully understand this.

I know I can simplely use MATLAB or minitap, etc to analyze such statistics problem, but I need to understand how it works?

Could you give me a example of it, please?

Thank you!

I don't know the proprietary algorithms they use but for unspecified non-linear regressions, it's probably an iterative ML estimate.

[tex]CI= [\hat\theta-2SE, \hat\theta+2SE][/tex]

[tex]SE=\frac{1}{\sqrt{nI_{X_i}(\hat\theta)}}[/tex]

[tex]I_X(\theta)={E(\theta)-\frac{\delta^2(lnp(X(\theta))}{\delta\theta^2}[/tex]

http://learning.eng.cam.ac.uk/zoubin/SALD/week3b.pdf
 
Last edited:
SW VandeCarr said:
I don't know the proprietary algorithms they use but for unspecified non-linear regressions, it's probably an iterative ML estimate.

[tex]CI= [\hat\theta-2SE, \hat\theta+2SE][/tex]

[tex]SE=\frac{1}{\sqrt{nI_{X_i}(\hat\theta)}}[/tex]

[tex]I_X(\theta)={E(\theta)-\frac{\delta^2(lnp(X(\theta))}{\delta\theta^2}[/tex]

http://learning.eng.cam.ac.uk/zoubin/SALD/week3b.pdf

Correction to the third equation above:

[tex]I_X(\theta)=E_\theta\frac{(-\delta^2(ln p(X|\theta))}{\delta\theta^2}[/tex]
 
SW VandeCarr said:
Correction to the third equation above:

[tex]I_X(\theta)=E_\theta\frac{(-\delta^2(ln p(X|\theta))}{\delta\theta^2}[/tex]

Thank you.
 

Similar threads

Replies
6
Views
3K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 22 ·
Replies
22
Views
4K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 6 ·
Replies
6
Views
4K
Replies
4
Views
2K