Calculating the confidence interval of your data

[tsaitea]

Hello guys,

I would like to calculate the confidence interval of the data in which the data is correct. In otherwords I would like to know how much confidence we have that the data is correct.

Could anyone direct me where to start?

Thank you.

 

[Simon Bridge]

Google for "confidence integral" is a good place to start.
Note: you decide how much confidence you want i.e. 95% - the confidence interval the the range within which you can have that confidence.

 

[tsaitea]

Thank you for your reply Simon.

That definitely helped me understand the confidence interval. I am struggling to put it into context now. Say for example I am loading 300 lines of data into a database. Now I want to figure out the # of errors that would occur during the load (data not loaded properly).

What I was thinking is maybe I would have to perform the load, and calculate the sample size I would need based on a 95% confidence interval and randomly check each line until I have checked up to the sample size. And based on the # of errors found, I could determine the confidence interval?

 

[DrDu]

Let me try to repeat your problem. You maybe want to load millions of lines into a database and what to estimate what percentage of the lines is corrupt. You decide to check at random say N=250 lines. Let's assume that the distribution of corrupt lines follows a poisson distribution and you find that n=25 lines (i.e. 10% ) are corrupt, hence your estimate of the probability or fraction of corrupted lines is p=n/N=0.1. The variance is also 25 for a poisson distribution and the standard deviation $\sigma=\sqrt{n}$. Using a normal approximation you can construct a 95% confidence interval
for the true percentage as $[ (n- z \sigma)/N, (n+z \sigma)/N]$ where for a 95% interval z=2 (1.96 to be exact).
So your true value is in the range [15/250, 35/250].

 

[blue_raver22]

Hello,

Calculating the confidence interval of your data is an important step in determining the accuracy and reliability of your results. The confidence interval is a range of values that we are confident contains the true population parameter. To begin, you will need to determine the level of confidence you want to have in your results. This is typically expressed as a percentage, such as 95% or 99%.

Next, you will need to calculate the standard error of your data. This is a measure of the variability of your data and is used to determine the size of the confidence interval. The formula for standard error may vary depending on the type of data you have, but it is typically calculated using the standard deviation of your sample.

Once you have the standard error and the desired level of confidence, you can use a statistical calculator or a spreadsheet program to calculate the confidence interval. This will give you a range of values within which you can be confident that the true population parameter lies. It is important to note that the larger the sample size, the smaller the confidence interval will be, indicating a higher level of confidence in the data.

I hope this helps guide you in calculating the confidence interval for your data. If you need further assistance, I recommend consulting with a statistician or using online resources for step-by-step instructions. Good luck with your analysis!