Uncertainty for p = 0 for binomial distribution?

[apsiloritis]

I have some data (4 runs each of about 10 trials) which is binomial with n_hits/N_trials

n/N = 0/11, 0/9, 0/10, 0/10

So, I estimate the probability p = n/N = 0
But how can I calculate an uncertainty on this value?

I thought to try
total N_tot=40 and n_tot=1, so p_tot=1/40 = 0.025
(i.e., assume one of the trials happened to be successful instead of not)

Then s = sqrt[ 0.025 * (1-0.025)/40] = 0.0247 (approx. 1/40)

is this a valid way of doing this? Or should I be looking at some confidence test?

Hope someone can help! Thanks.

 

[SW VandeCarr]

I have some data (4 runs each of about 10 trials) which is binomial with n_hits/N_trials

n/N = 0/11, 0/9, 0/10, 0/10

So, I estimate the probability p = n/N = 0
But how can I calculate an uncertainty on this value?

I thought to try
total N_tot=40 and n_tot=1, so p_tot=1/40 = 0.025
(i.e., assume one of the trials happened to be successful instead of not)

Then s = sqrt[ 0.025 * (1-0.025)/40] = 0.0247 (approx. 1/40)

is this a valid way of doing this? Or should I be looking at some confidence test?

If you're asking what the probability of x=0 is given a binomial distribution, the calculation (using the binomial PMF) reduces to $$(1-p)^n$$. The variance is np(1-p). Using the square root of the variance, you can construct the right confidence interval using a Z score value if the distribution is reasonably symmetric. Otherwise you need to do a Poisson approximation where np=$$\lambda$$ and the sd estimate is $$\sqrt\lambda$$

 

[apsiloritis]

If you're asking what the probability of x=0 is given a binomial distribution, the calculation (using the binomial PMF) reduces to $$(1-p)^n$$. The variance is np(1-p). Using the square root of the variance, you can construct the right confidence interval using a Z score value if the distribution is reasonably symmetric. Otherwise you need to do a Poisson approximation where np=$$\lambda$$ and the sd estimate is $$\sqrt\lambda$$

Thanks for your reply: I've been mulling over your suggestions.

One problem I have is that I don't know what the population p is, I estimate p = 0 from the results. I know my distribution is totally asymmetric: I know that p$$\geq$$0. The poisson approximation is asymmetric, so I see how that could work.

But... I have only used t-test for comparing normal distributions, so not sure how to apply it properly here. I'm hoping to say something like "My estimated value of p is 0, and I am confident (to say 90%) that the true value of p is less than Y".

 

[apsiloritis]

OK, how about this... I'd be very grateful if someone could check this.

p is the true (population) probability. Given that, with binomial, prob. that I measure k = 0 from N trials is $$(1-p)^N$$.

We know $$0\leq p\leq 1$$. So an experimental measurement of p_exp = 0 could have come from any value of p in that range.

So I know $$\int_0^1 (1-p)^N dp=1=1/(N+1)$$

Then I need to solve $$(N+1)\int_0^a (1-p)^N dp=b$$ for a, where b is the confidence I choose, say b = 0.9.

For N = 50, I find a = 0.044, so I'm 90% confident $$p\leq 0.044$$

and clearly we check for b = 1, a = 1, i.e., must be 100% confident that $$p\leq 1$$.

 

[SW VandeCarr]

So I know $$\int_0^1 (1-p)^N dp=1=1/(N+1)$$

That's wrong.$$P(x=0)=(1-p)^N$$

For P=1 you must sum over the full PMF, not the special case where P(x)=0. That's just a point for which the integral is 0. You have to choose a non zero value for x (or k since you working with a discrete distribution). Since you are actually looking for the expectation to be very close to zero, I would use the Poisson approximation to the binomial.

 

[apsiloritis]

That's wrong.$$P(x=0)=(1-p)^N$$

For P=1 you must sum over the full PMF, not the special case where P(x)=0. That's just a point for which the integral is 0. You have to choose a non zero value for x (or k since you working with a discrete distribution). Since you are actually looking for the expectation to be very close to zero, I would use the Poisson approximation to the binomial.

Sorry, I meant to write $$\int_0^1 (1-p)^N dp = 1/(N+1)$$, where N is fixed (total number of trials). The integrand looks just like an exp. decay exp(-Np), i.e., Poisson function, as you pointed out.

I can treat the integrand like a density function, and want to find the value of p = a, which accounts for 90% of the value of this integral. This is equivalent to saying that if I do measure p_exp = 0, then there's a 90% chance that it happened because the true value of $$p \leq a$$.

So I renormalize the function in the integrand to $$(N+1)(1-p)^N$$, and solve the equation $$\int_0^a (N+1)(1-p)^N dp = 0.9$$ to find a.

The method actually gives a value very close to the original approx. solution in my first post. For single s.d., confidence=68% and for N=40, I calculate a = 0.0274 (answer above was 0.0247).

What do you think?

 

[SW VandeCarr]

Sorry, I meant to write $$\int_0^1 (1-p)^N dp = 1/(N+1)$$, where N is fixed (total number of trials). The integrand looks just like an exp. decay exp(-Np), i.e., Poisson function, as you pointed out.

I can treat the integrand like a density function, and want to find the value of p = a, which accounts for 90% of the value of this integral. This is equivalent to saying that if I do measure p_exp = 0, then there's a 90% chance that it happened because the true value of $$p \leq a$$.

So I renormalize the function in the integrand to $$(N+1)(1-p)^N$$, and solve the equation $$\int_0^a (N+1)(1-p)^N dp = 0.9$$ to find a.

The method actually gives a value very close to the original approx. solution in my first post. For single s.d., confidence=68% and for N=40, I calculate a = 0.0274 (answer above was 0.0247).

What do you think?

If the probability of 0 is 0.9, then P(>0) is 0.1. So are you saying that lambda = a = 0.0247? In that case, it makes sense since lambda will be small, but I can't say whether or not it's correct.

 

[apsiloritis]

For those that may be interested, the solution to this problem is to use the Wilson interval. Details are available in the following paper. This interval can be calculated for any value of n and k, including for k = 0 (which was what I was interested in).

http://projecteuclid.org/euclid.ss/1009213286"

Note that the textbook interval $$\sqrt{np(1-p)}$$ is called the Wald interval and is generally acknowledged to be the wrong thing to use for any binomial problem!

The paper also discusses other solutions, and it is worthwhile looking at the paper by Agresti and Coull for a readable discussion of these.

Cheers,
A

 

[bpet]

I have some data (4 runs each of about 10 trials) which is binomial with n_hits/N_trials

n/N = 0/11, 0/9, 0/10, 0/10

So, I estimate the probability p = n/N = 0
But how can I calculate an uncertainty on this value?

I thought to try
total N_tot=40 and n_tot=1, so p_tot=1/40 = 0.025
(i.e., assume one of the trials happened to be successful instead of not)

Then s = sqrt[ 0.025 * (1-0.025)/40] = 0.0247 (approx. 1/40)

is this a valid way of doing this? Or should I be looking at some confidence test?

Hope someone can help! Thanks.

The 95% confidence would be [0,p1] where p1 is the solution to (1-p1)^40=0.95.