Computing probability that p₁ ≥ p₂ from binomial trial data

[TaPaKaH]

Suppose we have two series of independent Bernoulli experiments with unknown probabilities $p_1$ and $p_2$. The first series registers $x_1$ successes in $n_1$ trials while the second series registers $x_2$ successes in $n_2$ trials. Is there a way that we can compute probability $\mathbb{P}(p_1\geq p_2)$?

 

[RUber]

I have seen a two-sample test with null hypothesis $H_0: p_1 = p_2$ and alternative hypothesis $H_a: p_1>p_2$. This would call for a 1-tailed T test using pooled variance.
This question seems like a variation on that standard problem.

 

[WWGD]

Isnt this equivalent to the two distributions being equal? If the two ratios are equal, then E|X_1=X_2|=0 , right?

 

[Stephen Tashi]

Is there a way that we can compute probability $\mathbb{P}(p_1\geq p_2)$?

No, not unless you have some assumption or knowledge about the "prior" probability of that happening.

If you make specific assumptions about the probabilities, you can can compute the probability of various aspects of the data being what was observed. However the assumption that $p_1 \geq p_2$ isn't specific enough to do that. I think you must at least assume that $p_1 = p_2$.

 

[gill1109]

Suppose we have two series of independent Bernoulli experiments with unknown probabilities $p_1$ and $p_2$. The first series registers $x_1$ successes in $n_1$ trials while the second series registers $x_2$ successes in $n_2$ trials. Is there a way that we can compute probability $\mathbb{P}(p_1\geq p_2)$?

Yes it is called Bayes' theorem. You must start by assuming some joint prior probability distribution of p_1 and p_2, representing your (lack of) knowledge about p_1 and p_2 before doing the experiment. Then you observe x_1 and x_2. Now you compute the probability distribution of p_1 and p_2 given x_1 and x_2 using the formula posterior density is proportional to prior density times likelihood. Normalise it to integrate to 1. Compute the probability that p_1 is greater than or equal to p_2.

Trouble is, the result will depend on your prior probability distribution of p_1 and p_2. According to the principles of Bayesian statistics, this prior distribution should represent your beliefs about p_1 and p_2 before doing the experiment.

The likelihood for p_1 and p_2 is proportional to lik(p_1, p_2) = p_1^{x_1} (1 - p_1}^{n_1 - x_1} . p_2^{x_2} (1 - p_2}^{n_2 - x_2}. So if for instance you start with uniform independent priors for p_1 and p_2, the answer is the integral of the likelihood over all p_1 greater than or equal to p_2, divided by the integral of the likelihood over all p_1 and p_2.

This results in an exercise concerning two independent beta distributed random variables. See for instance http://stats.stackexchange.com/ques...lity-px-y-given-x-bea1-b1-and-y-bea2-b2-and-x