It hit the news in a last few days, you have probably heard about it. That's what I have found about the trial:

And it makes me wonder - what is statistical significance of this diference? What is a confidence level that those that were given vaccine were in fact more immune? Or in other words - what is a confidence level that probability of contracting AIDS was lower in those that received vaccine? (Sure, we have to assume both group were otherwise identical).

In theory I should know how to calculate it, but after 20 years of lying somewhere in the dark corner my statistics is so rusty I prefer not to touch it, as it may fall apart :grumpy:

If a group consists of N total individuals, n of whom were infected, then each individual within that group can be modeled as a Bernoulli trial with mean p = n/N and variance = p(1-p). A Bernoulli trial is an individual "experiment" with probability p of getting infected.

N trials are distributed Binomial with mean = Np = n and variance = Np(1-p) for the group as a whole.

Control group (m = 74, M = 8198):
...For an individual:
......Bernoulli probability: p = m/M = 0.00902659
...For the group:
......Binomial mean: m = 74
......Binomial variance: u = Mp(1-p) = 73.332.
Experiment group (n = 51, N = 8197)
...For an individual:
......Bernoulli probability: q = n/N = 0.00622179
...For the group:
......Binomial mean: n = 51
......Binomial variance: v = Nq(1-q) = 50.6827.

Unequal variances t-test:
...s = Sqrt[(Nu+Mv)/(MN)] = 0.122997
...t = (m - n)/s = 186.997
...d.f. = (u/M + v/N)^2 / ((u/M)^2/(M - 1) + (v/N)^2/(N - 1)) = 15,864.5
...The null hypothesis "true m = true n" can be rejected at a very high degree of statistical significance.

Equal variance t-test:
...S = Sqrt[((M-1)u+(N-1)v)/(M+N-2)] = 7.8745
...s = S Sqrt[1/M + 1/N] = 0.122998
...t = (m - n)/s = 186.995
...d.f. = M + N - 2 = 16,393
...Again, the null hypothesis "true m = true n" can be rejected at a very high degree of statistical significance.