Covid Vaccine Phase 3 significance

  • Thread starter Grinkle
  • Start date
  • #1
Grinkle
Gold Member
680
187

Summary:

How much risk is one taking if one is a phase 3 participant?
Russia's vaccine news has me pondering.

How often after a successful phase 2 study do phase 3 studies show that the vaccine causes harm?

If I were offered a COVID-19 vaccine that had passed phase 2 trials, I'd accept it. Is that foolhardy of me?

Is it possible to use existing data to model the expected deaths of waiting for phase 3 completion vs skipping phase 3? I am thinking there is risk of ineffectiveness and over-confidence causing more infection, risk of overt harm from the vaccine on the one hand. On the other hand there are the deaths that will occur that wouldn't if the vaccine is effective and introduced early.
 

Answers and Replies

  • #2
Ygggdrasil
Science Advisor
Insights Author
Gold Member
2019 Award
3,083
2,891
Summary:: How much risk is one taking if one is a phase 3 participant?

Russia's vaccine news has me pondering.

How often after a successful phase 2 study do phase 3 studies show that the vaccine causes harm?

If I were offered a COVID-19 vaccine that had passed phase 2 trials, I'd accept it. Is that foolhardy of me?

Is it possible to use existing data to model the expected deaths of waiting for phase 3 completion vs skipping phase 3? I am thinking there is risk of ineffectiveness and over-confidence causing more infection, risk of overt harm from the vaccine on the one hand. On the other hand there are the deaths that will occur that wouldn't if the vaccine is effective and introduced early.
Overall (for all pharmaceutical drug development), around 50% of drugs that enter phase III trials eventually go on to be approved by the FDA. Reasons for failing to reach approval, however, are not just safety related, and the failure could reflect a lack of efficacy or a level of efficacy below existing standards of care. This figure covers drugs for all conditions, which may not reflect the risks for vaccine development.

For a back of the envelope calculation on safety risks for skipping phase III trials, assume a phase II trial with 100 individuals receiving the vaccine (this seems to be the numbers for Moderna's phase II trials). Even if you observe no serious adverse events in the 100 individuals who were doesed, there is a >80% chance that you would miss any serious adverse events that occur in <0.2% of people. Now, for individual risk this seems small, a two in a thousand chance of a serious event. However, at a population level, this is quite significant. If you dose all 300 million people in the US with a vaccine that causes serious adverse events at a rate of 0.2%, you have just caused 600 thousand adverse events (~4x the number of current coronavirus deaths in the US). Even if the vaccine is effective (not a guarantee if you have skipped phase III trials), you may have caused a larger problem than you have solved.

Note that the 1976 Swine Flu vaccine is widely seen to be a huge failure because it caused Guillain-Barré syndrome (a serious neurological condition) at a rate of about 1 per 100,000 vaccinated.
 
  • Like
  • Informative
Likes Buzz Bloom, bhobba, Grinkle and 2 others
  • #3
Vanadium 50
Staff Emeritus
Science Advisor
Education Advisor
2019 Award
25,182
8,305
Even if you observe no serious adverse events in the 100 individuals who were doesed, there is a >80% chance that you would miss any serious adverse events that occur in <0.2% of people.
Where does that come from? If I ask what Poisson mean I need so I get zero 20% of the time, it's 1.61. That is, given an observation of zero, I am 80% confident that the true mean is less than 1.61%. Are we calculating the same thing?

That means the 600K adverse effects becomes almost 5M.
 
  • Like
Likes bhobba and Ygggdrasil
  • #4
Ygggdrasil
Science Advisor
Insights Author
Gold Member
2019 Award
3,083
2,891
Where does that come from? If I ask what Poisson mean I need so I get zero 20% of the time, it's 1.61. That is, given an observation of zero, I am 80% confident that the true mean is less than 1.61%. Are we calculating the same thing?

That means the 600K adverse effects becomes almost 5M.
Given an adverse event probability p with N individuals, the probability of observing no adverse events is ##(1-p)^N##, so for my calculation, I found the value of p for N=100 where you would have an 80% chance of observing no adverse events. This may not be the best way of calculating the statistical power of the trial (it was only meant to give a back of the envelope estimate), so it may be better to defer to someone like @StatGuy2000 with training in biostatistics.

However, I agree with your calculations that dosing 100 individuals would give a >80% chance of catching at least one instance of any adverse reaction that occurs with a probability of >1.6%, and that's probably a better bound to consider here.
 
Last edited:
  • Like
Likes bhobba
  • #5
atyy
Science Advisor
14,003
2,285
Is the difference due to binomial and Poisson models?
 
  • #6
Summary:: How much risk is one taking if one is a phase 3 participant?

Is that foolhardy of me?
I read the forum rules and believe answering this question directly violates the rules. I think that should give you the answer.
 
  • #7
Ygggdrasil
Science Advisor
Insights Author
Gold Member
2019 Award
3,083
2,891
Is the difference due to binomial and Poisson models?
The difference is that we're calculating two different things. He's calculating the rate of adverse events that a 100 person trial that would be found with 80% confidence. I calculated the rate of adverse events that a 100 person trial that would be missed with 80% confidence.
 
  • Like
Likes atyy
  • #8
9,479
2,563
If I were offered a COVID-19 vaccine that had passed phase 2 trials, I'd accept it. Is that foolhardy of me?
Regardless of any other considerations there is simple basic concern for your fellow human beings, what we call mateship here in Australia. I would put my hand up in an instant. Here in Aus they had thousands of volunteers for the 120 needed in stage 1 trials of the UQ vaccine, and thousands have already put up their hands for the 1000 they want in stage 2 trials. Some people still hold to the old values my parents had, and instilled in me, of mateship that led even an ex-prime minister to put his life at risk as a firefighter during our recent bushfires. That of course is not a recommendation, merely a comment on the sociological culture that may also come into play.

Thanks
Bill
 
Last edited:
  • Like
Likes russ_watters and Grinkle
  • #9
phinds
Science Advisor
Insights Author
Gold Member
2019 Award
16,343
6,539
Russia's vaccine news has me pondering.
From BBC News:

https://www.bbc.com/news/world-europe-53751017

Russia has dismissed mounting international concern over the safety of its locally developed Covid-19 vaccine as "absolutely groundless".

On Tuesday, it said a vaccine had been given regulatory approval after less than two months of testing on humans.

But experts were quick to raise concerns about the speed of Russia's work, and a growing list of countries have expressed scepticism.

Scientists in Germany, France, Spain and the US have all urged caution.
and
And in the US, the country's top virus expert, Dr Anthony Fauci, said he doubted Russia's claims.

"I hope that the Russians have actually definitively proven that the vaccine is safe and effective," he told National Geographic. "I seriously doubt that they've done that.
 
  • #10
9,479
2,563
I read the forum rules and believe answering this question directly violates the rules. I think that should give you the answer.
With my mentors hat on I think the question is fine. We just do not discuss philosophy here, but elucidating queries with a probabilistic analysis so you can make up you own mind is fine. We just can't make a recommendation - simply give the facts. We can also, as I did, point to sociological factors that may come into it because of the culture of where you live.

Thanks
Bill
 
  • Like
Likes russ_watters
  • #12
atyy
Science Advisor
14,003
2,285
I personally just used the Binomial model in my analysis and didn't even think a Poisson model could be used. Here is an interesting paper I found on the issue:
https://www.tandfonline.com/doi/pdf/10.1080/21645515.2018.1433972
Interesting paper, but for the difference between the estimates by @Ygggdrasil and @Vanadium 50 it is as Ygggdrasil said, that they were calculating different things, and just for estimation, it is good enough to use the Poisson approximation to the binomial.

Using Ygggdrasil's formula ##\text{Pr(no adverse events)} = (1-p)^{N}##

if ##p=2/1000##, we get ##\text{Pr(no adverse events)} = 0.82## (same as Ygggdrasil's calculation)

if ##p=16/1000##, we get ##\text{Pr(no adverse events)} = 0.20## (same as Vanadium 50's calculation)
 
  • Like
Likes bhobba
  • #13
StatGuy2000
Education Advisor
1,795
900
Given an adverse event probability p with N individuals, the probability of observing no adverse events is ##(1-p)^N##, so for my calculation, I found the value of p for N=100 where you would have an 80% chance of observing no adverse events. This may not be the best way of calculating the statistical power of the trial (it was only meant to give a back of the envelope estimate), so it may be better to defer to someone like @StatGuy2000 with training in biostatistics.

However, I agree with your calculations that dosing 100 individuals would give a >80% chance of catching at least one instance of any adverse reaction that occurs with a probability of >1.6%, and that's probably a better bound to consider here.
Since my name has been mentioned, I should note that in general, the statistical power of the trial is almost always calculated based on the test of the null hypothesis based on efficacy. So in the context of a trial for a brand-new vaccine, one would, for example, look at the geometric mean titres of antibody levels in comparison to a placebo, and determine whether, say, the antibody levels persist after a given period of time by a certain percentage. The null hypothesis could be, say, that the mean titres may be no different than the placebo (I'm generalizing here, as I'm not currently involved in COVID-19 vaccine trials, and my past vaccine experience had involved a different design).

The probability of rejecting the null hypothesis would be the power, so one would calculate the null hypothesis by setting the power for efficacy to be, say, 90% (which we could calculate through simulations).

The main reason we don't compute statistical power through safety is that it is difficult to model what potential safety issues may arise and by what proportion or severity. And in all clinical trials, the default is to report all adverse events (whether serious or not, or whether it may be linked to treatment or not).
 
  • Like
  • Informative
Likes berkeman, Grinkle, bhobba and 1 other person
  • #14
Laroxe
Science Advisor
302
285
I feel as if I'm going to commit an act of heresy with this one when I say that the maths doesn't really help in answering the question. We go into vaccine trials with a great deal of information about what is the likely response, there are really only small parts that can be considered new and even then not totally new. We also have some limited information from animal trials and from the phase 1 & 2 human trials. We can't really compare drug trials as vaccines tend to be held to a higher standard, they have a different function. Phase 3 trials are still conducted in many thousands of humans before vaccines are made more widely available but Grinkle's concern is a valid one. There may be some risk to early release, but this is most likely a risk to public confidence, this has to be put against the risk of the uncontrolled pandemic.

Russia has decided that they will vaccinate medics and teachers early, they say this will be voluntary and then make the vaccine more widely available. If these people are carefully monitored they would be using this group as the subjects of the phase 3 trials. If the vaccine proved to be ineffective it would be withdrawn, if serious side effects occurred, these should be very quickly identified, but the same thing would happen during the normal phase 3 trials. China has apparently taken a similar decision using the army as subjects and the issue of vaccine challenge trials is also still on the agenda in the west.

The fact is that even with the results of the phase 3 trials, the vaccines will still be introduced to particular groups first and we will still be waiting to find out which of the vaccines are most effective.
I think what we are seeing are decisions being made based on a risk / benefit analysis, perhaps contaminated with some competitive national pride. Maybe the issue of reassuring the public and countering the antivax propaganda is so important in the west that the cost of the delay is worth it, I don't know, but there appears to be 100's of 1000's of people worldwide, who have volunteered to take that risk.
 

Related Threads on Covid Vaccine Phase 3 significance

Replies
14
Views
5K
Replies
2
Views
4K
  • Last Post
Replies
11
Views
4K
  • Last Post
3
Replies
58
Views
13K
  • Last Post
Replies
2
Views
3K
  • Last Post
Replies
6
Views
3K
  • Last Post
Replies
3
Views
3K
  • Last Post
Replies
2
Views
2K
  • Last Post
2
Replies
31
Views
6K
Top