The efficacy achieved at various Covid-19 vaccine trials is a marvel of medical science. Because of ballyhooing around 90%-plus numbers, the media roll eyes each time they discover someone who tests positive despite getting vaccinated. This is not surprising, though. Here are 8 ways it happens - they are hiding in the text of the trial reports.
1) Mirage of measurement. Realize that when the investigators computed 90%+ efficacy during the vaccine trials, they only counted cases that surfaced from 7 to 21 days after the second shot (the exact day depends on individual trials). Mathematically, this definition nullifies all cases that happened right after getting the first shot, during the interval between shots, and the 6 to 20 days after the second shot. Paradoxically, the trial data showed that a lot more people get infected during those time windows than during the analysis frame! (See this post for a visualization of the time-line.)
2) The trials report lower vaccine efficacy prior to the second shot (see chart below and explanation here). The vaccine tempered down the growth rate of cases over time, and in the case of Pfizer/Biontech, eventually attaining about 80 percent reduction. (If we count all cases after the first shot, the VE was 80%, not 95%.) During the Pfizer trial, the efficacy was roughly 50% on the day before the second shot so it's clear some people will get infected in the 20 days after taking the first shot.
3) The infections after vaccination may be asymptomatic. With the exception of the U.K. edition of the AO trial (I discussed the findings here), all other trials with published results so far have measured only symptomatic, and PCR-confirmed cases. By not counting asymptomatic cases, the trial results mislead us into inferring there were no asymptomatic cases. (Nevertheless, it is reasonable to believe that one reason for finding fewer symptomatic cases among the vaccinated trial participants is the partial suppression of asymptomatic spread - although the vaccine trials were not designed to confirm this hypothesis, and no amount of post-hoc analyses can prove it.)
4) New variants of the coronavirus may evade the approved vaccines. There is emerging evidence that these vaccines may be less effective (or ineffective) against certain new variants of the coronavirus. AO just reported that their South African trial (finally a double-blind trial) showed no efficacy, and it is known that the dominant variant there is new.
5) The unlucky few. Even if a vaccine is 90% effective (pretending that this number measures all infections from the first shot to 60 days after the second shot), this means one's chance of getting infected has been cut by 90 percent, and is not zero.
6) The baseline may deteriorate. Vaccine efficacy is defined relative to placebo (i.e. the baseline rate of cases among an unvaccinated population). If the baseline rate rises, for example, due to viral mutation, the case rate among the vaccinated will also increase, even if the 90% reduction is maintained. This is why epidemiologists are very concerned about the higher transmissibility of new variants.
7) The unlucky whole. Last but not least is statistical error. Statistical analysis takes a sample of data (the thousands of trial participants) and generalizes the result to entire populations (tens of millions and more). Every result has a margin of error, usually 5 percent. The trial result used to assert 90% efficacy could be an outlier. The real efficacy may be less than reported.
8) Vaccinations affect our behavior. Many people expect to return to normal life after getting the vaccine. The baseline rate of infection measured during the vaccine trials occurred in the context of the lockdown and mitigation policies of the time. If the level of contacts in the population jumps as a result of the vaccination campaign, the baseline rate may increase, and so follows the expected case rate among the vaccinated.
***
Even if the case trend does not quickly drop, it's not clear the vaccine is ineffective. On the other hand, even if the vaccine is 50% effective, it would still be helpful in the fight against the pandemic. Finally, approval by the FDA (and other regulators) is not an endorsement of the 90% efficacy number, despite that number being touted all day long. The threshold for emergency approval of these vaccines was 30% efficacy (in some places, 20%). (If the advisors were asked to vouch for the 90% number, I'm sure many of them would have refused.)
Comments