Readers may have wondered why I have said little about vaccine safety in my reviews of vaccine trials. Today's post, which fills that gap, also serves to explain why.
As is well known, the FDA proclaimed that the mRNA vaccines (marketed by Pfizer and Moderna) are both efficacious and safe. Each trial enrolled about forty thousand participants, split randomly into vaccine and placebo groups, with the latter receiving a saline solution instead of the vaccine.
For efficacy, the primary outcome of interest was infections (measured by its proxy, reported symptomatic cases). As observational data drifted away from this narrative, the outcome shifted to hospitalization and death attributed to Covid-19. Whether the endpoint is cases, hospitalization or death, the clinical trial design ensures that an analyst can look at the difference (ratio) of the event rates to assess whether the vaccine performed better than the placebo. [That is, until after the emergency use approval was granted, because the FDA then allowed the pharmas to dismantle the placebo groups, destroying the trial design.]
Measuring safety - on the surface - may proceed in a similar way. However, safety outcomes present two enormous challenges that make safety analyses much harder - and thus less convincing - than efficacy analyses for the same cohort of trial participants.
First, it's difficult to know which safety outcomes are of greatest concern. It's a bit of a fishing expedition. Lots of outcomes are measured to screen for signals. Such data mining always raises the probability of false signals. Thus, all else equal, one would require a larger sample size to analyze safety outcomes. Since the sample size is fixed for both safety and efficacy outcomes, this implies that the confidence we have for safety is necessarily lower.
Second, the truly horrible safety outcomes - known as "severe adverse events" (SAEs), which include hospitalization and death - are expected to be rare events. Since a vaccine is indiscriminately injected into the population, most of whom are healthy people, even a small rate of occurrence can be alarming. Let's assume a hypothetical vaccine causes death with probability 1 in 500,000 shots (this is a thought experiment for illustration purpose only). Bloomberg reported that over 600 million shots have been administered in the U.S. alone. On average, 1,200 people would have died from this vaccination.
I chose 1 in 500,000 as the hypothetical rate of vaccine-induced death for a reason - the vaccine trials as designed would not have been able to detect this signal, let alone death rates below that level. For an event that happens at a rate of 1 in 500,000 attempts, the chance of observing zero occurrences in 25,000 people is 95%, and the chance of observing a maximum of two occurrences is 99.99999%. In other words, any SAE with a rate of occurrence of 1 in 500,000 would likely produce no events during the vaccine trials, and given the amount of shots that went into arms, that level of harm would have killed 1,200 people (while the analyst legitimately touts a perfectly safe vaccine).
***
In their respective trials, at the interim analysis that happened when half the participants have reached two months of observation beyond the second shot, Pfizer and Moderna observed few SAEs, and concluded that these event rates were "balanced" between the vaccine and placebo arms. This was the basis of the safety claim.
A group of researchers have recently revisited the analysis, and drawn a different conclusion in a peer-reviewed article published in the journal Vaccine. (link) They argued that the mRNA vaccines caused an excess of 10-15 extra SAEs of special interest relative to placebo. That's a difference. As a ratio, the mRNA vaccines caused a 43% increase in SAE risk. Most of the excess risk in the category known as "coagulation disorders".
It's instructive to learn why the same data led to different conclusions.
The first difference is what is counted as an SAE. It turns out that the FDA and U.S. pharmas created their own definition of what counts as SAEs. Meanwhile, the international community, something called the "Brighton Collaboration", agreed on a priority list. The new analysis reclassifies the events reported by Pfizer and Moderna in an attempt to perform an analysis that follows the international standard. Even here, the researchers conducted two analyses, based on the "original" list and an "expanded" list.
[For those wanting to peep into the sausage factory, the reclassification involves three experts readjudicating the events based on subjective evaluation of case history and other information. The original classification during the trials also involved such subjective judgment.]
The second difference is the proper handling of differential observation time among participants. Remember that the interim analysis were conducted while people were still in trial. I noted when I read the efficacy analyses the fact that the earlier participants were observed for longer periods of time, so that, even though the "median" participant reached the two-month point post second dose, the interim analysis involved many participants who had not yet reached the start of the case-counting window, which means that for the purpose of the interim analysis, these participants were "immortal"!
It turned out that the FDA-endorsed analysis included participants who have not yet received the second shot, since they counted any SAE at data cutoff, i.e. the same cutoff date applied to all participants regardless of enrollment date. The new analysis removed participants who had not yet taken the second shots.
***
A few other noteworthy nuggets can be found in the article. (link)
First, the authors noted that the Pfizer (Moderna) vaccine reduced risk of hospitalization by Covid-19 infection of 6 (2) per 100,000. So, for every 100,000 vaccine recipients, around 2-6 are expected to have avoided hospitalization relative to not getting vaccinated, but 10-15 are expected to experience serious adverse events relative to not getting vaccinated. As the authors noted, such a benefit-harm analysis should be conducted by age group but one of the enduring mysteries of Covid-19 vaccine science is the lack of available data of sufficient granularity for third-party researchers to conduct independent studies - all the more baffling given the claim of massive benefit and zero harm, which presumably should withstand even moderate- to large-sized error bars!
Just for illustration. The 2-6 hospitalizations are likely concentrated in older people and if the 10-15 excess SAEs occur mostly in younger people, then the risk in the young is higher than average while the benefit is lower than average.
Second, Pfizer stopped reporting safety outcomes beyond one month after the second shot.
Third, Moderna counted Covid-induced SAEs in the SAE analysis, which the authors pointed out is not the "traditional" analysis. (Since supposedly the vaccine's side effects are similar to the effect of Covid-19 infection, differentiating between them requires experts to determine the cause of the observed SAEs. That's definitely not something you or I am qualified to do!)
Recent Comments