WHO has approved the Chinese "Sinopharm" vaccine for emergency use (link), soon after an interim analysis of its Phase 3 trial was published in Journal of American Medical Association (report). We have heard nothing from the FDA about Sinopharm, nor has there been any news on the Astrazeneca/Oxford vaccine, so the only vaccines that have been approved in the U.S. remain three homegrown vaccines.
The Sinopharm vaccine is notable for using the long-established science of inactivated viruses. The vaccine does not require special cold storage, and its long-term safety record is supported by successes in fully approved flu and polio vaccines, an advantage that the newer technologies do not enjoy. The primary unknown to be resolved in the Phase 3 vaccine trial was the efficacy against SARS-CoV-2, as the flu shot based on inactivated viruses have "low" but still accepted efficacy in the 30-50% range (link).
The Phase 3 trial brings the welcome news that the Sinopharm vaccine also has efficacy in the 70-80% range against placebo, on par with Johnson & Johnson and Astrazeneca, both of which come from the adenovirus family. The new result clears well above the minimum acceptable standard of 50% required by regulators.
VE of 50% means vaccination cuts the chance of developing symptomatic Covid-19 by half. The background case rate - as measured by the placebo group - is in the low single digits. In the Chinese clinical trial, the reported placebo case rate was around 1.2% in the four-month study period. Cutting that number by 70% means the chance of getting sick with Covid-19 for those vaccinated is just under 0.4%, or 4 in 1000.
Recall that when generalizing the VE from a single clinical trial to the general population, we should look at the lower end of the 95% confidence interval for VE, in which case, we should say the VE of Sinopharm when applied to the general population is expected to be 60-65%. This proviso applies to all vaccine trials, not just the Chinese one.
The design of this Phase 3 trial is essentially identical to those of the U.S. trials that I have discussed before. If anything, the JAMA report and appendix provide more details than I have seen in the other scientific reports.
Similarities
Let's first describe the commonalities. The Sinopharm vaccine also requires two doses, taken 21 days apart. The control group is given two doses of a placebo. The trial ran from July to Nov 2020 in UAE and Bahrain.
The trial only counts adjudicated, lab-confirmed, symptomatic cases of Covid-19. The trial investigator - that is, the vaccine developer - only investigates a case if the participant experiences qualifying symptoms from a pre-published list. The infection must be confirmed by a PCR test. Finally, a committee of "experts" determined if the PCR-positive, symptomatic case should be counted or not. In broad terms, this procedure is how all the other pharmas have run their vaccine trials.
The headline VE number of 70-80% counts only cases that are detected from 14 days after the second dose. This also is in line with current practice. I am not a fan of this maneuvre. You are forced to interpret the efficacy as a "conditional" metric. Infection rate is cut by 70% - but not for everyone, only if one does not fall ill between the first dose and 14 days after the 2nd dose.
This maneuvre is a spiritual cousin of how the Bureau of Labor Statistics computes unemployment rate. When the unemployment rate is 4 percent, one cannot infer that 96 percent of working-age people have jobs. It actually means 96 percent of the "labor force" have jobs. The labor force excludes people for many different reasons, such as anyone who has given up on the job market. So a proper way to interpret a 4 percent unemployment rate is as a conditional - 96% of people are employed only if they are classified as part of the work force. Conditional metrics can be highly misleading as it shoves a chunk of the population under the rug. (See this post on unemployment rate - or read Chapter 6 of Numbersense (link) for a deeper study).
Similarly, a college might report that the average salary of its graduates is $80,000. That usually means it's an average of $80,000 only for those who have found jobs (and self-reported their salaries to the careers office). If you're a parent, and you want to know if your kid might earn $80,000 after graduating from this college, it depends on whether the kid finds a job. The $80,000 number provides zero information about the chance of graduates landing jobs. If the kid fails to find a job, the relevant salary number is 0.
Sinopharm also computed a variety of VE metrics, just like everyone else. If they had counted cases from first dose, the VE drops to 50-66%. If they had included asymptomatic cases (but counting only from 2D+14), the VE drops to 64-74%. None of this is surprising as we have seen this movie before in all the other trials.
The Chinese vaccine also claims 100% efficacy against severe Covid-19. As with the other trials, I find this claim absurd. None of these trials have enough people in them to measure the rare event of severe Covid-19. This Sinopharm claim is based on 2 cases in placebo versus 0 cases in the vaccine arms. Also available to the Chinese team is the other dubious statement that "no one who's been vaccinated died in the trial". I'm sure you've heard that one too about the U.S. trials. (No one died in the trial.)
Also as with other trials, subgroup analyses were computed robotically but nothing of value could have come out of them as sample sizes were too small.
In a nutshell, using the rules of the game that have been set by the regulatory agencies, the Sinopharm vaccine demonstrated similar "spectacular" levels of efficacy - even though it is based on old technology. If, as some people have argued, J&J's 67% efficacy should be considered equal to Pfizer and Moderna's, then this vaccine should be considered in the same class. (Same with Astrazeneca).
Differences
There is no obvious deviation from the protocols of the U.S. trials.
One minor difference is the Chinese trial tested two vaccines (developed from different strains of the virus) against placebo. When I stated the results in ranges, that's because the two vaccines showed slightly different numbers. Thus, there were three arms to the trial, each with equal number of participants (roughly 14,000 per arm).
In one respect, the trial result is stronger in my view - the follow-up period is longer. The follow-up time for the median participant in the Chinese trial was described as 77 days - wait a minute, they defined that as 77 days from the start of the case-counting window, thus this is 91 days (3 months) from the second dose. Recall that the FDA set a rule that they will not approve any vaccines for which the data do not have a median follow-up period of two months - and that clock starts ticking from second dose. Eventually, the FDA approved two vaccines (Pfizer, Moderna) when the published data did not even meet the two-month rule; and for Johnson & Johnson, they condoned a flexible interpretation of two months as 8 weeks (56 days).
I was very excited to find eFigure 2 in Supplement 3 of the JAMA report (which erroneously pointed readers to Supplement 2). This set of charts lays out exactly why the VE number computed as an interim analysis must be interpreted carefully.
Look at the enrollment timeline in Bahrain, from Aug to Oct 2020. In particular, consider someone who enrolled after late September, which was easily over a quarter of the participants. They took the second shot in the second half of October but the study does not count cases until 14 days later (Nov). The cutoff date of the analysis is end of December so this set of participants have at most 1.5 months of follow-up beyond the 2D+14 time point.
Now look at the cumulative case curve. This subset (marked by the brown box) can only influence the curve between 0 and 45 days. Meanwhile, those who enrolled at the start of the trial (blue box) took their second shot by mid September so their follow-up period lasted from late September to late December (up to 3 months).
It would have been fine if the timing of enrollment was randomized - that every weekly cohort of participants have the same characteristics. But this was not true of any of the trials. Pfizer added more minorities (higher risk) later in the trial. Astrazeneca started with younger and lower risk participants. The Chinese trial enrolled older people (60 years plus) later. None of these reports discussed possible biases related to staged enrollment, and none of them corrected for such biases.
You might be wondering how the scientists found data to draw out the case curve beyond 60 days, given that only a fraction of the Bahrain participants have reached that amount of follow-up. The answer is data from the UAE sites. In other words, the right side of the case curve contains predominantly data from UAE while the left side contains data from both UAE and Bahrain.
That is no small matter either. In the same eFigure 2, the researchers reveal that the case rates in UAE and Bahrain looked very different.
UAE has twice the number of cases (but 5 times the population) of Bahrain. Also cases were increasing during the study period in UAE while they were decreasing in Bahrain.
When I reviewed the previous vaccine trials, I was only able to speculate about these potential biases because those reports omitted this type of data. With this JAMA report, it's clear the biases are present, and I urge the other pharmas to publish similar data. Even Sinopharm can publish more. It's not enough to know the timing of enrollments; it's important to know the potential covariate imbalance between weekly cohorts.
***
I have split my notes into two parts. Part 2 of my review of the Sinopharm report will be posted this week.
Comments