The planning phase of analysis does not get half the attention it deserves. In reviewing the final report of J&J's vaccine trial, I found an example of how definitions have definitive impact on outcomes.
The one-line conclusion of the report was "A single dose of Ad26.COV2.S provided 52.9% protection against moderate to severe–critical Covid-19." Unlike the Pfizer and Moderna trials, the primary endpoint for J&J has always been stated as "moderate and severe--critical" cases. This outcome lends itself to the narrative around vaccines not necessarily preventing infections but protecting against severe disease.
Here is an excerpt of Table 1 from the paper:
The one-line conclusion is displayed in the first row of the table, right column. To be specific, this endpoint is adjudicated, symptomatic, PCR-confirmed Covid-19 cases that were classified as not mild occurring at least 28 days since the vaccination.
The second section of the excerpt show how cases are classified by severity: mild, moderate, and severe--critical.
Surprisingly, "mild" cases constitute <2% of the cases. Thus, the distinction between "mild" and "not mild" is a distinction without a difference. Given our lived experience that the vast majority of Covid-19 cases are mild, this statistic suggests that the definition of "mild" is mistaken.
This misspecification is non-trivial, especially if one believes the hypothesis that the vaccine reduces disease severity. With this definition of severity, an analysis including or excluding "mild" cases will yield the same conclusion. If many more of the "moderate" cases were reclassified as "mild", we would expect the adjustment to affect vaccine and placebo arms differentially, and thus the VE would be different.
***
In my view, the result of the J&J vaccine trial should be read as protection against disease of any severity. This brings the endpoint in line with those used by Pfizer and Moderna.
Finally, if the FDA wanted to demonstrate that these vaccines reduce disease severity, they should have required different endpoints that directly measure it.
Comments