I've been trying to reconcile a graph shown in the Lancet paper covering the Astrazeneca-Oxford (AO) Phase 3 trials, and a table in Appendix 1 of the same publication. This gets at the heart of the illogical way in which the investigators of vaccine trials (not limited to AO) are counting "cases".

As I have described on this blog, all so-called primary analyses of any vaccine trial start counting cases X days after the second shot, where X ranges from 7 to 21 days depending on the developer. This counting rule is justified by the claim that the vaccines take 7 to 21 days *after the second shot* to induce enough antibodies to fight the virus.

However, most vaccine developers have also been using the same trial data to argue that the vaccine offers partial protection after the first dose - there are even scientists who claim that the first dose by itself is sufficient. Sharp-eyed observers will realize that such an analysis would be impossible if they start counting cases from 2D + X days! Shouldn't there be zero cases from 1D + 1 day up to 2D + X days?

Apparently not, according to this graph from the Lancet paper. Notice the x-axis is labeled "days since first standard dose". Apologies for cutting out the y-axis labels. It's the cumulative case rate, and each tick is 1% of the analysis population.

Notice the "exclusion period." Instead of counting all cases after the first dose, they start counting at 1D + 22 days.

***

Now note the size of the analysis population (indicated below the graph), 6307 receiving the vaccine and 6297 taking the placebo. Appendix 1 offers more details on how they determined the analysis population, starting with everybody that were enrolled into the trials.

The relevant TableS3 looks like this:

The two columns COV002 and COV003 are the U.K. and Brazil trials respectively. For the key AO analyses, the two datasets are combined ignoring the underlying differences. In each trial, about 60% of the enrolled participants make it to the analysis population.

What puzzles me is the 8th exclusion reason described as "PCR+ test <= 14 days post-second dose". (By the way, these are not reassuring numbers: in the U.K., the vaccine arm shows twice the cases as the placebo arm and in Brazil, it's about the same.) **I just don't understand why something that happens post second dose should determine whether or not someone should be included in an analysis that starts at 21 days after the first dose**.

Because the size of the analysis population aligns with what's shown in the graph, I can confirm that those cases are indeed removed from the analysis.

It is possible that they have an unedited typo: should the line have read "<= 21 days post-first dose"? I have no idea.

If it's not a typo, then it appears that someone who got sick between 1D and 1D+21 days would be included in the analysis population but those cases would not count. Then, those who got sick between 1D+21 days and 2D would be counted as both cases and participants (thus contributing to the graph shown above). Then, those who got sick between 2D+1 and 2D+14 days would be excluded completely. And they restart the tally from 2D+15 on. That's a highly convoluted, illogical way of including and excluding cases based on when they occurred in the timeline.

***

DETOUR: Going back to Table S3 for a minute. The primary reasons for being dropped from analysis differ markedly between the two trials. For Brazil, it's because the trial started later, and many have not reached 1D + 21 days at the time of analysis. For the U.K., it's because the analysts decided to drop the low-dose cohort (LD/SD) from this analysis. I've already written elsewhere about the many built-in biases to the AO results so I won't repeat that here.

DETOUR 2: The graph shown above is **not** an analysis of a single-dose treatment. There is no exclusion for not yet receiving the second shot so it is a mixture of people who has had both doses and people who has only had the first dose. In addition, that second shot was not well targeted to a particular day (or week) of the trial. If you get the drift here, **there is not a single analysis in the AO paper that can be interpreted without examining underlying biases.**

***

What is doubly puzzling is that the numbers shown in TableS3 differ from those found in TableS2, which computes the analysis population for the primary analysis, that which starts counting from 14 days after the 2nd dose.

In the U.K. trial, there are three times as many exclusions in the primary analysis than the secondary analysis while in Brazil, there are slightly fewer exclusions in the primary than the secondary analysis. The two trials again show their differences. I'd guess that the number of exclusions due to testing positive before 2D+14 should be higher for secondary analysis as that analysis includes more participants by definition.

Someone smarter than I must be able to make sense of the above. I'm stumped.

## Comments

You can follow this conversation by subscribing to the comment feed for this post.