On the half-dose accident of AstraZeneca: do they know or do they assume

The AstraZeneca / Oxford team that also has developed a viable vaccine admitted that they "accidentally" tested a treatment that consists of a half dose followed by a full dose (I wrote about that here). So, they reported results in three arms: the half-full treatment, the full-full treatment, and the placebo.

This is no longer a gold-standard randomized controlled test. First, the number of participants in the half-full treatment cell was not statistically designed - clearly, there are not enough participants in the half-full arm to attain the desired level of confidence. Second, the half-full treatment does not have a half-full placebo control. Third, who gets the half-full treatment is not randomly assigned.

This post is about how they know how many got half-strength first doses, and which participants. Because knowing that some first doses were half full is not the same as being able to identify which participant ids got half doses!

I have no idea how they did it. I can say based on experience running experiments it is not simple to figure this out. It seems like they would first need to isolate specific cases of half doses. (I'm not even sure how that would come to light.) Then, they would need to trace the half doses to specific production runs at the manufacturer, then follow these doses as they shipped to test sites, and assume that every such dose was halved. Remember this investigation was happening after the shots already went into arms. How would they ascertain after the fact that they were indeed half filled? If there were audit reports on the filling process, then why weren't these errors caught and corrected?

Conversely, how can they be sure that the other three-quarters of the vaccinated participants received a full first dose?

tl;dr

I use this example to show why execution of tests is a key success factor. You can't undo the effect of operational errors. But you can make assumptions to mitigate their impact. And when you read results from tests with execution errors, you must read them bearing in mind those additional assumptions. In this case, the strongest assumption is that they could identify which participants got the half doses, and which got the full doses. Figuring that out is absolutely not a piece of cake! It boils down to: do the researchers know? Or do they assume?

***

Disclosure: I am not up to date on the latest explanation for how the half-dose treatment arm came about; they may have switched their reasoning. I will be sharing my notes on the AstraZeneca vaccine trial results soon and will note any new insight on this issue. The point I'm making about knowing vs assuming is valid regardless.

Comments

[Ken]

The Lancet paper partly discusses the problem. Part of teh manufacturing process is determining the dose concentration. For the Phase I trial they manufactured the vaccine themselves and the concentration was determined by spectrophotometry. They then used a contractor to manufacture the vaccine and it found that spectrophotometry and qPCR gave different results for the concentration. They went with spectrophotometry. They tested subjects for immune response and found that it was poorer than in the Phase 1, and concluded that there was a problem with teh spectrophotometry. So they switched to qPCR to measure concentration.

I agree with Steven Senn's comment that the difference is likely to be one of those spurious results that happen at times in clinical trials.

[Kaiser]

Ken: I will be reviewing the Lancet paper soon. My brief read of it is the explanation is not detailed enough. If what you said was true, did they conduct an unblinded analysis while the trial was ongoing? Further, the explanation does not address my question. How did they go backwards and identify which specific subjects got the lower dose? I'll say more after I take in the whole paper.