In yesterday's post, I set the stage for discussing how science has been practiced during this pandemic.
I first outlined the key challenges for any data science during this period - especially in the early stages of the pandemic. What subsequently happened was not preordained. There was always the possibility that Covid-19 could run out of steam long before killing millions. In the history of pandemics, there hasn't been many that have hit so far.
Next, there has been an absolute avanlanche of research studies coming from all corners. The Medrxiv is averaging 900 new papers a month. No one has the ability to read all of these papers, not even the abstracts. The traditional science by peer review has been replaced with science by press releases and public relations. I can't imagine how long it would take for the peer review process to digest these thousands of preprints.
Unfortunately, coverage of these research studies tend to focus on the headline results, ignoring important scientific issues such as the research method, how data were collected, whether data were biased, etc. I gave an example in which the reporter did not distinguish between (a) a Phase 3 trial with 40,000 participants and a Phase 1 trial with 33, and (b) a result backed by a scientific publication and a result announced by a vaccine manufacturer in a press release. These results were presented together as if they were equally valid.
I then identify six major classes of research methods behind these research studies. The rest of my presentation addresses the strengths and weaknesses of these methods.
***
In Part 2, the focus is on the two most popular data-driven research methods: statistical experiments (aka randomized controlled clinical trials) and observational studies (aka real-world studies).
I address the following questions:
- What are the key elements of a statistical experiments?
- How are vaccine trials like/unlike a statistical experiment?
- Why is the experiment considered the gold standard?
- Does vaccine effectiveness coming out of a real-world study give the same information as the similar outcome from an experiment?
- What is different about real-world studies from experiments?
- How do researchers correct for selection biases in real-world study populations?
- How does immortal time bias and calendar time bias affect real-world studies?
Of course, experiments and real-world studies are both necessary. Once the vaccination campaign started, it is considered not ethical to run experiments so we only have observational data. My conclusion is that these methods should be utilized for different purposes.
***
There is a Part 3. Come back next week!
The Novel Coronavirus Research Compendium (NCRC) at Johns Hopkins isn't reviewing everything, but they're hitting the most important/influential papers including on pre-print servers. Coming from this seroius group of experts, the NCRC should be considered a major resource for understanding the vast amount of evidence coming out about Covid-19.
https://ncrc.jhsph.edu/
Posted by: Morgana Mongraw-Chaffin | 05/27/2021 at 12:52 PM