It pays to know some basic statistics when "experts" are putting out bad information these days. Business Insider publicized a recent study that claims that rapid Covid-19 tests may fail to catch Omicron before the infection becomes infectious unless a throat swab is used in place of / in addition to a nasal swab. The test was designed to evaluate a nasal swab, and therefore, the FDA has come out to recommend against "taking matters into your own hands".
The recommendation to use a throat swab sounds reasonable and logical but is actually flawed. It only makes sense if one ignores the statistics of testing - and with that, one is ignoring the inexactitude of testing science.
I covered a lot of this material in Chapter 4 of Numbers Rule Your World (link) using steroids testing in sports as the motivating example.
***
Imagine making a soup, and there is a bay leaf somewhere in the pot, which you want to fish out before serving. That's the wrong analogy for a steroids test or a Covid-19 test. Testing is not about fishing out a whole organism from a soup.
Realize that typical tests are not looking for the whole virus but some marker - or proxy, which may be some chemical. It's a guessing game. If the sample has an amount of that marker above some threshold, it will be marked positive.
There is tremendous variability in humans, so that the amount of that marker varies from one to another, while the test developer has to decide on one threshold that works for everyone. This just means that no test can be 100% accurate. A false positive is a sample meeting the threshold but the person is actually not infected while a false negative is a sample that comes under the threshold but the person is actually infected. Wherever the developer draws the line, some proportion of the samples will be false positives or false negatives. The stricter the threshold, the fewer samples are classified as positives, and it follows that the test produces fewer false positives, and more false negatives.
How does the test developer calibrate this threshold? It typically must find a set of definitely positive samples and another set of definitely negative samples. The former is expensive to obtain while the latter relies on swabs that were taken pre-Covid-19 and somehow preserved. Now, the lab can measure the rate of false positives and false negatives, with different thresholds. The selected threshold depends on the tolerance of false positives and false negatives.
Notice that the popular Binax rapid home test indicates that they favor having more false negatives to having more false positives. That's why they tell you a positive test is highly credible while a negative test has a chance of error.
Now that you know how tests are calibrated, you should see why the FDA says don't switch to throat swabs using the same test kits. While the conceptual principle of the test works whether you're taking throat or nasal samples, the statistics may not work! With throat samples, or mixed throat-nasal samples, we have no idea if the previously measured false positive and false negative rates apply. The test threshold was selected using the amount of markers in the nasal samples. It is likely that a different threshold would be required if the input materials are altered.
The test developers can recalibrate the tests. So let's wait.
***
The study that Business Insider cites uses a small sample (30 people) that doesn't sound like it was selected randomly, and therefore it's unclear they are representative. The report tells us, interestingly, that all 30 people have had 3 vaccine shots, and 28 out of 30 were infected with Omicron.
Comments