The shift to antibody testing is a choice of death over life

As the world rushes to re-open the economy, all the talk is on “test, trace, and isolate”. In places that have fallen down on rolling out diagnostic testing (the U.S. and the U.K., for example), the officials have started talking up “antibody testing”. We’ve got to talk seriously about what this means.

Diagnostic testing and antibody testing are not substitutes

The way “test, trace and isolate” and “antibody testing” are being discussed simultaneously creates the false impression that the “test” in test, trace and isolate refers to antibody testing. That’s deadly wrong. What’s needed to support contact tracing is diagnostic testing.

Antibody testing is not a substitute for diagnostic testing. These tests serve different purposes. The diagnostic test (e.g. the PCR test) finds evidence that the novel coronavirus currently lives in the sputum of the patient. A patient is declared recovered if s/he tests negative for a period of time. By contrast, the antibody test looks for antibodies that are produced by our bodies to fight the coronavirus. They are signs that the patient has overcome the virus and recovered. Hence the talk of “immunity”.

The diagnostic test tells about the present while the antibody test reveals the past. The goal of contact tracing is locating people who might have come into contact with an infected person so that they could be examined and isolated, preventing further spread of the virus. To be effective, you need to find the infected people while they are infected, not after they have recovered at which point they are not infectious.

Imagine a contact tracing app with dots for people on a map. People who tested positive are colored red. If the red dots are antibody-positives, what good does that do us? These are recovered and non-infectious people seen moving around the map. On the other hand, if the red dots are coronavirus-positives, then people near the red dots know they are at risk.

 

The morbid chase after herd immunity

Antibody testing results are being widely used by authorities to reveal a path to “herd immunity”. This epidemiological theory promises that some small proportion may be spared infection once a sufficiently large part of the population become immune. If the results from the antibody tests could be believed (see next section), the proportion having antibodies is below 20 percent, well below in many places. It’s not yet clear for how long these antibodies confer immunity. Even if they did, 20 percent is far from sufficient to attain herd immunity. For smallpox to reach global herd immunity, 80 percent had to be immunized through vaccination.

As long as we don’t have a vaccine, herd immunity against SARS-CoV-2 can only be achieved by infection (assuming immunity after infection). Every infected person has a chance of dying. With a 0.1% fatality rate, conservatively set at that of influenza, the U.S. would suffer over 200,000 deaths to get there. The path to herd immunity is paved with body bags.

 

Inflated estimates of the coronavirus prevalence

The prevalence of SARS-CoV-2 antibodies coming from antibody testing is very likely to be overly optimistic. For test results to apply to the general public, the test population must be broadly representative. Random selection, the gold standard, is basically impossible to achieve under the roof of shelter-in-place rules. No matter how many times New York governor Cuomo claims to have conducted random testing in my state, while providing no data to back up his declaration, it defies belief. Random selection from a biased subset results in a biased sample.

There is anecdotal evidence that people are seeking out these tests due to test rationing. Those confusing this with the diagnostic test may have worrying symptoms. Those who fully understand the purpose of antibody tests won’t show up unless they have reason to believe they were infected. The sample is therefore likely to be biased towards the infected.

In addition to selection bias, antibody tests have low positive predictive value. This jargon means a high proportion of those who tested positive actually do not have antibodies. This is true even if a test advertises an apparently impressive false-positive rate of 2 percent, correctly diagnosing 98 out of 100 negatives. The 2 percent error rate when applied to 90 percent of the population who do not have the coronavirus generates 1.8 positive results per 100 tests while the other 10 percent truly positive begets 10 positives per 100, assuming zero false negative. Thus, when prevalence is 10 percent, 3 out of 20 positive results are erroneous. [Note: prior calculation misstated the true positive counts, as spotted by readers in the comments.]

If we assume 10 percent prevalence and 10% false negative (90% sensitivity), 1 out of 6 positive results are wrong. If we assume 10 percent prevalence, 10% false negative rate and 10% false positive rate, then for every true positive, there is a false positive. Such a test is rated 90% sensitivity, 90% specificity. Because the false positives are more numerous, this test returns a higher proportion (18%) of positive results.

 

Antibody testing is an unregulated business

You’d think the FDA has to approve antibody tests before they are marketed to the public. Wrong!

It has emerged that the vast majority of the 100+ antibody tests that are already on the market have not been vetted by the FDA. The lack of regulation produces a ripe environment for scamming. Note again that antibodies are for those who have recovered and therefore may have immunity. Many of us wish we already had it, was asymptomatic and have become immune but most of us have just never been infected. So, we may be tempted to keep testing for antibodies until, hopefully, one day the test returns a positive. Each test is money for the test vendor.

Further, people will pay for the antibody testing thinking it is diagnostic. This FDA "fact sheet" for one of the "approved" tests (provisional) specifically claims falsely the test can tell patients whether they currently have coronavirus.

Having inaccurate tests on the market also damages the public health efforts. Even the properly calibrated test has a false-positive problem. We may end up with lots of people roaming around thinking they are immune when they aren’t.

A few days ago, the FDA supposedly has new rules requiring test vendors to submit validation data within 10 days. I wonder how they generate such data. The note from the FDA is much more measured, and contradicts the fact sheet which raises the possibility that the "fact sheet" was written by the test vendor and no one at the FDA actually read it.

***

Antibody testing is a rear view mirror. If someone is found to have antibodies, it’s because the person has recovered from the disease. It follows that the person was infected, and so the person could have died. Antibody testing without diagnostic testing means death over life. It means testing those people who survived the disease, and letting others die.

Comments

[Parenthetical]

I'm confused about the true positive rate. If 10% of the population has antibodies and the false negative rate is zero shouldn't we get 10 true positives for 100 tests?

[Kaiser]

(P): Be careful about the baseline number. Out of 100 people, 10 has antibodies; out of those 10, all 10 test positive so false negative rate is zero. And the true positive rate is 100%. But you can't say 10 true positives for 100 tests because the other 90 tests were done for those without antibodies - and there's where the false positives come in and make trouble!

[Parenthetical]

Maybe I am confused about the definition of prevalence. This is the phrase I'm stuck on: "while the other 10 percent truly positive begets 1 positive per 100". If ten out of every hundred people are truly positive, how come we only get one positive result from testing them?

[TBW]

"The 2 percent error rate when applied to 90 percent of the population who do not have the coronavirus generates 1.8 positive results per 100 while the other 10 percent truly positive begets 1 positive per 100, even assuming zero false negative. " - if the false negative rate is zero, then for those 10 people that actually have the virus they would all test positive. Compared to the 1.8 people out of 100 who would not have the virus but test positive anyway. It feels like the comparison is 1.8 false positive to 10 true positives, not 1 true positive.

[Kaiser]

(P), TBW: Yes, I see where I flipped the numbers. Fixed the text. Thanks!

[Antonio Rinaldi]

Or assume 2% prevalence, so with 2% false positive rate and 0% false negative rate you obtain 2 true positives among every nearly 4 positives (~50%).

[Ben]

"after they have recovered at which point they are not infectious."

This has yet to be proven. A fair assumption, but yet to be proven.

[Kaiser]

Ben: Yes that point can't be underlined enough although should we start with it, the entire enterprise of antibody testing is pointless! My understanding is that it is technically true that antibodies confer immunity but sometimes the duration of immunity is short especially if the virus may mutate.