The power of groupthink

In a congressional hearing this week, a leader from the CDC (or was it NIH, I couldn’t recall) mentioned “group testing” as a new tactic to be rolled out. Unfortunately, this may be another case of too little, too late for the United States. But the concept of group testing is rooted in statistics, so let’s talk about why it is an attractive idea.

Imagine we need 20 cc of blood to conduct a diagnostic test. Test kits are short in supply, so we must make some magic. Normally, we use one test kit per person tested. Is it possible to test N people using fewer than N kits?

Statistics enter the picture. We’ll consider 10 people. Take 2 cc of blood from each sample and mix them thoroughly. Use one test kit on the pooled sample. If this pooled sample tests negative for the virus, we conclude none of the 10 are infected, and so save nine test kits. On the other hand, if the pooled sample tests positive, we have to next test each individual, so in this case, we use 11 kits for 10 people.

There is a chance we may use fewer than 10 test kits. But it’s not guaranteed. Whether we will end up with savings depends on the proportion of pooled samples that test negative. A key calculation is the probability that when forming groups of 10 people, all ten are not infected.

Suppose 10 percent of test-takers have the virus. Then, in an independent pool of 10 people selected at random, the chance that no one is infected is (0.9)^10 = 35%. Thus, we expect 35% of these pools of size 10 to test negative, resulting in a saving of nine kits each. For the other 65% of pools, we run one extra test relative to the base case of 10 tests for 10 individuals. On average, we save 2.5 tests for every 10 people. The saving rate is 25%, a quarter fewer test kits. Not bad!

***

For the U.S., is this development coming too late? The group testing strategy confers savings only if the virus is not too widespread. If there are too many infected individuals, then most pools will have at least one infected person, and instead of saving kits, we may burn up even more kits than normal. Let’s repeat the previous calculation but assume 20% of the test-takers have the virus. In this case, about 10% of the pools have no virus while 90% comprise at least one infected sample. On average, we break even – the expected number of tests using group testing of size 10 is ten tests, the same as one test per person. All for naught.

For those interested, you can analyze different pool sizes.

***

Breakeven is not neutral though. If I were the decision-maker, I’d need the group testing strategy to show a clear, positive benefit. That’s because the group testing method has several non-obvious costs.

First is the cost of administration. One test per person is simple to execute. Group testing requires splitting samples, storing samples until the pooled test result is available, matching samples, etc. Much better book-keeping and tracking are required.

A second cost comes from potential errors due to complexity. Samples can get contaminated, mixed up, or lost.

Third is its impact on test-takers. More blood or sputum must be collected in anticipation that some of the samples will be tested twice. Also, it will take longer for the results to be confirmed.

The fourth cost is more false negatives. Imagine a pooled sample with exactly one infected individual. By taking one-tenth of each sample and mixing them together, we dilute the viral load of the infected individual 10 times. It is possible that the pooled sample – and thus all 10 individuals - will be declared negative. If the infected person were tested alone, the viral load would have been high enough to return a positive but the pooling causes a false negative. Such false negatives may make kit savings a pyrrhic victory.

***

Significance magazine has a recent article about group testing.

Comments

[rw]

Is there a way to subscribe or follow the RSS feed of this blog? I can't find it anywhere :(

[Kaiser]

rw: For RSS, there are two links:
https://junkcharts.typepad.com/junk_charts/atom.xml
https://junkcharts.typepad.com/numbersrulelyourworld/atom.xml
Or use my twitter feed: https://www.twitter.com/junkcharts, which combines the two blogs.