Two types of lockdowns: why US must rethink its testing policy

(Image credit: Beth H)

In the last few days, New York City (where I live) is slowly being locked down. Broadway is closed, most event venues are shut, many restaurants voluntarily closed, and then were required to operate at half capacity, and finally decreed to shut in-person service. The latest meme appeared to be "assume everyone has it".

It appears that the U.K. has taken a similar approach to this crisis, from what I see on BBC.

It would appear that in a matter of days, the U.S. may be headed down the path of China, South Korea, Italy and others like Germany and Spain in locking down the country.

***

Please be clear that there are two types of lockdowns.

The first type of lockdown - as exemplified by China, South Korea and Italy - is based on more complete and accurate data (at least as complete and accurate as one can hope for in a time of crisis). These countries are conducting lots of tests, and finding many cases of infection. They are publicizing where the infections are, they are being strategic in tracing those cases, and paying special attention to preventing community transmission.

The second type of lockdown - soon to be exemplified by the U.S. - is based on severely limited data. It is based on the assumption that everyone has been infected. The necessity of this kitchen-sink assumption is driven by the lack of knowledge. It is a position of weakness. The duration of the lockdown will be longer, the spread of infections will be wider, and the long-term costs will be higher. It is not the case that everyone has the coronavirus but we have to assume it because we are not testing nearly enough people.

In the U.S., we are also assuming wrongly (as I explained this morning) that the places with reported cases are the "epicenters" of infections. This conclusion is correct in China, South Korea and Italy because they have test data. In the U.S., lack of cases more likely reflects lack of testing. This might explain why there is as yet no restriction in domestic travel in the U.S.

***

Testing is the the key. Testing is needed to identify the clusters of infections at the start, and to guide targeted lockdown efforts. Testing is then needed to monitor the spread of the virus, to measure the effectiveness of the containment measure, and to guide ongoing decisions to expand or end the lockdown. In short, testing produces the stream of data needed to guide policy.

The lack of testing does the opposite. You can't nail down the clusters, you can't assess the efficacy of the containment, you can't monitor the transmissions. You have to resort to kitchen-sink assumptions, and the lockdown will then have to be indefinite until some medical professionals feel sufficiently confident that the virus has gone away or an effective vaccine has been created.

The worst thing to fear is fear itself. With incomplete data due to lack of testing and selective testing, people tend to harbor fear. If we believe the government has a clear idea where the clusters are, is locking those places down, and is conducting enough tests to know where infections are spreading, then people living outside the infected zones can feel a sense of security. In the U.S., we are facing the opposite. We are told to assume everyone has it (even though it is false for now).

The first type of lockdown has already proven to work. China and South Korea are reporting much fewer new cases while the other day, BBC reported that the lockdown of northern Italian towns has slowed the spread in those places. I doubt that the second type of lockdown will achieve these results.

***

The policy of "triaging" testing is risky. It was announced that Google is building a tool to tell people whether they should get tested.

Triaging is a strategy of targeting. This strategy is familiar to anyone who practices predictive modeling for a living (which I did for much of my career). For example, businesses might "target" their marketing offers (e.g. buy 2 get 1 free) to specific segments of customers predicted to be more likely to utilize such offers, instead of blanketing every customer with these offers.

Targeting involves a tradeoff. It reduces false alarms at the expense of missed sales. This makes sense for marketing offers. It doesn't make sense for public health! To put it bluntly, triaging testing reduces false alarms at the expense of missed positives. Missed positives are the infected people who are told they don't have the coronavirus and therefore don't require testing, and may thus spread it around.

If everyone is tested, then everyone who may test positive will be revealed (assuming the test has an acceptable level of accuracy, what is technically called sensitivity.) By targeting, we limit the tested population to people who are deemed more likely to have the virus, and if this predictive model has any value, the false positive rate should go down. But the false negative rate shoots up. No predictive model is close to perfect, and thus, some infected people will not be tested, and will spread the virus unknowingly. This leads to a false sense of security I outlined this morning - similar to tourists who switched their trips to "safe" countries who report low numbers of cases.

I explained the statistics behind this type of issue in Chapter 4 of Numbers Rule Your World (link) in the context of diagnostic testing. Also, Chapters 3-5 of Numbersense (link) discuss the context of predicting marketing offers.

***

The predictive model to predict who should get the test is destined to be highly inaccurate because all models rely on existing historical data. When a country conducts few tests, there is not much data from which to build a reliable predictive model.

Another harm of a "triaging" strategy is that it introduces bias into the historical record. If the type of people who are deemed "safe" by the model are not tested, they will not be part of the training data and will never be targeted for future testing. In order to build such a model, it is necessary to test some people with low risk of infection.

There is the reality that tests have to be rationed due to limited supply of test kits. But the number of tests conducted in the U.S. is ludicrously low, and the refusal to use WHO test kits is baffling.

***

Aside from the science, policymakers must consider the psychology of our response to uncertainty. Testing brings data which beget calm. Anyone denied testing is asked to live in fear. This is aggravated by the call to assume everyone around you is infected.

***

tldr;

Countries that conduct extensive coronavirus testing are smart even though the high number of reported cases draws unfavorable media attention. The media has focused on horse-race reporting (and maps) of case and fatality counts while ignoring the biases and missing data problems. Countries with low case count are not safe. In fact, places with extensive tests and high case counts are safer. More knowledge and less uncertainty is safer.

Countries doing testing are collecting a pool of valuable data to guide policy-making. Their lockdown policies are well-designed, can be targeted, and have achieved success. The U.S. is playing catch-up, and by not committing to extensive testing, we are forced to assume "everyone has it". This type of lockdown without proper data to guide policy-making will not attain containment; we're hoping to wait out the crisis.

 

 

P.S. [3/17/2020] I've said it. Here's WHO asking for more testing. Locking people down assuming everyone has it isn't going to work as well.

 

 

Comments

[Ken]

According to the South Koreans they didn't have a lockdown. They did put in travel restrictions and advised people to minimise possible exposures. What people did was to do much the same as the Australian partial lockdown. We have been asked to only travel for essential reasons. Many shops are shut because they don't sell essentials. All food is exempt so I bought a very nice seafood spaghetti the other night as a takeaway, as restaurants are closed. In most states you are not allowed to visit anyone else residence except in certain circumstances. It seems to be working. We are doing a lot of testing per capita, but one of the worlds highest for tests per case. Hopefully it will continue to look good.

[Kaiser]

Ken: When is the normal flu season in Australia? I suspect it's not now so with the extra time to prepare and taking it seriously, you'd hope the situation is a bit more under control. Good luck!

[Ken]

Kaiser: We are just about to start flu season. Vaccines for high risk people available now, everyone else in May. The public health people are aware that it will make life more difficult, but the public aren't. The government is making the point that when we get our rates down to a reasonable level it is not back to normal. One of our politicians, who has had coronavirus, pointed out that a dinner party with 20 guests could easily turn into a couple of hundred cases. We have had a bit of trouble with young tourists who have trouble returning home and want to party on. What the public health people are aiming for is to minimise cases with unidentified transmission, and so areas where this is high they are testing anyone who has respiratory infections.