Bittersweet

There is this bittersweet moment that all statisticians and data scientists experience in any successful project. It's the moment where we finally cleared the forest, and saw the light. It's the moment for claiming our prize -- a simple, coherent story explaining the data mess. Now that we've got the map of the forest, it's simple to retrace the path to the light. We wonder how we took all those wrong turns. The taste is bittersweet.

***

I thought about all that while hearing the know-it-all pundits shouting at public health officials, epidemiologists, and politicians for not acting fast enough, for ignoring the dangers, for letting the pandemic grow so rapidly.

One simply cannot look at the coronavirus growth curve today and complain about a decision made two weeks ago; that was two weeks ago when we didn't have the two weeks of data we do today. Everything always feels simple after you've seen the data you didn't have.

***

Consider two countries A and B. Country A gets its first case, and eventually the virus arrives at country B. When country A made decisions related to containment, it had only five days of data. The cost of containment is likely economic collapse. It chose to wait and see.

Then, the case counts started to jump, and country A took more drastic action after 10 days. In this period, Country B started seeing its first cases. It faced exactly the same dilemma as Country A. Case counts were low by day 5 and growing slowly. Drastic measures would destroy the economy. It chose a wait-and-see approach.... until the cases began to rise sharply.

Country B blamed country A for not moving fast enough to contain the virus, thus allowing it to spread. But on day 5, it was not certain, not even close, with the limited initial data, that the contagion would be become serious. The only reason why Country B felt wronged is because the virus got there later. Even with the benefit of more data, Country B mimicked Country A's decisions. In fact, one can say Country B's action validated Country A's earlier decision to stall.

***

When we finally emerge from this crisis, we will reach that bittersweet moment. We've conquered the virus, but we'd lament that we didn't do it sooner. That's the deceptive power of hindsight.

 

Comments

[Michael Droy]

Hindsight is still a long way ahead of us.
Is destroying the economy the right solution? - no one has a clue - or rather the argument is largely based on QALYs and no one is talking about QALYs so either the calculations aren't being done or more likely they are too brutal for politicians to dare use them.

No one has any real data.
Death rates are death with Covid-19 tested, not death by Covid-19 and certainly not all deaths with Covid-19 which for all we know may go back to last summer or before.
Infection rates are only infection rates of patients with symptoms who demand a test - most people with symptoms are expected to self-isolate and shut up. We don't know infection rates at any place or any time within a factor of 10 (other than emergency sites like Wuhan where the guessing range is down to a factor of 2).
Outbreaks are defined by hospitals overwhelmed rather than infection rates, and tests are only common after outbreaks.
On a top down level the evidence across countries is that Covid-19 has not lead to more deaths overall (though that does not mean it can't be shown at local levels).

And likewise there is no proper data that confirms that Wuhan or China is over it. (In Wuhan the economy leading to much lower levels of pollution may have been a major factor absent from even the testing data)

Suppose you attempt a proper investigation. Assume 0.1% infection. You'd need 100k randomly controlled tests to get 100 cases - nobody is able to do that many tests - and certainly not of a truly random sample.

There is supposedly an anti-body test - though we have no idea when this will be widely available of if it will work. If it is cheap enough to uncover all those who have got over the virus it will tell us real info if used in scale. Then we will have data.

[Noah Silbert]

You're right that hindsight feels deceptively clear, and some arguments about any particular country's (lack of) response is based on this, but it's also worth noting that there were a lot of people arguing for an early response (very) early in the process, i.e., without the benefit (and false clarity) of hindsight.

Also, I don't understand what you mean by "In fact, one can say Country B's action validated Country A's earlier decision to stall." How does Country B's later decision to stall validate Country A's earlier decision to stall?

[Kaiser]

NS: What I mean is: by choosing to follow a similar path as country A, country B basically said, "Well, A is right. Even with the extra time and data, we will follow A's lead and do the same thing."
So in this sense, country B has validated what country A did (at least up to the time Country B had to decide). Now, it is still possible that both countries made bad choices. We won't know until later by comparing them to country C that chose a different path.

[Antonio Rinaldi]

Read on twitter:
Everything we do before a pandemic will seem alarmist. Everything we do after will seem inadequate

[Ken]

The problem with these arguments is that everything that has happened has been perfectly predictable since early February. It was known from Chinese studies that the baseline transmission rate (number of new cases that each case generates) was probably about 2.5 but could be higher. It has since been shown to be about 3 in most Western countries but higher in Spain and Italy. Social life changes things. That means without interventions there is a doubling of cases about every 3 to 4 days and that is what happened in Wuhan until their lockdown took effect. We also knew that mild cases were difficult to distinguish from influenza, so unless someone actually does a lot of testing, nobody has a clue how many cases there are until people start dying in unusual numbers. So it was known that unless we intervened in some way there was going to be a huge epidemic. The doubling every 3-4 days is repeated in every country in the period when the epidemic is uncontrolled. Doublings are faster in the early period when the public health people are catching up, so the number of real cases doesn't increase a that rate only the observed. We also knew roughly the effectiveness of interventions and the answer is reasonable but not enough. What is needed is good testing and case tracking. It can work because South Korea and Taiwan have made it work. In Australia it seems to be working with a partial lockdown and we haven't implemented some of the control measures, probably because of political reasons. One of the characteristics of successful Asian countries seems to be the belief that things are done properly. Old wealthy nations generally get away with doing things poorly.

[Kaiser]

Ken: "Perfectly predictable" is according to your assumptions though. Someone with different assumptions may disagree. I was pointed to some projections for the U.S. (not January but yesterday's update) and its estimate for peak hospital beds needed has a confidence interval from 2K to 200K. Plus, the Chinese studies were not available till recently. What would you have said if the situation in China were not under control today? I do agree with your last point. Loss aversion is real. The richer a country, the more there is to lose by shutting things down. The other thing one can infer is how much business interests run the respective governments.

[Ken]

2 to 200k is probably dependent of the effect of interventions and is actually fairly small. If they let the epidemic just run, a reasonable estimate might be 60% infected, one third asymptomatic, gives 40% and of those 20% would require hospitalisation, so 8% of the population would require hospitalisation of 328 million Americans or 26million. Everything was known in those figures except the asymptomatic proportion which may be as high as 50% in early February. You could read http://www.imperial.ac.uk/mrc-global-infectious-disease-analysis/covid-19/report-3-transmissibility-of-covid-19/ The only question has been how effective are interventions. It was very obvious from Wuhan that once they lose control of the contact tracing no nice interventions will work well enough to make a difference. The true transmission rate in Western country is probably about 3, unenforced social distancing may reduce that by 25%, but a transmission rate of 2.25 still gives a decent epidemic. A value of about 3 is the same as SARS but I expect SARS is more controllable because someone notices that they have SARS whereas for COVID19 they may just think it is flu. The problem with all of this was that they should have known that they had a moderately infectious disease, they didn't have the capacity to do enough testing, they didn't do anything important to restrict social contact and then when it was obviously out of control hoped for a miracle until it was looking like a disaster when they locked down. Then as major changes take about a week or two to affect the growth curve it was an even bigger disaster. In Australia we went to partial lockdown a little earlier than we possibly needed but it is much better than slightly late. Is there anyway I can send you some graphs other than twitter?

[Kaiser]

Ken: Somewhat amusingly, in the FAQ of the IHME forecast, they clearly say that they assume full adherence to strong social distancing rules so it's a best case scenario in this sense.
What is not discussed enough so far is the variance in risk tolerance amongst people. We completely accept that when discussing financial investments but somehow we think everyone must respond to a set of uncertain forecasts in the same way. How do we bring people with different tolerances together?