In the days ahead, you're going to start hearing a lot of commentators try to say something smart about testing accuracy - and fail. This is a very confusing area, with multiple redundant sets of terminology used by practitioners from different backgrounds and no one willing to standardize to the same jargon. It is also an area where the drive for simplicity has doomed many explainers.
Take this explainer published by the New York Times on coronavirus diagnostic testing, written by a Yale doctor. (Tip from Antonio R.) The author made the following statement:
The good news is that the tests appear to be highly specific: If your test comes back positive, it is almost certain you have the infection.
The bad news is "specificity" and PPV are different concepts.
I am working on a bunch of posts at the moment so I will point you to my 2010 post that covers this subject for now. The two key numbers you should look for are PPV and NPV.
Another key takeaway is that the accuracy of a test cannot be reduced to a single number. To understand this material, you must be capable of holding two or three numbers in your head at the same time. I hope that is not too much to ask!
It's actually easy to mix up the terminology, even when you know your stuff. So I wouldn't read too much into the published mistake.
If you read Numbers Rule Your World (link: Chapter 4 covers steroids testing and terrorist prediction) or Numbersense (link: the Marketing chapters cover the accuarcy of targeting models), this should just be a review.
***
The insufficient accuracy of testing has become a popular excuse by governments that have continued to pursue a test-less policy. There is no perfect test. All tests face a trade-off between two types of errors. Reducing one increases the other. A serious critique of testing accuracy needs to involve actual numbers and target levels. What level of accuracy do these officials consider as adequate? And what are those numbers for the tests that they have rejected?
That's all I'll say on this issue for now, and I'll return to this subject soon. For now, here is the post that will give you everything you need to know.
Comments