« The inutility of raw data 2 | Main | Why tests are not perfect »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Antonio Rinaldi

"False negatives are considered less damaging than false positives in the context of antibody testing because these people with antibodies would be taking unnecessary caution. It's not completely harmless since they might suffer from lost income. False positives are feared as they might have a false sense of security and take unknowing risks, and get infected."

What do you think about the policy to not inform the tested people about their test result? Would it be feasible? useful? ethically acceptable?

Dean Abbott

regarding the bias in the sample, all you wrote are certainly possible. I believe he said (in an interview I heard from him) that they corrected the sample by geo location; they didn't interview participants to break down all of the demographic groups. To me, that would cover a good pct of the concerns but not all. And, of course, FB doesn't cover everyone--no sampling methodology does--but it has greater reach than any other digital communications method. That stated, I'm not sure the biases mentioned, while possible or even likely, would impact the results. In other words, I'm not sure what effect bias in interest in getting a free test would have on positive rates (do people who want a free test have a greater risk of being infected? It's not clear to me they would, but maybe I'm just not thinking of them).

He also stated in the interview I heard from him that we are far from herd immunity. That's interesting but not a part of this study and from what he said, not a part of what is of interest to him.

Last, we don't know the false positive or false negative rates of this kind of test. I would presume that a test of this sort would strive to bias in favor of false positives (so you can rule out those FP later...FN are more dangerous). But given the relative sizes of the population, because the number of potential FNs is so much larger, I'd be more concerned about undercounting infections than overcounting (the tail of the distribution of uninfected is much broader). But that's conjecture because we don't know the FP or FN rates.

I wish we did far more of this. In spite of the limitations of this study, it provides valuable data. If we did 100 of these, just like political polling aggregates, we would get a much better sense of the true extent of infections, and by extension, the true mortality rate of this virus in the US.

Kaiser

AR: If antibody testing is about getting people back to work, then you have to inform people of the results. I have to understand the context a bit better for why one should withhold the results.

DA: I didn't specify the directions of the biases because as you pointed out, it's hard to hazard a guess. I'm sure you know, it's dangerous to assume there is no bias, or that the various biases balance out, just because we don't know the direction/magnitude of them. If we don't know, how can we know that correcting the demographics will correct the biases?

The comments to this entry are closed.

Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

See my Youtube and Flickr.
Numbers Rule Your World:
Amazon - Barnes&Noble

Numbersense:
Amazon - Barnes&Noble

Search3

  • only in Big Data

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here

Courses

R Fundamentals, Principal Analytics Prep

Numbersense: Statistical Reasoning in Practice, Principal Analytics Prep

Applied Analytics Frameworks & Methods, Columbia

The Art of Data Visualization, NYU

Signed copies at McNally-Jackson, NYC

Excerpts: Numbersense Ch. 1, 7, 8. NRYW

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Community