The professional disease of data enthusiasts

The episode around the EPA disclosure rules points out a professional disease that affects many data enthusiasts. We think that data should settle matters when most of the world do not operate that way at all. The EPA Commissioner is playing into this weakness by complaining that the science is not solid because there isn't enough disclosure or precision.

The reality is that most of the world believe lots of things without requiring disclosure or precision.

Example #1: superfoods and most foods supposedly good for you. Vitamin water, coconut water, fruity water, oat milk, cashew milk, almond milk, etc. etc. Lots of people change their eating habits based on hearsay, and no one is demanding more disclosure or precision.

Example #2: organic foods, grass-fed cows, wild fishes, etc. Lots of people do not question the vendors or restaurants usage of those adjectives. However, quite a few studies have shown that, for example, most fish marketed as wild are falsely advertised as wild. There isn't an outcry for more disclosure or precision.

Example #3: technologies such as Waze. There is no way anyone can verify if you really did save 7.4 minutes of travel time on a given trip but the voice of Waze loudly pronounced that you have taken the fastest route and saved 7.4 minutes. Fanatics believe strongly that the number is correct, but have no evidence. There isn't an outcry for more disclosure of how the estimated time saving is estimated, and validated.

Example #4: web usage metrics. It's pretty well-known that most web metrics are easily manipulated, whether we are talking about Instagram followers, Facebook friends, Twitter followers, ad clicks, email opens, etc. It's suspicious when you go to a hotel or air ticket reservation site and they claim 354 other people are looking at the same page, or that 89 people have just booked the same trip, whether those numbers are real or not. Many site users might be hurried into buying based on believing those numbers. But there isn't an outcry for more disclosure.

Example #5: despite a lot of effort by Andrew and I to debunk the "science" of the power pose, lots of people continue to believe it. There is a whole industry of consultants offering training in this area. There isn't an outcry for more disclosure. And this goes for a lot of other similar types of research findings.

I can go on and on to build out this list.

All these examples show that most people do not need more data or more disclosure to believe in something. When they don't want to believe, they may ask for more data. The request becomes a stalling tactic. This tactic is particularly effective on data enthusiasts, including most scientists, because we tend to think that more data would or should convince the skeptics. The ugly truth is that it usually doesn't.

Comments

[William Sims]

I think your examples are strawmen as I don't have any weight to "wild caught" or "grass-fed", I have friends help determine best drive times, and I don't care what a web site says about visitors. You are looking at the dumbest people and trying to make an argument that nobody needs data if they don't.

I'm not arguing for what the EPA is doing, but what you are doing is ignoring the problem in the scientific community where a lot of data is unavailable. To me, that is not the best science.

[Andrew Gelman]

William:

I don't think Kaiser's examples are strawmen. All the examples he gives are big business. Or, at least, medium-sized business. People are spending money on all these things, and I don't think it's just the dumbest people. And Kaiser's not arguing that "nobody needs data"; he's arguing that a lot of decisions get made without the use of data or even any apparent sense on people's part that data could be relevant.

[Kaiser]

William: I am definitely not arguing that "nobody needs data". I'm pointing out that for lots of people, the amount of data required as evidence is very low. Lots of people I know whom I'd not call dumb buy some or all of those things listed, and believe these products do good based on marketing mumbo-jumbo. I love data - but I realize that for many others, giving them even more data will not change their minds. Agree with Andrew on the delineation: In the business and political settings, asking for more data/more precision becomes the strawmen! Pure science is a different matter, when players are more honest. Applied science is a mess because industry is not held to the same disclosure standards as academics; the EPA should work on that problem first.

[jlbriggs]

William - I think you've created the perfect example of a strawman in your comment :)