The episode around the EPA disclosure rules points out a professional disease that affects many data enthusiasts. We think that data should settle matters when most of the world do not operate that way at all. The EPA Commissioner is playing into this weakness by complaining that the science is not solid because there isn't enough disclosure or precision.
The reality is that most of the world believe lots of things without requiring disclosure or precision.
Example #1: superfoods and most foods supposedly good for you. Vitamin water, coconut water, fruity water, oat milk, cashew milk, almond milk, etc. etc. Lots of people change their eating habits based on hearsay, and no one is demanding more disclosure or precision.
Example #2: organic foods, grass-fed cows, wild fishes, etc. Lots of people do not question the vendors or restaurants usage of those adjectives. However, quite a few studies have shown that, for example, most fish marketed as wild are falsely advertised as wild. There isn't an outcry for more disclosure or precision.
Example #3: technologies such as Waze. There is no way anyone can verify if you really did save 7.4 minutes of travel time on a given trip but the voice of Waze loudly pronounced that you have taken the fastest route and saved 7.4 minutes. Fanatics believe strongly that the number is correct, but have no evidence. There isn't an outcry for more disclosure of how the estimated time saving is estimated, and validated.
Example #4: web usage metrics. It's pretty well-known that most web metrics are easily manipulated, whether we are talking about Instagram followers, Facebook friends, Twitter followers, ad clicks, email opens, etc. It's suspicious when you go to a hotel or air ticket reservation site and they claim 354 other people are looking at the same page, or that 89 people have just booked the same trip, whether those numbers are real or not. Many site users might be hurried into buying based on believing those numbers. But there isn't an outcry for more disclosure.
Example #5: despite a lot of effort by Andrew and I to debunk the "science" of the power pose, lots of people continue to believe it. There is a whole industry of consultants offering training in this area. There isn't an outcry for more disclosure. And this goes for a lot of other similar types of research findings.
I can go on and on to build out this list.
All these examples show that most people do not need more data or more disclosure to believe in something. When they don't want to believe, they may ask for more data. The request becomes a stalling tactic. This tactic is particularly effective on data enthusiasts, including most scientists, because we tend to think that more data would or should convince the skeptics. The ugly truth is that it usually doesn't.