Reading Everything is Obvious by Duncan Watts

In his book, Everything is Obvious (Once You Know the Answer): Why Common Sense Fails, Duncan Watts, a professor of sociology at Columbia, imparts urgent lessons that are as relevant to his students as to self-proclaimed data scientists. It takes only nominal effort to generate narrative structures that retrace the past, Watts contends, but developing lasting theory that produces valid predictions requires much more effort than common sense.

Watts’s is a perfect foil to the pop sociology books such as The Tipping Point and Outliers. Many a time while reading Everything is Obvious, I was reminded of Malcolm Gladwell by a passage here or there but the lessons drawn by Watts are decidedly more complex and nuanced.

Several chapters in the book begin with stories that could easily have come from Gladwell’s pen. In Chapter 9, Watts recalled the night an NYPD veteran, James Gray, eventually killed a family while driving drunk.

What happened next isn’t completely clear, but the record indicates that as Officer Gray drove north on Third Avenue, under the Gowanus Expressway overpass, he ran a red light. Definitely not good, but also perhaps not a big deal. On any other Saturday evening, he might have sailed right on through and gotten safely to Staten Island…

Using this case study, Watts shows how society cares more about outcomes than processes. For to replay the scenario in one’s mind’s eye is to realize that the unfortunate victims would have been safe if any number of events would have intervened causing Officer Gray to be “late” to the scene by just one second.

I imagine Gladwell would have embraced the James Gray story and folded it into one of his crowd-pleasing narratives wrapped around some simple truth about humans. Gladwell’s universe of truths such as “stars are made, not born.” One anecdote, a couple of research papers, a trifle of interviews - Gladwell’s ammo has dazzled and swayed the public, who crave broad-brush descriptors of our world.

Thankfully for this reader, Watts does not go there. The milieu of Everything is Obvious is far more complex. Watts resists reducing sociological phenomenon to simple terms. His stories tend to leave readers hanging; we learn that researchers have not quite figured out the goings-on. The James Gray story, Watts explains, is about how bad deeds can have harmless outcomes, good deeds can have bad outcomes, bad outcomes can arise from harmless deeds, and so on. Watts challenges us to think more broadly, behind the outcomes unto the processes. On this very issue, I wish sportscasters would read Everything is Obvious, and then they might cease calling for the coach’s head when he ordered a pass play that failed on the last play of a football game, with his team inches from the goal line. The coach made a bad call because the play failed but it would have been celebrated as the clinching moment on the coach’s resume had a touchdown been the outcome.

Watts is not afraid to admit when sociologists have no answers. Such indeterminacy may frustrate some readers, or startle them. Popularizers of science, following the tradition of Gladwell, have invented a world high on consensus and low on uncertainty. In Everything is Obvious, Watts, who converted to sociology from physics, repeatedly describes failed theories and false starts.

The book also works well as an overview of the field of sociology, its key research areas and avenues. Many of the topics covered are thoroughly modern, such as the rise of crowd-sourced data, the running of large-scale experiments, the complexity of systems, and the futility of chasing after a “grand unifying theory” of sociology (and by extension, economics, and other social sciences).

Watts and his associates made valuable contributions to some of these subjects. He holds a research position at Yahoo! Research Labs at the time the book was written, and thus has access to industrial-grade systems and datasets. Several chapters of the book are concerned with the nature of making predictions. I particularly enjoy the materials about choosing one’s battles - the idea that some things are just not meant to be predicted. This important lesson has been laid to waste in the Big Data age. There are too many data scientists who issue pointless predictions just because they have datasets that can be fed to algorithms.

I share Watts’s obsession about measuring the accuracy of predictions. One of his research papers investigated “predictive markets,” concluding that they were not meaningfully better than experts, contradicting the unwarranted hype about predictive markets (and its crowd-sourcing ilk). The only surprise here is the miserly mention by the media, who otherwise eagerly publish any hearsay promotion about wisdom of crowds.

The title of the book is worth another moment’s thought. Everything is obvious, Watts says, when viewed in hindsight. The hyperventilation after each “terrorist” attack is instructive. Inevitably, a reporter files a note, disclosing that the accused killer has left a warning on his Facebook account. Such hindsight is taken as evidence of law enforcement’s failure to secure the nation. If right now, we were to flag every Facebook message that contains a threat of violence, we might have to investigate thousands, if not hundreds of thousands, of people, nearly all of whom will never become real killers. The menacing Facebook note becomes known only after its writer has committed a heinous act.

By its nature, Watts’s book does not read as quickly as your average pop sociology book. It contains quite a bit of philosophising, and urges readers not to take things at face value. Common sense is helpful, but limited.

Comments

[AB&T Diesel Service]

Brilliant !!

No doubt, after reading the book, that everything is obvious once you know what happened

In a very entertaining and educating way, Duncan makes his case about how good we are to deceive ourselves in order to make sense of things we just don't know.