I left a comment on one of Andrew Gelman's recent posts about Malcolm Gladwell (link). This post discusses briefly a review of Gladwell's recent book.
A commenter ("Haile") made the following defense of Gladwell:
My point is that many of these criticisms are based on Gladwell’s failure to present rigorous statistical evidence of arguments that are not statistical in nature in the first place.
when (if) reading Gladwell, it’s time to put down the statistical hammer, because virtually nothing that he talks about has anything to do with phenomena that are statistical in nature.
The argument is simply that if the author is merely describing one person, a sample size of one, and not attempting a generalization, then statistics has no standing in this conversation.
In my response, I will gloss over the important issue of whether Gladwell is making general comments about human psychology or sociology in his books. I will consider what it means to study an individual sample by itself.
It turns out that reducing the scope of the data makes the analysis harder, not easier. This may feel counterintuitive.
In my response to the comment, I used one of the examples from the new book. The greatness of David Boies, the famous lawyer, is argued to be a result of his dyslexia, which is billed as a "desirable difficulty". Here is what I said:
Let’s examine the theory that Boies is a great lawyer because he is dyslexic. How would you prove/disprove such a statement? All we observed are two separate things: Boies is a great lawyer; and Boies has dyslexia. Well, maybe a third thing, which is that Boies said he became a great lawyer because he is dyslexic. A correlation does not a causation make. First-person speculation is not science either.
If Gladwell is attempting a generalization, he could expand his data set to multiple Boies, and hope to find stronger evidence for the theory. If he is forced to use N=1 samples, his task is more formidable, not less.
Note that saying Boies benefited from his dyslexia is very different from saying Boies is X feet tall. The latter is purely descriptive and can be verified easily. The former is not descriptive, and a hard problem.
Andrew's other post on the Gladwell genre is really good. It captures many of my own sentiments.