For those who have found it tough to keep up with Andrew Gelman's prolificacy, here are some brief summaries of several recent posts:
On people obsessed with proving the statistical significance of tiny effects: "they are trying to use a bathroom scale to weigh a feather—and the feather is resting loosely in the pouch of a kangaroo that is vigorously jumping up and down." (link)
[I left a comment. In Big Data, we have thousands, no millions, of kangaroos jumping out of sync, but still one feather.]
On people testing a zillion things hoping to land on the one that "works": "I suggest you should fit a hierarchical model including all comparisons and then there will be no need for such a corrections." (link)
[This is something Andrew has been advocating for a while. The idea is that such models have in some sense a built-in correction for the multiple comparisons problem. Unfortunately, some researchers are wrongly interpreting Gelman. I recently read a report that cites Gelman's paper as evidence that "multiple comparisons" is not a real problem, and then proceed to fit dozens of regressions without any mechanism to control for multiple comparisons!]
On when to throw out all your data, the lot of it: "Sure, he could do all this without ever seeing data at all—indeed, the data are, in reality, so noisy as to have have no bearing on his theorizing—but the theories could still be valuable." (link)
On why some professors should be fired (link)
Comments