Carl Bialik (The Numbers Guy, WSJ) is the first business journalist to talk about the "margins of error" around the economics statistics that are a staple of the business media empire.
In a previous post ("Agreeing to disagree"), I documented how the entire business press decided to ignore the Census Bureau's warnings about the unreliability of many statistical estimates based on very large margins of errors, due to very small samples. This is the article that wasn't deemed fit to print.
My example was the retail sales growth statistics, where the previous month's "revision" was larger than the reported growth in the current month. Bialik showed the same problem with unemployment estimates, new home sales data, and so on. Specifically, he cited a margin of error for the February unemployment to be between -200,000 and +511,000. The width of this margin of error is such that it is utterly useless. Did it stop the business press from flushing out stories about an unemployment decline? No.
(Glossary: The 90% confidence level means that 1 out of 10 times, the true value of the economic statistic will fall outside the advertised margin of error. I'm warning you: do not count how many statistics are issued by the Census Bureau each week.)
***
I happen to also be reading Dean Starkman's "How Could 9,000 Business Reporters Blow It?" (Mother Jones), recommended by Brad Delong. Starkman attributed the problem to: "Increasingly, business coverage has addressed its audience as investors rather than citizens", and the "triumph of Wall Street insiderism." He has a dim view of the future, given the dire straits print journalism finds itself at the moment.
I prefer Felix Salmon's simpler explanation, and a very courageous act to flog himself in public:
journalists, myself included, read economics papers: we generally have no ability or inclination to try to understand the details of the formulae and regression analyses, so we confine ourselves to reading the stuff in English, and work on the general assumption that the mathematics is reasonably solid.
Comments