For a detailed introduction to the book, you can now read the sample chapter, which the publisher kindly made available here.
***
Statisticians will be the sexy job in the next 10 years, so said Google's chief economist, Hal Varian. In many ways, 2009 was a banner year for business analytics. IBM, a giant of corporate IT and consulting, shelled out $1.2 billion to buy SPSS, one of the most well-known vendors of analytics software. The #1 analytics software provider, SAS, still privately-owned, enjoyed record earnings (see co-founder John Sall's take here, in which he mentions my book). The Netflix contest -- a showcase for how statistics influence our entertainment choices -- came to a thrilling conclusion. Amazingly, the R software, long a favorite of statisticians, got some exposure in the New York Times.
But -- if you ask anyone who works in a statistics or analytics function in industry, you'll find that there is a severe shortage of properly-trained workers. It turns out statistical thinking is a specialized skill that does not come naturally to us. This was first observed by psychologist Daniel Kahneman: yes, the same Kahneman who won the Nobel Prize for his foundational work on behavioral economics (both lines of research done in collaboration with Amos Tversky).
***
Based on the results of a series of experiments, Kahneman and Tversky hypothesized that our natural way of thinking is non-statistical. Put differently, they found that statistical thinking is often non-intuitive. Consider the following question:
Consider two hospitals, one with 15 child births per day, the other with 45. Which hospital will have more days with 60% or more male births? Or will both experience the same frequency?
Most people believed that the frequency would be the same, since each birth should be roughly equally likely to be male as female, regardless of day or hospital.
Statistical thinking leads to the realization that the smaller hospital will have a greater number of days with large excess male births because the variability is much higher (this also means the smaller hospital will have more days with large excess female births). Variability is higher when the sample size is smaller.
This is no mere toy problem: Howard Wainer's evisceration of the rationale behind the small schools movement relied on the same idea. (See my discussion here.) In Chapter 1 of Numbers Rule Your World, I look at why statisticians obsess about the notion of "variability".
***
Not long ago, statisticians could hardly dream of an article in the New York Times titled "For Today's Graduate, Just One Word: Statistics". For graduates, and anyone else interested in number-crunching, recognize that statistical thinking is learnt and developed over time. A good degree is nice but practical experience is crucial.
Recent Comments