Big Data, Plainly Spoken (aka Numbers Rule Your World)

Significance of early voting

Andrew Sullivan highlights the significance of early voting in U.S. Presidential Elections (link). His second quote (from The Fix) requires a visit to your statistical adviser.

Andrew considers this an "important point":

Democrats generally vote early more than Republicans. In the five 2012 swing states where a 2008 early voting breakdown is available, Democrats voted early more than Republicans in all five. Even in a very good GOP year in 2010, Democrats voted early more often than Republicans in North Carolina, Iowa and Nevada.

I'm afraid many will misinterpret this statement. What really matters is which party succeed in pushing more wavering voters to vote for them early, and thus nail down those votes. It would give Democrats no advantage if those early voters are hardcore party-line voters. The hardcore Democrat can cast his/her vote on Election Day or any day prior but it's still one sure vote.

In other words, not every early voter is alike. Early voters who are independents, previously undecided, wavering, etc. are more useful than early voters who are party-line voters.

There may be an adverse selection effect, in the sense that undecided voters are less likely to vote early since, well, they haven't made up their minds. So the people who are voting early are really the ones you have in your bag.

***

For me, the significance of early voting are these two points :

• Early votes skew later voting propensity (which is why I don't think it's a good idea). This is similar to the East Coast/West Coast issue: by the time people in the West Coast wake up, there would already be results from the East Coast, or at the minimum, results from exit polls. This can have the effect of causing some people to stay home or other people to rush out to vote. It also allows campaign managers to take remedial action before Election Day. 
• Early votes is an insurance policy. It guards against gaffes leading up to Election Day.

The tyranny of algorithms

This WSJ book review on "The Tyranny of Algorithms" (link) is well worth reading for those interested in how computer algorithms are used in business and government. I agree with most of what this author has to say.

There is an elephant in the room whenever it comes to discussing computer algorithms, particularly highly automated ones. Almost all such algorithms are inaccurate. They are inaccurate for many reasons, the most important of which is that human behavior is fickle. The inaccuracy could be shockingly high. Take an algorithm that tries to predict what banner ad you would click on. It would not be surprising that the algorithm will direct a million ads to be displayed, and only 30 clicks would be obtained. This outcome will be extremely exciting to the person who designed the algorithm because without the algorithm, there would only be 10 clicks. So the algorithm tripled the click rate. That's the glass-half-full view that is out there. The glass-half-empty view is that close to a million wrong decisions are ignored while we celebrate 20 right ones.

This more introspective point of view is also missing from the WSJ piece.

 

A no-difference conclusion arises after positive findings are explained away

The recent articles proclaiming the death of organic foods could as well be written by big agribusinesses. The  reporting raises a whole bunch of questions. Here's the NYT on the study (link) and here's Time (link).

Here are some points to ponder:

1. There is no new research. Time's headline gets this wrong even though in the article itself, the author says the research is a review of existing studies (so-called meta-analysis) but on this point, the NYT is clear, the study is an "extensive examination of four decades of research".

2. Despite the headline claim that organic food is no more nutritious and no less contaminated than non-organic food, both articles devote the majority of words to reporting the statistics that contradict the claim. We learn that organic food has less pesticide residue, less bacteria, more omega 3 fatty acids, more phosphorous, more phenols, etc.

3. The researchers then tell us why none of those negative findings can be trusted. This runs the gamut from the difference is not practically significant because the levels are too low to matter, or because the levels are so high it doesn't matter, or in the case of bacteria, because bacteria will be killed during cooking.

4. A particularly shocking claim is the one cited by NYT to dismiss the more phenols finding: 'While the difference was statistically significant, the size of the difference varied widely from study to study, and the data was based on the testing of small numbers of samples. “I interpret that result with caution,” Dr. Bravata said.' Statistical significance was invented for one purpose, which is to deal with sampling variability, and sample size. It makes little sense to blame those two factors after testing for significance, otherwise they shouldn't bother testing for significance. In addition, since this is a meta-analysis, they could easily have thrown out studies in which the sample size is considered too small, or the sampling variability is too high -- doing so upfront will reduce the suspicion of cherrypicking.

5. In fact, these researchers seem to already have hit the tip of an iceberg. At the end of the NYT article, they admitted that they erroneously omitted a study that showed organic food had a benefit. Given that the selection of which studies to include in a meta-analysis is such a crucial step, it does not help their credibility to have committed this error.

6. As far as I can tell, none (or almost none) of these studies are randomized controlled trials (RCT) in which the health benefits of organic foods and non-organic foods are assessed. This is quite surprising given the assertion that the two groups of foods are the same. Given that it is likely that no one would die from such a study, there just isn't any reason why we should rely on observational studies. In fact, according to Time, only 17 of the over 200 studies included in the meta-analysis measured "health outcomes"; all the other studies compared "nutritional content".

7. The bulk of both articles accepts the underlying logic that consumers buy organic food because of better nutrition or lower contamination. I know many people who buy organic food to support small businesses and others would prefer to know that the chicken they are eating were able to run around rather than get cooped up inhumanely. This is a fundamental flaw of a study that is being interpreted as a reason not to buy organic food. The NYT reader would have to read all the way to the end of the article to see this point raised.

***

For those nutritional scientists out there, do you know why RCTs aren't popular in your field? Almost everything we read from the newspapers report from longitudinal observational studies that really do not seem robust enough, resulting in all kinds of contradictionary conclusions, and embarrassing reversals.

Speaking analytics

I was a guest on the Analytically Speaking series, organized by JMP. In this webcast (link, registration required), I talk about the coexistence of data science and statistics, why my blog is called "Junk Charts", what I look for in an analytics team, the tension between visualization and machine algorithms, two modes of statistical modeling, and other things analytical.

Reading Networked Life: 20 Questions and Answers

I highly recommend the soon-to-appear book, Networked Life: 20 Questions and Answers by Mung Chiang (pre-order at Amazon) for anyone who wants to know more about the technologies -- technical philosophies, in some cases -- underlying the businesses of high-tech superstars like mobile phone companies, Google, Netflix, Amazon, Wikipedia, Skype, and so on. If you like Wired magazine but want a bit more meat, and are not intimidated by a few equations, then you will like this book.

Networked Life is written for an introductory-level college course, and it’s written in the textbook-lite style. Readers need to know basic mathematical language but the book does not give full mathematical proofs. What appeals to me is the three-step exposition: each chapter starts with a posed question (e.g. “How does Google rank webpages?”), and then gives firstly a general explanation emphasizing big concepts, and secondly, a deeper answer containing more mathematical details, and finally, advanced topics and references to technical materials. As my readers will notice, I used a similar strategy, placing the denser materials in the Conclusion chapter of my book.

Mung is also offering a free online course via Coursera, based on this book. (link)

For disclosure, Mung and I have known each other since high school. He’s currently Professor of Electrical Engineering at Princeton, where I took many courses in my undergraduate years. I thoroughly enjoyed his book, and provided feedback on several chapters, but would have highly recommended it even if he's not a friend. His academic home page is here, and the course web page is here.

The 20 topics in the book cover a wide variety of areas in applied mathematics. Throughout, Mung points out the big concepts that underly many apparently dissimilar applications. A book like this tends to get bogged down in mathematical details but this one always looks at the forests, more so than at the trees.

One of the chapters I reviewed concerns the topic of six degrees, the idea that anyone is linked to anyone else in the world within six connections. The starting point is Milgram’s experiment, in which he sent letters to about 300 people, asking them to forward to a stockbroker with an address in Boston, Mass., or to someone who might know the stockbroker. The median number of steps from sender to recipient was 6, hence six degrees. Mung quickly takes readers to the two key issues: the obvious one of whether short paths exist within these networks, and the deeper question of how one might discover such paths, given the large number of longer paths that exist, or one’s knowledge of only the local topology of the network.

This discussion leads to an introduction to the modern study of constructive "models" that assume random jumps from one point to another, demonstrating how these simple models provide insights into the difficult problem (or not, depending on your perspective!)

I not only enjoy that discussion but it also guided me to think about Milgram's experiment in a different light. The average of 6 is computed only over the 60 or so paths that were successful. About 70 percent of the letters never reached the stockbroker. So the original analysis suffers from survivorship bias. It is only accurate to describe those relationships in which the two parties could eventually discover each other. But the parties that could not discover each other are likely to be separated by more than 6 connections, and if they were accounted for, the median steps would surely be more than 6.

One of the pleasures of Mung's book is its tendency to provoke one's thinking. Hope you'll enjoy it as much as I did.