Pollsters, forecasters, and the likes were embarrassed by the Bernie Sanders upset in Michigan Tuesday night. Nate Silver called it among the greatest polling error in primary history. Now, they are struggling to explain the big miss. Recall the polls conducted close to the contest showed a Clinton lead of about 20 percent points. The actual outcome was a gap of 1.5 points, with a million votes cast.
This type of miss is hard to explain because any plausible explanation must deal with these facts:
- The miss of 20 percent points is a huge gap to explain
- The same methodology used in other states has generally been accurate
Take, for example, the claim that polls missed young voters because of a bias toward landlines. This explanation doesn't fly because pollsters use the same methodology in each state. If they are not polling enough young voters in Michigan, they most likely did not poll enough young voters in other states either. Besides, the math doesn't work.
Here's an excerpt from Huffington Post (link):
The pre-election polls also appear to have underestimated turnout among young voters, who overwhelmingly support Sanders. That same NBC/Wall Street Journal/Marist poll showed 18- to 29-year-olds making up 15 percent of the electorate, whereas the exit poll showed 20 percent of the electorate in that age group. In the pre-election poll, Sanders won the age group 74 percent to 25 percent; in the exit poll, his margin was bigger -- 81 percent to 18 percent.
But a back-of-the-envelope calculation shows that this could account for perhaps 7% of the gap only. Based on the data cited, the other 85% of the people in the pre-election poll had given Clinton a 32% edge in order for the average to be 20%. Assume now that 80% of the primary voters gave Clinton a 32% edge and 20% (i.e. the 18- to 29-years old group) gave Sanders a 63% edge. The result is a Clinton edge of 13%, still far from accurate.
Many of the other explanations offered by Huffington Post (link) or FiveThirtyEight (link) has a similar tone. They are plausible but the numbers don't quite work. Perhaps it is combination of factors - the perfect storm. Or perhaps voters did make up their minds or change their minds late in the game.
Also note that all these explanations assume the existence of statistical bias. This source of error is very different from statistical variability - the argument that the error made by the pre-election polls is a one-in-a -hundred year storm.