Saw an interview by Christiane Amanpour on PBS two nights ago, and was highly disappointed by her cherry-picking of poll results, and running down the dead-end path of "electability". These are two topics I have already addressed before but they keep popping up.

First, **electability has a circular logic** that makes it just a vessel for someone intent on arguing for a particular candidate. Any campaign that argues for its candidate based on electability is not to be trusted. As I said here, the electable one is the one who will win in the general election. That is unknowable. This much was proven by HIllary Clinton's defeat to Donald Trump in 2016, when every pundit predicted her to win, most assigning over 90 percent chance!

So it was disappointing that when Amanpour had a Bernie Sanders's campaign chair on the air, she did not ask about his policies but the entire interview was wasted on discussing electability. This is like asking the candidate - any candidate - do you think you will win? The answer is predictable: I think I will win, otherwise I would not be running.

When Amanpour got that answer, she persisted. That's saying: you think you will win but I don't think so, and here's another reason why. But hello, no one knows the answer until the election is over.

***

The second disappointment was her **cherry-picking of poll data**. This is a major issue for any casual consumers of the media. The only way to combat this is to go review the poll results yourself, or to find an author that does not take sides, and presents both confirming and diverging evidence. The latter is increasingly hard to find because of click-bait headlines. People click on sensational, misleading leads, and all digital publishers have learned that.

**Poll data are very powerful. But the result of a single poll question in a single poll is not very useful. The power of polls come from aggregating results from different questions and from different polls. **

I like this analogy. Imagine the parlour game: how many pieces of candy is in the jar? If we have 20 people in the room, each making a guess, the average of the guesses should come relatively close to the answer.

Aggregating different polls is like getting multiple such jars, and then asking different groups of people to make guesses. N jars, N average guesses. Assuming the manufacturer packs roughly the same amount of candy in each jar, we will obtain not just an estimate of the candy count but also a range of probable values.

So, someone may dishonestly cherry-pick and quote one poll in which, say, Bernie Sanders might lose to Donald Trump in the general election, and declare he is not electable. But there may be 50 other polls that say the opposite. If we gather the results from all 50 polls, we will have an average Sanders-Trump margin, plus also an estimate of the range of possible outcomes. That's the power of aggregating polls.

***

Next, imagine that the candy in the jar comes in three colors (red, green, yellow). Each group makes two guesses: how many overall, and how many reds. If all I'm told is the guess of reds, I can use that data point to infer what the group's guess of all colors would be, but I wouldn't be surprised if that estimate is off. In making that inference, I assume a degree of consistency that probably does not hold in real life. If I was told both the guess of reds and the guess of all colors, then I'd notice the inconsistency, and will want to explain it.

So I thought about this when Amanpour cited the poll result that said that two-thirds are uncomfortable with the label "socialist", and used that to further her argument that Bernie Sanders is unelectable. In the very same poll (see my previous post), the same responders (a) selected Senator Sanders as the top choice for Democratic nominee and (b) selected Sanders by roughly 60/40 in hypothetical head-to-head matchups with either Michael Bloomberg or Pete Buttigieg!

Disappointingly, Amanpour did not mention either of these two additional data points. That's what we all have to guard against. The one poll is not definitive; reasonable people can disagree on how to interpret those three results. **The power of polls comes from asking correlated questions and looking at the answers of these correlated questions. A pundit is doing us all a disservice by cherry-picking one of these questions while ignoring the others. **

P.S. A simpler example of using answers to correlated questions is that the head-to-head matchup questions are asked not just about Bernie Sanders but also about other top Democratic candidates. The reality is that most of these margins against Trump is within the margins of error, meaning that each such hypothetical contest is too close to call. In most cases, Sanders and Biden have a statistically insignificant lead over Trump while many of the other candidates suffer a statistically insignificant lag behind Trump. If you listen to the media, you'd think Sanders is the only candidate who might lose to Trump when in fact, he is one of the two most likely to beat Trump based on not particularly conclusive data so far; that's because they cherry-pick one out of this whole set of correlated questions.

(Here is FiveThirtyEight's compilation of Democrat-Trump margins, in which they have a pollster rating but not the margin of error. Here is RealClearPolitics's compilation, in which they have margin of error but no pollster rating. But don't worry about margin of error: none of these polls have large enough sample sizes for this type of matchup question so almost none of those are statistically significant. Remember the gap has to be larger than twice the reported margin of error to be significant.)

## Recent Comments