Big Data, Plainly Spoken (aka Numbers Rule Your World)

How to act like a data scientist 3: deep reading of polls

Saw an interview by Christiane Amanpour on PBS two nights ago, and was highly disappointed by her cherry-picking of poll results, and running down the dead-end path of "electability". These are two topics I have already addressed before but they keep popping up.

First, electability has a circular logic that makes it just a vessel for someone intent on arguing for a particular candidate. Any campaign that argues for its candidate based on electability is not to be trusted. As I said here, the electable one is the one who will win in the general election. That is unknowable. This much was proven by HIllary Clinton's defeat to Donald Trump in 2016, when every pundit predicted her to win, most assigning over 90 percent chance!

So it was disappointing that when Amanpour had a Bernie Sanders's campaign chair on the air, she did not ask about his policies but the entire interview was wasted on discussing electability. This is like asking the candidate - any candidate - do you think you will win? The answer is predictable: I think I will win, otherwise I would not be running.

When Amanpour got that answer, she persisted. That's saying: you think you will win but I don't think so, and here's another reason why. But hello, no one knows the answer until the election is over.

***

The second disappointment was her cherry-picking of poll data. This is a major issue for any casual consumers of the media. The only way to combat this is to go review the poll results yourself, or to find an author that does not take sides, and presents both confirming and diverging evidence. The latter is increasingly hard to find because of click-bait headlines. People click on sensational, misleading leads, and all digital publishers have learned that.

Poll data are very powerful. But the result of a single poll question in a single poll is not very useful. The power of polls come from aggregating results from different questions and from different polls. 

I like this analogy. Imagine the parlour game: how many pieces of candy is in the jar? If we have 20 people in the room, each making a guess, the average of the guesses should come relatively close to the answer.

Aggregating different polls is like getting multiple such jars, and then asking different groups of people to make guesses. N jars, N average guesses. Assuming the manufacturer packs roughly the same amount of candy in each jar, we will obtain not just an estimate of the candy count but also a range of probable values.

So, someone may dishonestly cherry-pick and quote one poll in which, say, Bernie Sanders might lose to Donald Trump in the general election, and declare he is not electable. But there may be 50 other polls that say the opposite. If we gather the results from all 50 polls, we will have an average Sanders-Trump margin, plus also an estimate of the range of possible outcomes. That's the power of aggregating polls.

***

Next, imagine that the candy in the jar comes in three colors (red, green, yellow). Each group makes two guesses: how many overall, and how many reds. If all I'm told is the guess of reds, I can use that data point to infer what the group's guess of all colors would be, but I wouldn't be surprised if that estimate is off. In making that inference, I assume a degree of consistency that probably does not hold in real life. If I was told both the guess of reds and the guess of all colors, then I'd notice the inconsistency, and will want to explain it.

So I thought about this when Amanpour cited the poll result that said that two-thirds are uncomfortable with the label "socialist", and used that to further her argument that Bernie Sanders is unelectable. In the very same poll (see my previous post), the same responders (a) selected Senator Sanders as the top choice for Democratic nominee and (b) selected Sanders by roughly 60/40 in hypothetical head-to-head matchups with either Michael Bloomberg or Pete Buttigieg!

Disappointingly, Amanpour did not mention either of these two additional data points. That's what we all have to guard against. The one poll is not definitive; reasonable people can disagree on how to interpret those three results. The power of polls comes from asking correlated questions and looking at the answers of these correlated questions. A pundit is doing us all a disservice by cherry-picking one of these questions while ignoring the others.

 

P.S. A simpler example of using answers to correlated questions is that the head-to-head matchup questions are asked not just about Bernie Sanders but also about other top Democratic candidates. The reality is that most of these margins against Trump is within the margins of error, meaning that each such hypothetical contest is too close to call. In most cases, Sanders and Biden have a statistically insignificant lead over Trump while many of the other candidates suffer a statistically insignificant lag behind Trump. If you listen to the media, you'd think Sanders is the only candidate who might lose to Trump when in fact, he is one of the two most likely to beat Trump based on not particularly conclusive data so far; that's because they cherry-pick one out of this whole set of correlated questions.

(Here is FiveThirtyEight's compilation of Democrat-Trump margins, in which they have a pollster rating but not the margin of error. Here is RealClearPolitics's compilation, in which they have margin of error but no pollster rating. But don't worry about margin of error: none of these polls have large enough sample sizes for this type of matchup question so almost none of those are statistically significant. Remember the gap has to be larger than twice the reported margin of error to be significant.)

 

How to act like a data scientist 2: what's a mean tweet

One of the enduring themes of the last three Democratic primaries is the accusation that the Twitter supporters of Bernie Sanders are particularly mean. As a data scientist, I like to see the evidence.

***

What data would convince a data scientist that the accusation is real?

First, we need a definition of "mean". (This is tough.)

[Is that a mean tweet? I am ignoring the elephant in the room. Our President is sending mean tweets complete with demeaning nicknames almost every day.]

Second, we need a statistical analysis - not some screenshots. Anecdotes is the antithesis of statistics. We need to count tweets. We can use a statistical sampling approach and pull down random tweets, or in this age of Big Data, some people prefer to pull down every tweet.

Third, we need to define "disproportionate". This part is easy to compute but the mainstream media keeps failing here. It's not the number of mean tweets but it is the proportion of mean tweets. Sanders's supporters will definitely have the highest number of mean tweets - beceause they also send the most tweets.

Sanders's fans send the most tweets because (a) Sanders is the front-runner with the largest base of supporters; (b) Sanders's base is younger and younger people are more likely to be on social media (e.g. compared to 60+ year olds who are least likely to support Sanders); and (c) Sanders's supporters are more likely to believe that mainstream media ignore them, and thus seek alternative platforms to express themselves.

In addition, social-media platforms deliberately exploit the network effect, so that more begets more. Popular tweets are surfaced to the top where they get more attention; more attention leads to more likes and retweets, which is a positive reinforcement loop. [Mainstream media does this too. A single mean tweet by the President is repeated multiple times by each journalist who writes an article about it.]

As the late and great Hans Rosling once said, Big Data is a bunch of numerators without denominators. What we are missing here are the total number of tweets that are sent by each candidate's supporters. Then, you need the number of "mean" tweets by each candidate's supporters. Finally, you can compute the rate (proportion) of tweets that are "mean".

***

Fourth, the computation so far assumes every tweet has equal value. Should our analysis account for the popularity of tweets? When the media cites a mean tweet, is this coming from a pseudonymous account with few followers or from a celebrity with millions? Should tweets with likes count more? Should retweets count more or less?

Lastly, a really good analysis has to deal with the fake data menace. It's hard to ascertain which of the Twitter accounts are humans, which are bots, and which are humans paid by campaigns to tweet.

***

The biases mentioned above cannot be corrected unless the analyst goes beyond social media sources. To really answer the question of which campaign has the meanest supporters, you'd also have to assess the mainstream media, like CNN, MSNBC, and Fox News. The talking heads also say mean things. While the voices of Sanders's supporters are disproportionately heard online, the voices of fans of other candidates dominate traditional media.

A great exercise for a budding data scientist is to come up with a principled way to answer the question of whether supporters of some candidates are "meaner" than others. You'd have to use a range of skills from defining a metric to scraping the data, from labeling to cleaning the data, from correcting bias to weighting the samples.

 

P.S. [3/5/2020] Just saw this last night, Elizabeth Warren being interviewed by Rachel Maddow. She was complaining about mean tweets again. I respect her usually but her statement that "it's simply a fact" when Maddow prompted her saying mean tweets are particularly bad coming from Bernie Sanders's supporters is very disappointing. What makes a fact a fact? Where is her evidence? No one as far as I know have done a proper statistical analysis as outlined here.

Insights from the N-person election model: Nevada and beyond

The Nevada Democratic caucus results are out - sort of. The pundits have all crowned Bernie Sanders a runaway winner. (Some outlets called the race as early as 4-percent of precincts in.)

If you've been following the pundits' logic, as I have on this blog (link), you might be scratching your head here.

After New Hampshire, they told us that Sanders did really poorly since his vote share this year was merely half of what he got in New Hampshire against Hillary Clinton four years ago.

I checked Nevada 2016. Clinton beat Sanders by 53% to 47%. As of Sunday afternoon, with 70 percent of precincts reporting, Sanders took 47% of the votes this year. So, according to the logic previously embraced by these pundits, Sanders did equally well this Saturday as he did in 2016.

Wait a minute - but the 47% which resulted in a loss in 2016 is described as a landslide in 2020.

So what went wrong with that logic? You simply cannot judge the vote shares in an N-person contest (N>2) as if they came from a 2-person contest. I've been pounding this point the last two weeks. Finally, these pundits might see the light.

***

If you can't compare a 2-person contest to a 7-person contest (what Nevada 2020 boiled down to) by directly comparing vote shares, then how can you compare performance across elections with different numbers of contestants?

I've a series of posts that address this question (1, 2, 3). In the last post, I presented a method to turn any vote share in an N-person contest (N>2) to a vote share equivalent in a 2-person contest.

I quickly ran the numbers for the Nevada result. The vote share table of (46%, 20%, 15%, 10%, 5%, 4%, 0.2%) is summarized by the DFER metric, which is the distance of this distribution of votes from the hypothetical even race, in which each candidate wins one-seventh of the vote. The DFER is 0.155. The maximum value is when one candidate wins all 100% of the vote, with DFER = 0.378. So, the result from this election is about 41% of the way between an even race and an extreme race.

A 2-person race that has SDFER of 41% is just above 70/30. So, the Nevada result in 2020 (as of now) is equivalent to a 70%-30% vote split in a 2-person contest, like the one in 2016.

***

With this model of the N-person contest, we can make some general comments about adding candidates to elections.

First, here is a visual analysis of a 3-person contest, compared to a 2-person contest.

First, think about this question: which is the better performance: winning 50% of the votes in a 3-person contest or winning 50% of the votes in a 2-person contest?

This simple question puts the lie to all the nonsense we're hearing about how Bernie Sanders failed to match his 60% vote share in this year's 11-candidate contest relative to the 2-candidate contest in 2016.

Winning 50% in a 2-person contest does not even win the contest but winning 50% in a 3-person contest is a pretty dominant victory. We need math to quantify this advantage. In essence, we want to adjust the 50% vote share upwards to reflect its equivalence in a 2-person contest.

We use the 2-person contest as a reference because it is so easy to understand. The entire contest can be summarized by one number, the vote share of the winner.

The method for translating an N-person contest to a 2-person contest deals with this head-on. As shown below, a crucial step is to turn the set of N vote shares into a single distance metric that can be ordered. Here is a simple diagram of the method:

Back to the pie charts. As you move from left to right on the diagram, the votes of the runners-up are made more even. In the second row, the 50% is split 40-10 while in the other case, it is split 25-25. Is a vote distribution of 50-40-10 more even thatn 50-25-25? Or more one-sided?

This illustrates why when you have more than two candidates, it's not simple to rank competitiveness. According to my model, the 50-25-25 vote distribution is (slightly) less even i.e. more one-sided than the 50-40-10 vote distribution. The way to understand it is that the 50-25-25 distribution has one clear winner while the 50-40-10 has one winner and a close second.

Another reveal from the pie charts. As you look down a column, the winning vote share increases. This increases the SDFER metric, moving closer to the extreme, one-sided case. The impact of the winner taking more share is greater than the impact of whether the runners-up are evenly matched or not.

Thus, if you are the leading candidate, the vanity candidates who each take a percent share of the total are like mosquitoes that you want to swat away. They are small but their stings hurt.

 

How to act like a data scientist 1

Today, I'm starting a series of posts that describe how data scientists may/should react to the use of data and information during this year's U.S. Presidential election cycle. If you are not a data scientist, I'm hoping you'll become smarter at detecting nonsense that comes online or on air. My head often explodes when I hear arguments that violate "numbersense". 

For budding data scientists, one of the greatest skills you need is to make convincing data arguments. These short essays are case studies of applying best practices and spotting mistakes.

***

Last night's debate between Democratic primary candidates in Las Vegas was messy.

But it was also a true debate, in which the candidates were allowed to talk to each other. Democracy is messy. (I wish the questions asked by the hosts could be more uplifting. Instead, many of the questions were mean-spirited and disguised personal attacks. I prefer questions that lead candidates to explain how their policies help Americans.)

Sanders Schooled Host on Reading Polls

The most data-relevant moment in the debate was when the host attacked Bernie Sanders with a poll finding: the host cited this just-released NBC News/WSJ poll, saying that 67% of respondents are uncomfortable with a candidate being a "socialist". And Sanders's response was perfect:

"Who won the poll?" he mused.

He was referring to the headline announcing the poll findings: "Sanders opens up double-digit national lead in primary race". The same poll found that Sanders had 27% support, which is significantly ahead of the next four candidates, all of whom have around 15%. The gap of 12% is larger than twice the margin of error and so this is a clear lead.

If a data scientist on my team said what Sanders did, I would be so proud!

When Sanders asked "who won the poll", the host literally went silent for a few seconds.

One of the ways people misuse survey results is to cherry-pick the answers they like and suppress the answers they don't. The data scientist sees through this right away.

The argument is not that the overall preference data is correct and the socialist label data is incorrect. That would make the same error in reverse. It's that they are either both correct or both incorrect.

The two results can both be correct... if the host thought they were conflicting, that's only because he assumed that the most important factor affecting one's vote preference is the label of a socialist.

In fact, this interpretation of the world is invalidated within the same poll because these same respondents picked Sanders as the front-runner.

If you have numbersense, you may notice a small flaw in the above sentence. Since there are so many candidates, even though Sanders had a clear lead, he only has 27% vote share. It is possible that all of the 27% came from those who are not uncomfortable with "socialist" (after all, 33% is more than 27%). Thumbs up if you saw that.

But... to complete my argument, I can use another pertinent question from the same poll: in a hypothetical head-to-head with Michael Bloomberg, these same respondents chose Sanders 57% to 37%.

The larger lesson here is that polls (and surveys) are wonderful statistical toys. The power of polls is not just in the headline results but the interplay between different results.

[Also note this: Using my methodology outlined here, the vote shares expressed in this NBC News/WSJ poll translate to just 58%-43% vote split in a 2-person race, which is very close to that head-to-head result they found! Nice validation for the methodology. By the way, a 60/40 split in a 2-person race is not a landslide no matter how many times the media mischaracterizes it.

***

Warren Hammered Bloomberg on NDAs

The most memorable moment in the debate was around Michael Bloomberg's use of NDAs to silence women who complained they were sexually harrassed. Elizabeth Warren wanted to know how many such NDAs have been signed, and whether Bloomberg would release the women from these agreements so they could tell the public about what happened.

This attack reminded me of the same against Trump. And that's because these NDAs are standard instruments used to prevent unfavorable information from leaking out. It's, dare we say it, a quid pro quo: the women must sign these documents and keep their mouths shut in order to receive a settlement.

The data angle here is that information (data) is very powerful. Withholding information leaves the data scientist cold.

Politicians have been in the forefront of suppressing information - ranging from Hillary Clinton's servers and deleted emails to Donald Trump's obstruction of investigations, from the Senate Republicans' suppression of witness testimony to these NDAs, from redacting documents to declaring state secrets.

All of these actions prevent information from the public's scrutiny.

This is why as citizens, we must demand more transparency - but also we must demand the right to protect our own information. Right now, if you are using any cloud services (which include most mobile apps), you are entrusting technology companies with private information.

Imagine that during the drafting of the NDAs, Bloomberg's staff used cloud services like Google Docs, then Google (plus possibly third-party vendors used by Google) has copies of those documents in the cloud. The cloud consists of a whole network of servers; the cloud architecture requires everything to be duplicated lots of times so that if one copy is lost, there is another copy available for use.

I don't know that the Bloomberg organization uses Google - since Bloomberg is a tech company, it might not. But we do know that in other situations, like Gmail, Google deploys algorithms that "read" our emails, and extract data out of them. Google even allows third parties (i.e. advertisers) to "read" our emails, e.g. look at electronic receipts in Gmail.

These Big Tech companies hold the key to a lot of potentially valuable data. Can they be trusted to do the right thing?

What did Grandma say? Nothing in life comes free.

The Verge's expose on the Lambda School is a must-read cautionary tale for anyone who is considering - or knows someone who is - enrolling in a private data science or coding bootcamp. It's a story of how Silicon Valley dropped their "free stuff" business model on education, and it scorched the earth. Finally, a journalist is calling out this scheme.

The Lambda School is a pioneer of the "no upfront cost" private training program. Under this scheme, students pay nothing until they land a job. When they get a job (that meets minimum requirements), they are required to send 17% of their pre-tax paycheck to Lambda for a number of years.

This business model isn't really new: it's "vendor financing" used in other industries, like equipment rental or car sales. In effect, Lambda (the vendor) fronts money to students (the customers) to buy its product in the hope that the students would repay it in installments while the product is consumed. This is different from a mortgage in which a third party (bank) lends the money and takes the risk (although the education loan model also exists in the bootcamp world).

Without a doubt, "free stuff" schemes are popular with prospective students. In the last year of running our bootcamp, we've been fielding more and more questions about whether we offer a pay-after-you-get-a-job plan. We politely turned these prospects away.

The scheme sounds too good to be true, and it is. The Verge documents a wide-ranging set of grievances from Lambda alumni, including:

• instructors unable to provide answers to even basic technical problems
• curriculum that "doesn't prepare people to pass even a first-round tech interview"
• courses that are "indistinguishable from any of the free resources available online"
• disorganized, unprofessional and incompetent instruction
• instructors seemingly reading off a script
• assignments seemingly lifted from free online training materials
• class projects uploaded to GitHub that contain obvious errors
• no feedback on class projects that were found to have errors
• some alumni being told by employers they don't qualify for job openings

Is it the fault of the Lambda School? Yes, but not fully.

Students have a choice, and many choose to ignore their grandma's precious advice: nothing in life comes free. Let me explain why the "free stuff" scheme is too good to be true, even for school management.

A majority of the students that the Lambda School enroll will not produce any cash till years out. But cash is required to build the school's brand, market the program, enroll students, pay instructors and staff, rent space, etc. Who's paying for all the expenses while waiting for revenues to arrive? Venture capital.

VCs chase moonshots - they fund near-term losses but demand "hockey-stick" growth that could be turned into billions. If money is lost on every sale, and sales are supercharged, the business is vaporizing cash non-stop. Even with the VC backing, managers have to control the expenditures.

The costs of operating a bootcamp fall into two large categories. Costs such as classroom rental and brand marketing are fixed costs, meaning that they don't grow (or grow slowly) as the number of students increases. Other costs such as teaching assistants, counsellors and supplies are variable costs, which means every incremental student brings the cost up.

In a regular school that charges tuition, revenues also grow as enrollment rises. In the "free stuff" scheme, not only do revenues not scale with class size, but there are no near-term revenues at all with more students.

So these "no upfront tuition" programs prefer to spend on fixed costs rather than variable costs. Fixed cost - say, cost of a building - is distributed over more students, bringing down the per-student expense.

Variable costs however are headaches. Let's say the school offers a mentorship program. Each additional student requires an additional mentoring relationship. Mentors are moonlighting industry professionals, paid on an hourly basis. Every extra student adds mentoring hours, and expenses.

How do managers control variable costs? They can pay a lower hourly rate to the mentors. It shouldn't surprise you that paying less means the mentors are less qualified. Or, they can ration the services. That's why colleges don't provide a career counsellor for every student.

Lambda School students complain that some instructors are reading off a script. This might be another cost containment technique: the school hires a more expensive instructor to create class materials, and then employ minimum-wage teachers to read off the scripts. The Verge reported that the "team leads" - who sound like teaching assistants - are paid a whopping $13 per hour.

Why are teachers paid so little? If you think about it, it's because the school is being run without collecting upfront tuition.

Remember what your grandma told you: nothing in life comes free.

How to compare a two-person and a six-person election? Take 2

Last week, I discussed one approach to answer the question: how to compare vote shares of an N-candidate election (N > 2) to the vote shares of a 2-candidate election? This post covers a second methodology. This new method is more rigorous but also harder to use - which is a common tradeoff in solving math problems, and why I didn't immediately leap here.

This is an important problem for the current Democratic primary elections in the U.S. Because Bernie Sanders is a repeat candidate, the media has carelessly compared his vote share in 2020 to his vote share in 2016. These comparisons are absurd because in 2016, the only other viable candidate was Hillary Clinton. So when he's facing 8-10 more candidates, his vote share is never going to match what he did in 2016. I've kept a whole list of instances in which the media reprinted this fallacy here.

While it's clear that no candidate in 2020 would get close to 60% of the votes, what is not clear is what vote share would be considered comparable? This is the elephant in the room.

***

You can read about the previous method here (link). The advantage of that method is its simplicity. I wanted something that is roughly accurate for any number of contestants (N). The question it solves can be simply stated as "In an N-person contest, if the winning candidate got X% of the votes, what vote split should this be equivalent to in a 2-person contest?"

The trick was to aggregate the (N-1) vote shares of the non-winners by taking the average (non-viable candidates are better combined into one invented candidate).

To use this method, you only need to know the winner's vote share. You don't even need to know the individual vote shares of the other candidates. Such simplicity comes at the price of precision.

***

The second method that I'm discussing in this post uses vote shares of every contestant. There is no need to combine non-viable candidates. The method makes use of the even race reference, which I mentioned in this other post.

To use this method, you first aggregate the vote shares into a single metric, which I call "Distance from Even Race" (DFER). This distance is then standardized by setting the maximum distance to 1. The maximum distance is attained in an extreme one-sided race in which the winner takes 100% of the votes, and no other candidates receive any votes. Then, this standardized distance from even race (SDFER) is mapped to a 2-person race, using the following table:

Since New Hampshire has finalized primary results, I use this data to illustrate how to use the table.

I took the vote shares from CNN (link). They listed 11 entries, one of which is "Other" which presumably aggregated some minor candidates. I treat this as an 11-person contest.

The even race is one in which each candidate gets one-eleventh (9%) of the votes. The distance of the actual results from the even race (DFER) is 0.10. The maximum distance is 0.30 (this is a hypothetical race in which the winner gets 100% of the votes). Thus, the standardized distance is 0.32. This means the actual result is roughly one third of the way between an even race and an extreme race.

The remaining question is what vote split in a 2-person race represents one-third of the way between an even race and an extreme race. From the table, you learn that it is roughly 66% - 34%.

***

Contrast what I just said with the media's hyperventilating over Sanders losing half of his support, and other such nonsense. The vote-share distribution in New Hampshire in the 2020 Democratic primary is equivalent to a 66%-34% split in a 2-person contest.

So, in the same ballpark as 2016, maybe even a bit better.

***

As mentioned before, this second method is more complicated to use: you have to make a separate calculation for each number of contestants (N). The number of candidates determine the vote share in the case of an even race. That in turn determines the distance from even race of the extreme case. And that decides the factor used in standardizing the DFER.

A reverse lookup shows that a 60/40 vote split (what was achieved in 2016 between Sanders and Clinton) is equivalent to a standardized distance of 20%. Only moderately far from an even race for which SDFER = 0%. I explained it before, the 60/40 split in a 2-person race is not a landslide! The even race is 50/50 so the winner got 10 percent more votes than even.

***

With this framework, one can analyze the dynamics of a multi-person race. Let me give you some high-level insights here, and I'll be back to explain them with some examples.

1) Having a group of non-competitive candidates, each taking a small bite out of the apple, makes it very hard for the top dogs to separate themselves.

2) Shuffling vote shares among the top candidates doesn't change the competitive picture by much.

3) Competitiveness is affected by the gap between the winner and the second-runner-up but also by the aggregate vote shares of top candidates versus bottom candidates.

***

The following details are for the nerds.

A given set of vote shares in an N-person race is a point in N-dimensional space. The even-race vote share is another point in the same space, and it's situated at the "center" of the set of feasible vote shares. Any feasible point in that set has an Euclidean distance to the center. Further, each feasible set has N vertices, each corresponding to an elementary vector (0, 0, ..., 1, ...,) with exactly one 1 and all other zeroes. These correspond to extreme races in which one candidate receives all the votes.

The distance from even race metric (distance from center) immediately places subsets of vote-share distributions on equal footing. Any distribution located the same distance from the center is regarded as equally competitive.

We are still in N-dimensional space. The feasible set is symmetric because the dimensions are interchangeable. There are edges in this set that represent shifts of votes from one candidate to a second candidate while all other vote shares are fixed. Each such edge passes through the center as well as an extreme point.

Given a particular vote share distribution, we have a point in the feasible set, and associated with it, a subset of feasible points that have the same distance from the center. Within this subset will be point(s) that fall on one of the edges named above. On such an edge, the center is mapped to 0 and an extreme point to 1. The distance of the given vote share distribution from the center is standardized to a value between 0 and 1.

This standardized distance is then applied to the case of a 2-person race. Vote shares in a 2-person race are points in the x-y plane. The feasible set is the line from (1,0) to (0,1) passing through the center of (0.5, 0.5). The equivalent point is determined by moving a standardized distance from the center towards one of the extreme points.

If anyone wants to help write this up in proper mathematics, let me know.

 

 

Statistical evidence is both powerful and limited: fraud in baseball and lotteries

In Chapter 5 of Numbers Rule Your World (link), I told the story of how Professor Rosenthal used statistical evidence to uncover a cheating scandal in a Canadian lottery. Statistics can be very powerful in identifying suspicious data - in this case, the winning probability of store owners who sell lottery tickets was found to be way higher than normal.

In a textbook presentation, you can simply do the math and compute the relevant probabilities and invoke statistical "significance" arguments. Hey, we've found evidence of cheating. End of story.

In real life, that would just signal the beginning of a long story. The statistical conclusion is merely that "someone cheated" or at best, that "someone cheated almost surely". But you can't catch the cheater(s) based on that evidence! The question left unanswered is: who cheated?

In the example from the book, we know that as a group, store owners cheated. That doesn't mean every store owner cheated. Who cheated? For that, you need old-school shoe leather. You need to find the receipts.

***

I was inspired to write this post after a friend asked me whether the Astros were the only baseball team that stole signs. Here is a summary of the cheating scandal engulfing professional baseball right now.

The statistical argument goes like this: the kind of sign-stealing using video can only happen at home stadiums. Teams who cheated in this way would have a performance advantage in their home games compared to the away games. But of course, every team enjoys a home-field advantage so we're talking about an abnormally high home-field advantage.

Where statistics demonstrates its power is here. We have guidelines for what level of home-field advantage should be considered abnormally high.

The question my friend asked is: If there was a second team who also did remarkably better at home games than away games - similar to the Astros, would that be sufficient evidence to convict that team as well?

Sadly, the power of statistics is also limited. We think cheating might be behind the abnormally high home-field advantage - but we can't be sure! We are not unsure about the observed outcome; the uncertainty concerns the cause of such an outcome.

Similar to the lottery fraud scheme described in Numbers Rule Your World (link). We are very confident that too many winners were store owners but nothing in the data tells us that the cause is cheating! The case didn't get solved until specific store owners were caught. You need to find receipts. 

Unless there is a confession by the cheaters, or a reporter or detective discovering incriminating evidence, we don't know what caused the anomaly in the data. Suspicion is not proof of wrongdoing.

In baseball's case, it was a former Astros pitcher who confessed to the scheme. Without receipts, the cheaters can always offer other explanations for the performance advantage. The most powerful alternative explanation is coincidence.

***

Here's an example of an alternative story. Someone could claim that their hitters started making "superman" poses before they get on the field, and that explained the advantage.

But spinning such stories is to run "story time". We have some data - the performance advantage - that show something. Then, we seamlessly move to a story that has nothing to do with the data just presented. Just because there is a presentation of data doesn't mean the data support the story!

 

 

The media is spreading a fallacy in ever more imaginative ways

I already did the math and it says that a 26% vote share for the winner of a 6-way race is about the same as a 60% vote share in a 2-way race. (The second error is to treat a 60/40 split as a blowout. It is not.) Many people actually intuitively grasp this. It does not take a math degree to recognize that when more people are sharing the pie, each gets less.

The media is trapped in this fallacy and can't get out. In this post, I keep track of all the articles that pretend that vote shares in an 10-plus-person election are equal to vote shares in a 2-person election.

No matter how many times they repeat this talking point, it is still wrong.

***

Benjamin Wallace-Wells (New Yorker): Bernie Sanders Leads a Jumbled New Hampshire Primary

In 2016, Sanders won sixty per cent of the vote in New Hampshire; this time, his totals were less than half that.

Hunter Woodall, Nicholas Riccardi (Chicago Tribune): A victory for Bernie Sanders, but

And Pete Buttigieg, the former mayor of South Bend, Indiana, again made a case to be that choice with another impressive performance, winning votes across broad demographic groups, by running close to Sanders, who defeated Hillary Clinton by 22 percentage points in 2016.

Sam Stein, Hanna Trudo (Daily Beast): Bernie Sanders Wins New Hampshire

The Vermont independent scored a far smaller percentage of the electorate than in 2016, when he breezed to victory in New Hampshire with 60 percent of the vote

Ruby Cramer (Buzzfeed): Bernie Sanders Won the New Hampshire Primary

Sanders, who spoke to supporters on Tuesday night at the Southern New Hampshire University field house, finished with a smaller margin of victory than his finish four years ago against Hillary Clinton

Ryan Lizza (Politico): Sanders ekes one out

Independents were a larger share of the electorate, but they did not break nearly as decisively for Sanders as they did in 2016. He received support from just 29 percent of self-described independents this time, as opposed to 73 percent (!) in 2016.

[Note: Subsetting to Independents does not do away with the fallacy.]

John Whitesides, Amanda Becker (Reuters): Sanders narrowly beats Buttigieg

For Sanders, who won New Hampshire in 2016 with 60% of the vote against eventual nominee Hillary Clinton, the results offered new momentum but not the overwhelming win he had hoped for given his history in the state. Exit polls showed he only won about two-thirds of his 2016 primary supporters.

[Note: if he kept all those supporters, he would have obtained similar vote share as in 2016. It's still the same fallacy at play here.]

Washington Post: Bernie Sanders wins New Hampshire

While he has emerged as one of the candidates everyone else is chasing — an unfamiliar spot for a politician much more accustomed to running as an insurgent underdog — his narrow winning percentage was on course to be historically low, below Jimmy Carter’s 28.6 percent in 1976.

[Note: Wiki says there were five candidates in 1976. There are over 10 candidates in 2020.]

Laura Bronner (FiveThirtyEight): Liveblog

[Note: FiveThirtyEight is still the class act here, and they have said many things that I agree with. This post is from their LiveBlog, so it is not as well thought out. This is a standard plot comparing vote shares in one election versus another. But it only works when the race has similar number of candidates. If Sanders's margin is going to sit above the diagonal, he probably would be running away with the nomination already.]

***

Meanwhile, this lady who voted in New Hampshire stated the obvious. (link) She even claimed she was so upset by the media repeating this fallacy that she voted for Sanders.

How to compare a two-person and a six-person election?

The last post left hanging the question of how to assess the 6-person race. The two-person race is easy because one person's gain is the other person's loss. When you have four more contestants, the math gets a lot more complex.

It took me a while to figure out how to do it. The following table gives the main story:

Recall from my last post, I'm treating the New Hampshire Democratic primary as a six-person race by lumping all the minor candidates into one invented person.

The above table says that if the winner in New Hampshire gets 17% of the votes, the contest is evenly matched so that is equivalent to 50/50 in a two-person race.

If the winner in NH gets 20%, the contest is equivalent to a 56/44 split in a two-person race.

Using this table, I estimate that if the race this year is similar to that in 2016, when Sanders got 60% and Clinton got 38%, then we should see the winner this year get 25% of the votes.

Yes, that is only 25% of the votes. That means if the winner gets 40% of what Sanders got in 2016, s/he would have done just as well!

This is all due to the expansion of the field. The key thing to bear in mind is this: in a two-person race, if the winner gets 5 percent more votes, the loser gets 5 percent less, widening the margin by 10 percent. This is not the case in a 6-person race.

If the leader gets 5 percent more votes, the bottom 5 competitors split the 5-percent loss. On average, each suffers a one-percent loss, so that the margin between the top two closes by 6 points. (In practice, it will likely expand by fewer than 6 points, because the less competitive candidates will take a larger loss than the runner-up.)

***

How did I come up with the table of numbers?

I use the metric of winner vote share divided by the average vote share of the other candidates. In a 2-person race, there is only one other candidate so it's just the vote share of the winner vs vote share of the loser.

In the 6-person race, the denominator is the average vote share of five candidates. So, in the case in which the winner earned 25% of the votes, the other 5 shared 75%, which is 15% on average. The ratio is 25/15 = 1.67. In a 2-person race, 63/38 has a ratio of 1.67.

***

Let me know what you think of this, and if you know of other ways of addressing the question.

In the meantime, please stop the media from saying silly things like "[the winner] got half of what Sanders got in 2016" or "the winning margin was much smaller than in 2016".

 

P.S. I'm writing this note with about 84 percent precincts reporting and Sanders is leading with a vote share of 26%. If it stays this way, then using my metric, he did just about the same as in 2016.

How to spot false narratives on election news

In these heady days of the 2020 U.S. presidential elections, it's hard to turn on the news shows and not be bombarded with false narratives. None of us have enough time to check everything. I'll be posting notes for things that rile me enough to verify. I hope you spot some yourself too.

But I'm not covering outright lies, for which there are fact-check services, and I won't be evaluating gut feelings, which are just opinions.

I will limit my comments to two categories of false narratives that exhaust my patience:

(a) story-time narratives: these are gut feelings garnished with data. The conclusion draws on a creative interpretation (or cherry-picking) of the data, which means someone else could easily produce other conclusions from other interpretations (or picking other cherries);

(b) inappropriate comparisons: context is extremely important when interpreting data, and comparing one number to another is a powerful way to add context. Sometimes, the wrong comparisons are offered, which muddies the data.

A good example of story time was early on during the slow drip of Iowa caucus results:

Because partial data showed Pete Buttigieg in the lead in most counties in Iowa, the CNN analyst keeps saying this proves that he can build a broad coalition across the country.

The data as stated is not false (although bear in mind population density). But the second part of the sentence is not a direct consequence of the cited data. Multiple hard-to-prove assumptions* are needed to bridge the gap. But the presence of the data makes the second part appear palatable, and if it's almost bed time, you might just have let your guards down. Hence story time.

What are these assumptions?

That the partial data selectively released are representative of the full data (see this post)

That in every county, the "leader" is significantly ahead of all other candidates (otherwise, there are several leaders)

That winning the Democratic primary in deep Republican counties suggests strength against a Republican opponent in the general election (if the proposed coalition were to contain moderate Republicans)

That the rural-urban dimension defines the coalition more than race (since Iowa's population is 90 percent white)

Notice that all of these assumptions can be checked against relevant historical data but the analyst basically demanded our blanket acceptance.

***

The second type of false narratives rests upon a poor choice of comparables. Take the following line from a recent column by Frank Bruni in the New York Times (link):

[Bernie Sanders] essentially tied Pete Buttigieg in the Iowa caucuses, and is slightly favored to win the primary on Tuesday in New Hampshire, where he trounced Hillary Clinton four years ago.

(Overall, I liked Bruni's balanced assessment of Sanders' potential in the general election.) Similar misleading statements flooded the airwaves the last week. Just last night, a guest on CNN opined:

Sanders is likely to win in New Hampshire [the next day] by a small margin but in 2016, he beat Hillary Clinton by 22 points; that was a two-candidate race but still.

Someone else said:

He won 60% of the votes here in 2016 and he won't get anywhere near that this time.

I am about to say something silly but still!

***

Let's check the underlying data.

New Hampshire primaries results from Wikipedia, 2016

Sanders 60%

Clinton 38%

Others 2%

#1 - #2: Sanders's winning margin 22%

 

Emerson College Poll, New Hampshire, Feb 5- Feb 6, 2020

Sanders 32%

Buttigieg 23%

Warren 13%

Biden 11%

Klobuchar 9%

Top 5 = 88%

All others, including undecideds 12%

Assuming these percentages hold out on election day, the gap between the first two finishers is 9 percent. The commentators compare 9% to the Sanders-Clinton gap of 22%, suggesting that Sanders is weaker this year than four years ago. It's not a two-candidate race anymore but still!

Is it realistic for the winner of this year's contest featuring five competitive challengers to either (1) earn the 60 percent vote share that Sanders won four years ago or (2) produce the 22 percent vote margin that Sanders achieved four years ago?

Let's play around with some scenarios.

(A)

If the top dog gets 60%, and the vote margin is again 22%, then the #2 would have earned 38% of the votes, which leaves peanuts for any other viable candidate, so it is clearly an impossible target.

Rank	Vote Share
#1	60%
#2	38%
#3	2%
#4
#5
Others

#1-#2 vote margin 22%

(B)

If the top dog gets 60% and the "all other" candidates received 10% as predicted by the polls (rounded down for easy computation), then #2-#5 candidates would be splitting up 30% of the votes, that is, 7-8 percent each. In this scenario, the top dog's winning margin would be 52 percent. That's also absurd.

Rank	Vote Share
#1	60%
#2	8%
#3	8%
#4	7%
#5	7%
Others	10%

***

Look at the 2016 results again. The 60/40 split is a comfortable win but actually not a blowout. In an even match, both candidates would get 50%. The 60/40 split is a 10-point shift from an even match. A 70/30 split is a 20-point swing from even, and so on.

I just went quickly to Wikipedia and learned for example, that on Super Tuesday 2016, out of 12 contests, 6 ended up with a margin greater than 65%. while 8 with a margin greater than 60%. So even the factual part of the statement - that the 60/30 win constituted some kind of big win - is wrong! It was a win but not a "trouncing", as Bruni wrongly claimed.

It felt like a huge win only because he and other analysts compared a 2-person contest to a 6-person contest! And some were unapologetic about it. But still!

Wait, did I just say it's a 6-person contest?

Yes, we can think of the top 5 contenders plus an invented sixth contestant, who captures "All Others." According to the above poll, this sixth person should get about 12% of the vote. The top five splits a pool of 88%, which means that if evenly matched, they each should win about 17% of the vote.

We can put the numbers in perspective, by computing the excess (or deficit) against the reference of an even match.

	Vote share	Excess
Sanders	32%	+15
Buttigieg	23%	+6
Warren	13%	-4
Biden	11%	-6
Klobuchar	9%	-8
Others	12%	-5

(There is a 2% imbalance due to rounding.)

This analysis suggests the top two candidates are the only two doing better than even. For anyone to get more vote share, s/he has to take share from one or more of the other candidates. This  zero-sum property keeps these numbers from going extreme.

***

To sum up, Bernie Sanders did not trounce Hillary Clinton in 2016. He won comfortably but the winning margin was below average. Also, because we have a six-person contest in 2020, relative to a two-person contest, it is impossible for the winner to earn a vote share or margin even close to what happened four years ago. The counter-intuitive conclusion is: a blowout win in 2020 will require less vote share and smaller voting margin than in 2016!

Corrected narrative: Bernie Sanders's win in 2016 was comfortable but not a blow-out. The polls also indicate a comfortable win although a tie or narrow loss is within the margin of error. He might get a blowout win even if his vote share or margin is much lower than in 2016 because this contest has at least four other viable candidates.