Big Data, Plainly Spoken (aka Numbers Rule Your World)

What did Grandma say? Nothing in life comes free.

The Verge's expose on the Lambda School is a must-read cautionary tale for anyone who is considering - or knows someone who is - enrolling in a private data science or coding bootcamp. It's a story of how Silicon Valley dropped their "free stuff" business model on education, and it scorched the earth. Finally, a journalist is calling out this scheme.

The Lambda School is a pioneer of the "no upfront cost" private training program. Under this scheme, students pay nothing until they land a job. When they get a job (that meets minimum requirements), they are required to send 17% of their pre-tax paycheck to Lambda for a number of years.

This business model isn't really new: it's "vendor financing" used in other industries, like equipment rental or car sales. In effect, Lambda (the vendor) fronts money to students (the customers) to buy its product in the hope that the students would repay it in installments while the product is consumed. This is different from a mortgage in which a third party (bank) lends the money and takes the risk (although the education loan model also exists in the bootcamp world).

Without a doubt, "free stuff" schemes are popular with prospective students. In the last year of running our bootcamp, we've been fielding more and more questions about whether we offer a pay-after-you-get-a-job plan. We politely turned these prospects away.

The scheme sounds too good to be true, and it is. The Verge documents a wide-ranging set of grievances from Lambda alumni, including:

• instructors unable to provide answers to even basic technical problems
• curriculum that "doesn't prepare people to pass even a first-round tech interview"
• courses that are "indistinguishable from any of the free resources available online"
• disorganized, unprofessional and incompetent instruction
• instructors seemingly reading off a script
• assignments seemingly lifted from free online training materials
• class projects uploaded to GitHub that contain obvious errors
• no feedback on class projects that were found to have errors
• some alumni being told by employers they don't qualify for job openings

Is it the fault of the Lambda School? Yes, but not fully.

Students have a choice, and many choose to ignore their grandma's precious advice: nothing in life comes free. Let me explain why the "free stuff" scheme is too good to be true, even for school management.

A majority of the students that the Lambda School enroll will not produce any cash till years out. But cash is required to build the school's brand, market the program, enroll students, pay instructors and staff, rent space, etc. Who's paying for all the expenses while waiting for revenues to arrive? Venture capital.

VCs chase moonshots - they fund near-term losses but demand "hockey-stick" growth that could be turned into billions. If money is lost on every sale, and sales are supercharged, the business is vaporizing cash non-stop. Even with the VC backing, managers have to control the expenditures.

The costs of operating a bootcamp fall into two large categories. Costs such as classroom rental and brand marketing are fixed costs, meaning that they don't grow (or grow slowly) as the number of students increases. Other costs such as teaching assistants, counsellors and supplies are variable costs, which means every incremental student brings the cost up.

In a regular school that charges tuition, revenues also grow as enrollment rises. In the "free stuff" scheme, not only do revenues not scale with class size, but there are no near-term revenues at all with more students.

So these "no upfront tuition" programs prefer to spend on fixed costs rather than variable costs. Fixed cost - say, cost of a building - is distributed over more students, bringing down the per-student expense.

Variable costs however are headaches. Let's say the school offers a mentorship program. Each additional student requires an additional mentoring relationship. Mentors are moonlighting industry professionals, paid on an hourly basis. Every extra student adds mentoring hours, and expenses.

How do managers control variable costs? They can pay a lower hourly rate to the mentors. It shouldn't surprise you that paying less means the mentors are less qualified. Or, they can ration the services. That's why colleges don't provide a career counsellor for every student.

Lambda School students complain that some instructors are reading off a script. This might be another cost containment technique: the school hires a more expensive instructor to create class materials, and then employ minimum-wage teachers to read off the scripts. The Verge reported that the "team leads" - who sound like teaching assistants - are paid a whopping $13 per hour.

Why are teachers paid so little? If you think about it, it's because the school is being run without collecting upfront tuition.

Remember what your grandma told you: nothing in life comes free.

How to compare a two-person and a six-person election? Take 2

Last week, I discussed one approach to answer the question: how to compare vote shares of an N-candidate election (N > 2) to the vote shares of a 2-candidate election? This post covers a second methodology. This new method is more rigorous but also harder to use - which is a common tradeoff in solving math problems, and why I didn't immediately leap here.

This is an important problem for the current Democratic primary elections in the U.S. Because Bernie Sanders is a repeat candidate, the media has carelessly compared his vote share in 2020 to his vote share in 2016. These comparisons are absurd because in 2016, the only other viable candidate was Hillary Clinton. So when he's facing 8-10 more candidates, his vote share is never going to match what he did in 2016. I've kept a whole list of instances in which the media reprinted this fallacy here.

While it's clear that no candidate in 2020 would get close to 60% of the votes, what is not clear is what vote share would be considered comparable? This is the elephant in the room.

***

You can read about the previous method here (link). The advantage of that method is its simplicity. I wanted something that is roughly accurate for any number of contestants (N). The question it solves can be simply stated as "In an N-person contest, if the winning candidate got X% of the votes, what vote split should this be equivalent to in a 2-person contest?"

The trick was to aggregate the (N-1) vote shares of the non-winners by taking the average (non-viable candidates are better combined into one invented candidate).

To use this method, you only need to know the winner's vote share. You don't even need to know the individual vote shares of the other candidates. Such simplicity comes at the price of precision.

***

The second method that I'm discussing in this post uses vote shares of every contestant. There is no need to combine non-viable candidates. The method makes use of the even race reference, which I mentioned in this other post.

To use this method, you first aggregate the vote shares into a single metric, which I call "Distance from Even Race" (DFER). This distance is then standardized by setting the maximum distance to 1. The maximum distance is attained in an extreme one-sided race in which the winner takes 100% of the votes, and no other candidates receive any votes. Then, this standardized distance from even race (SDFER) is mapped to a 2-person race, using the following table:

Since New Hampshire has finalized primary results, I use this data to illustrate how to use the table.

I took the vote shares from CNN (link). They listed 11 entries, one of which is "Other" which presumably aggregated some minor candidates. I treat this as an 11-person contest.

The even race is one in which each candidate gets one-eleventh (9%) of the votes. The distance of the actual results from the even race (DFER) is 0.10. The maximum distance is 0.30 (this is a hypothetical race in which the winner gets 100% of the votes). Thus, the standardized distance is 0.32. This means the actual result is roughly one third of the way between an even race and an extreme race.

The remaining question is what vote split in a 2-person race represents one-third of the way between an even race and an extreme race. From the table, you learn that it is roughly 66% - 34%.

***

Contrast what I just said with the media's hyperventilating over Sanders losing half of his support, and other such nonsense. The vote-share distribution in New Hampshire in the 2020 Democratic primary is equivalent to a 66%-34% split in a 2-person contest.

So, in the same ballpark as 2016, maybe even a bit better.

***

As mentioned before, this second method is more complicated to use: you have to make a separate calculation for each number of contestants (N). The number of candidates determine the vote share in the case of an even race. That in turn determines the distance from even race of the extreme case. And that decides the factor used in standardizing the DFER.

A reverse lookup shows that a 60/40 vote split (what was achieved in 2016 between Sanders and Clinton) is equivalent to a standardized distance of 20%. Only moderately far from an even race for which SDFER = 0%. I explained it before, the 60/40 split in a 2-person race is not a landslide! The even race is 50/50 so the winner got 10 percent more votes than even.

***

With this framework, one can analyze the dynamics of a multi-person race. Let me give you some high-level insights here, and I'll be back to explain them with some examples.

1) Having a group of non-competitive candidates, each taking a small bite out of the apple, makes it very hard for the top dogs to separate themselves.

2) Shuffling vote shares among the top candidates doesn't change the competitive picture by much.

3) Competitiveness is affected by the gap between the winner and the second-runner-up but also by the aggregate vote shares of top candidates versus bottom candidates.

***

The following details are for the nerds.

A given set of vote shares in an N-person race is a point in N-dimensional space. The even-race vote share is another point in the same space, and it's situated at the "center" of the set of feasible vote shares. Any feasible point in that set has an Euclidean distance to the center. Further, each feasible set has N vertices, each corresponding to an elementary vector (0, 0, ..., 1, ...,) with exactly one 1 and all other zeroes. These correspond to extreme races in which one candidate receives all the votes.

The distance from even race metric (distance from center) immediately places subsets of vote-share distributions on equal footing. Any distribution located the same distance from the center is regarded as equally competitive.

We are still in N-dimensional space. The feasible set is symmetric because the dimensions are interchangeable. There are edges in this set that represent shifts of votes from one candidate to a second candidate while all other vote shares are fixed. Each such edge passes through the center as well as an extreme point.

Given a particular vote share distribution, we have a point in the feasible set, and associated with it, a subset of feasible points that have the same distance from the center. Within this subset will be point(s) that fall on one of the edges named above. On such an edge, the center is mapped to 0 and an extreme point to 1. The distance of the given vote share distribution from the center is standardized to a value between 0 and 1.

This standardized distance is then applied to the case of a 2-person race. Vote shares in a 2-person race are points in the x-y plane. The feasible set is the line from (1,0) to (0,1) passing through the center of (0.5, 0.5). The equivalent point is determined by moving a standardized distance from the center towards one of the extreme points.

If anyone wants to help write this up in proper mathematics, let me know.

 

 

Statistical evidence is both powerful and limited: fraud in baseball and lotteries

In Chapter 5 of Numbers Rule Your World (link), I told the story of how Professor Rosenthal used statistical evidence to uncover a cheating scandal in a Canadian lottery. Statistics can be very powerful in identifying suspicious data - in this case, the winning probability of store owners who sell lottery tickets was found to be way higher than normal.

In a textbook presentation, you can simply do the math and compute the relevant probabilities and invoke statistical "significance" arguments. Hey, we've found evidence of cheating. End of story.

In real life, that would just signal the beginning of a long story. The statistical conclusion is merely that "someone cheated" or at best, that "someone cheated almost surely". But you can't catch the cheater(s) based on that evidence! The question left unanswered is: who cheated?

In the example from the book, we know that as a group, store owners cheated. That doesn't mean every store owner cheated. Who cheated? For that, you need old-school shoe leather. You need to find the receipts.

***

I was inspired to write this post after a friend asked me whether the Astros were the only baseball team that stole signs. Here is a summary of the cheating scandal engulfing professional baseball right now.

The statistical argument goes like this: the kind of sign-stealing using video can only happen at home stadiums. Teams who cheated in this way would have a performance advantage in their home games compared to the away games. But of course, every team enjoys a home-field advantage so we're talking about an abnormally high home-field advantage.

Where statistics demonstrates its power is here. We have guidelines for what level of home-field advantage should be considered abnormally high.

The question my friend asked is: If there was a second team who also did remarkably better at home games than away games - similar to the Astros, would that be sufficient evidence to convict that team as well?

Sadly, the power of statistics is also limited. We think cheating might be behind the abnormally high home-field advantage - but we can't be sure! We are not unsure about the observed outcome; the uncertainty concerns the cause of such an outcome.

Similar to the lottery fraud scheme described in Numbers Rule Your World (link). We are very confident that too many winners were store owners but nothing in the data tells us that the cause is cheating! The case didn't get solved until specific store owners were caught. You need to find receipts. 

Unless there is a confession by the cheaters, or a reporter or detective discovering incriminating evidence, we don't know what caused the anomaly in the data. Suspicion is not proof of wrongdoing.

In baseball's case, it was a former Astros pitcher who confessed to the scheme. Without receipts, the cheaters can always offer other explanations for the performance advantage. The most powerful alternative explanation is coincidence.

***

Here's an example of an alternative story. Someone could claim that their hitters started making "superman" poses before they get on the field, and that explained the advantage.

But spinning such stories is to run "story time". We have some data - the performance advantage - that show something. Then, we seamlessly move to a story that has nothing to do with the data just presented. Just because there is a presentation of data doesn't mean the data support the story!

 

 

The media is spreading a fallacy in ever more imaginative ways

I already did the math and it says that a 26% vote share for the winner of a 6-way race is about the same as a 60% vote share in a 2-way race. (The second error is to treat a 60/40 split as a blowout. It is not.) Many people actually intuitively grasp this. It does not take a math degree to recognize that when more people are sharing the pie, each gets less.

The media is trapped in this fallacy and can't get out. In this post, I keep track of all the articles that pretend that vote shares in an 10-plus-person election are equal to vote shares in a 2-person election.

No matter how many times they repeat this talking point, it is still wrong.

***

Benjamin Wallace-Wells (New Yorker): Bernie Sanders Leads a Jumbled New Hampshire Primary

In 2016, Sanders won sixty per cent of the vote in New Hampshire; this time, his totals were less than half that.

Hunter Woodall, Nicholas Riccardi (Chicago Tribune): A victory for Bernie Sanders, but

And Pete Buttigieg, the former mayor of South Bend, Indiana, again made a case to be that choice with another impressive performance, winning votes across broad demographic groups, by running close to Sanders, who defeated Hillary Clinton by 22 percentage points in 2016.

Sam Stein, Hanna Trudo (Daily Beast): Bernie Sanders Wins New Hampshire

The Vermont independent scored a far smaller percentage of the electorate than in 2016, when he breezed to victory in New Hampshire with 60 percent of the vote

Ruby Cramer (Buzzfeed): Bernie Sanders Won the New Hampshire Primary

Sanders, who spoke to supporters on Tuesday night at the Southern New Hampshire University field house, finished with a smaller margin of victory than his finish four years ago against Hillary Clinton

Ryan Lizza (Politico): Sanders ekes one out

Independents were a larger share of the electorate, but they did not break nearly as decisively for Sanders as they did in 2016. He received support from just 29 percent of self-described independents this time, as opposed to 73 percent (!) in 2016.

[Note: Subsetting to Independents does not do away with the fallacy.]

John Whitesides, Amanda Becker (Reuters): Sanders narrowly beats Buttigieg

For Sanders, who won New Hampshire in 2016 with 60% of the vote against eventual nominee Hillary Clinton, the results offered new momentum but not the overwhelming win he had hoped for given his history in the state. Exit polls showed he only won about two-thirds of his 2016 primary supporters.

[Note: if he kept all those supporters, he would have obtained similar vote share as in 2016. It's still the same fallacy at play here.]

Washington Post: Bernie Sanders wins New Hampshire

While he has emerged as one of the candidates everyone else is chasing — an unfamiliar spot for a politician much more accustomed to running as an insurgent underdog — his narrow winning percentage was on course to be historically low, below Jimmy Carter’s 28.6 percent in 1976.

[Note: Wiki says there were five candidates in 1976. There are over 10 candidates in 2020.]

Laura Bronner (FiveThirtyEight): Liveblog

[Note: FiveThirtyEight is still the class act here, and they have said many things that I agree with. This post is from their LiveBlog, so it is not as well thought out. This is a standard plot comparing vote shares in one election versus another. But it only works when the race has similar number of candidates. If Sanders's margin is going to sit above the diagonal, he probably would be running away with the nomination already.]

***

Meanwhile, this lady who voted in New Hampshire stated the obvious. (link) She even claimed she was so upset by the media repeating this fallacy that she voted for Sanders.