Big Data, Plainly Spoken (aka Numbers Rule Your World)

E.coli outbreak in Germany still unsolved

To understand what's going on behind the scenes in Germany - where about 16 people have died from E.coli related diseases, you can look at Chapter 2 of Numbers Rule Your World. Here is a decent report by the Wall Street Journal, summarizing what is currently known.

***

As in the E.coli outbreak investigations done here, the health body in Germany/Europe is hurriedly trying to determine which food product has spread the infections. They had initially suspected Spanish cucumbers. This has proven to be a false positive, as lab tests showed that these cucumbers had on them a different strain of E.coli than the one that killed people.

However, the false positive has temporarily wiped out the Spanish cucumber business. According to the report:

many nations have put partial bans on Spanish vegetables, including tomatoes and cucumbers. This has been a huge hit to Spain, where agriculture makes up about 15% of gross domestic product. The crisis is affecting 70,000 jobs in Spain, Fernando Marcen, head of Spain's Agricultural Cooperatives, said Tuesday.

The statistical challenge in this type of investigations is extremely small sample size, and wide exposure to the potential cause. We are talking about hundreds of cases of infections but millions of people who eat cucumbers, or whatever other food is that might be causing the outbreak. In other words, there are manyfold more people who are exposed to the source but have not fallen ill than those who have fallen ill.

***

Food for thought: What's missing from the picture in the WSJ report is the lack of government mandated "recalls" of suspected foods. That's a very different attitude than what we have here in the U.S. As I noted in the book, it is frequently true that by the time the investigation turns up a strong suspect leading to a recall, those batches of contaminated foods would have all been eaten or thrown away.

 

 

False negatives in criminal justice

Having written about false negatives in steroid testing leads me naturally to ask: how serious is the false negative problem in criminal justice?

It's pretty clear that in law, we also try to minimize false positives, which means that there are criminals who evade justice. I wonder if, and how, one would assess the probability of a criminal not getting caught. Anyone have seen or know about such studies?

It would, of course, not be easy to measure such a thing since one may need a lot of confessions. But there may be smart ways around it. For instance, maybe a large business has tracked the total amount of stuff stolen from its stores over a period of time, and also the total value of goods stolen by those caught red-handed.

Following up on economic fairy tales

The following recent news items interest me because they relate back to previous posts on this blog.

***

The NYT reports, in an article titled "Many with New College Degree Find Job Market Humbling", that  "Employment rates for new college graduates have fallen sharply in the last two years, as have starting salaries for those who can find work. What’s more, only half of the jobs landed by these new graduates even require a college degree."

This confirms that our economic policy makers have lost touch with reality. It wasn't long ago that Ben Bernanke put his weight behind the argument that the key to reducing our current unemployment crisis is more college education. I discussed this issue here, as well as here. Incidentally, the NYT also found data to support my other point about pushing youngsters into debt: in this article, they noted that for the first time ever, student debt outpaced credit card debt.

Further down the article on the dire job market for college grads, it says "Among the members of the class of 2010, just 56 percent had held at least one job by this spring, when the survey was conducted.".  So, a rough estimate of the unemployment rate among new graduates is 44 percent! That's why in this post, I pointed out the fallacy of using the overall unemployment rate for college grads (5%) to talk about new college grads. This is almost a 10-fold error.

The NYT reporter didn't get the memo because the chart used to illustrate the story refers to the overall unemployment rate for college grads. This number includes people who graduated for college 20-30 years ago and are in mid-career.

***

The Dealbook section of the NYT reports that the Treasury is winding down bailouts, and "trying not to lose money". To believe that it is possible to "not lose money" is to believe in fairies. In this post, I consider why banks can never repay taxpayers in full. To recap:

As the irrepressible Dean Baker constantly reminds us, our nation suffered from a mammoth housing bubble, which wiped out $8 trillion of equity from "homeowners" when it burst. This $8 trillion is not monopoly money. One can't wish it away. If the government bailed out the banks (100% on the dollar), and then the government also "broke even", who suffered the loss?

Dealbook is written by business journalists who should know a thing or two about running businesses. When banks move in to rescue corporations on the verge of bankruptcy, they make loans at extremely high interest rates (say, 15%), and demand upfront payment of deal fees (I'm sure there are many other onerous terms.) On the day that the bailout was negotiated, the government already lost a huge amount of money by setting low interest rates, and not receiving upfront "deal fees".

While such interest and fees may appear unseemly, almost feeling like extortion at the weakest hour, they are compensation for the rescuer taking enormous risks. As I pointed out before, if one waits till the risks pass, and it turns out these companies (banks) survive, then in hindsight, it would seem like not receiving just compensation for taking risk is okay. But that is hindsight; the lender could lose everything if the bankrupt companies fail to recover. No one with any business sense would take such an arrangement without proper compensation.

 

Never a failed test fails the statistical test

Lance Armstrong is being accused of doping by former teammates, many of whom directly helped him to win 7 Tour de France championships. According to this Yahoo! report, Armstrong's response, via Twitter, was:

20+ year career. 500 drug controls worldwide, in and out of competition. Never a failed test. I rest my case.

***

I don't know if Armstrong doped or not. Given that his racing days are years in the rear-view mirror, there is little chance we will ever have direct evidence either way.

However, as I pointed out in Chapter 4 in Numbers Rule Your World, "never a failed test" is not a great basis on which to rest one's case!

We have quite a few examples of athletes who never failed any drug test during their competitive careers but later confessed to doping. Marion Jones and Bjarne Riis are two examples I used in the book. Why is this the case?

The sad truth of steroid testing is that most dopers do not test positive. A recent example (discussed here) illustrates that about 50% of dopers would pass the test -- and that was measured in a controlled laboratory experiment. The reason for such high false negative rates is that the anti-doping labs want to minimize the chance of a false positive error. The underlying statistics dictate a trade-off between false positives and false negatives; the harder one tries to eliminate false positives, the more false negative results will be produced!

I call for more false positives in drug testing in this post.

***

The media has gotten the statistics totally backwards.

On the one hand, they faithfully report the colorful stories of athletes who fail drug tests pleading their innocence. (I have written about the Spanish cyclist Alberto Contador here.) On the other hand, they unquestioningly report athletes who claim "hundreds of negative tests" prove their honesty. Putting these two together implies that the media believes that negative test results are highly reliable while positive test results are unreliable.

The reality is just the opposite. When an athlete tests positive, it's almost sure that he/she has doped. Sure, most of the clean athletes will test negative but what is often missed is that the majority of dopers will also test negative.

We don't need to do any computation to see that this is true. In most major sports competitions, the  proportion of tests declared positive is typically below 1%. If you believe that the proportion of dopers is higher than 1%, then it is 100% certain that some dopers got away. If you believe 10% are dopers, then at least 9 out of 10 dopers will test negative!

***

While researching the book, I learned that trying to catch dopers is extraordinarily hard. Here are some reasons why a doping athlete could get a negative test result:

• he's using a drug that is also produced naturally by the body, which means that the test needs to detect "unnatural" levels of the chemical, rather than the presence of a foreign substance
• he's using a new drug that has no test yet
• he's using "masking agents" that hide the performance enhancing drugs
• he's used steroids during training but not during competition (many sports don't conduct out-of-season testing, and even if they do, you can't possibly test all athletes all the time)
• he's received a "therapeutic use exemption" (I'm not sure why the sports bodies have never disclosed which athletes have been allowed to use which drugs based on TUE)
• he's following a drug schedule that attempts to evade testing

Those are not the only reasons. Notice that all of these tactics are not replicable in a lab so the accuracy rates reported by labs are almost for sure overly optimistic.

***

Armstrong's latest accuser is Tyler Hamilton, who features in my book. When I first started writing, he had failed one test, got banned, came back, and failed another test. Even after the second failed test, he had maintained his innocence. By the time I finished writing, he had come back yet again and failed a third test, upon which he retired.

One other statistical point of note: test results for a given individual are not necessarily "independent"! It's not too surprising that people like Hamilton kept failing tests while other dopers like Marion Jones kept passing tests. A failed test indicates that the doping program of that athlete isn't foolproof, and we should expect that athlete to have a higher chance of failing again in the future. A negative test, by contrast, may indicate that the doping program is robust against the testing regime. (In Jones's case, she was using a new designer steroid that took years for the test labs to notice.)

 

 

Blink and statistics: a close reading

This is the second part of my review of Blink by Malcolm Gladwell. The first part is here.

***

Blink has an awkward relationship with statistics. In a book on prediction, statistics should be a protagonist but it isn't. In certain parts where one expects to find statistical thinking, it's not there. Yet, one can't say statistics doesn't exist within Blink; it does, but it hides in the nooks and corners. Statistics in Blink is like a lone diner in a busy restaurant; he's present but not seen.

Statistical thinking shows up beautifully in a few spots.

In describing the work of psychologist John Gottman, who made some claims about predicting divorces, (the section starting on page 31) Gladwell masterfully describes the idea of "data reduction" which is central to statistics: much of the data, in this case, recorded conversations between couples, contain "noise"; such noise interferes with the project of finding predictors of divorce; the key to good analysis is the ability to fish out the "signal", leaving behind the noise. This is exactly what a statistical model does. (This is, however, not an endorsement of Gottman's analysis. As Andrew Gelman pointed out in a comment to my first post, statisticians have issues with Gottman's methodology, which I will get to in a moment.)

Then, in Chapter 5, Gladwell narrates numerous case studies of businesses using analytics, including music companies developing song ratings, Pepsi and Coca Cola running various taste tests, focus groups that gave margarine its yellow color, panels ranking jams for Consumer Reports, Herman Miller designers gathering consumer feedback on new chairs, and so on. While many of these efforts, all of which involve deliberate, data-driven analyses, are branded as failures by Gladwell, readers should keep an open mind. For me, this collection is a feast of real-world applications of statistics.

***

One gets to certain sections of the book expecting a statistician or two to pop up, but no such luck. So in what follows, I will add some filler materials relating to statistics.  Because Gladwell describes the research in words, without enumeration, if one seeks answers to my queries, one must review his sources, which I haven't done.

Let's start with Gelman's comment: "Gladwell in Blink is notorious for taking at face value the exaggerated post-hoc predictions of divorce guru John Gottman." The concern here is whether the wisdom lauded by Gladwell is foresight, or hindsight. This is a question that pops up regularly in my head as I read Blink.

I'll illustrate the hindsight problem using the malpractice lawsuit example (pp.40-43). The researchers found two groups of doctors, those who were sued at least twice in the past, and those who weren't. They then recorded conversations between these doctors and their patients. These conversations were analyzed along a lot of dimensions, and some factors emerged as statistically different between the sued-doctors and the not-sued-doctors. Chief among these differentiators was how much time the doctors spent with their patients.

This sounds like sensible stuff... until you realize that those recorded conversations took place after the doctors got sued. This methodology uses current measurements to explain past happening but what took place during the conversations could not possibly have caused the doctors to get sued in the past. The conclusion requires that we believe these doctors also, say, spent less time with their patients in the past but that is a guess only, not a direct measurement.

One can imagine these opposing scenarios: perhaps it's a matter of luck who gets sued, and then, regression to the mean implies that the not-sued group of doctors should on average get sued more often in the future; or perhaps doctors never learn, and those who make mistakes are doomed to repeat them. Or perhaps, doctors prefer not to get sued again, and adapt their behavior. The interpretation endorsed by Gladwell picked the second of these scenarios as true but why?

One test of their methodology would be to go out and find two groups of doctors, those who spend more time with their patients and those who spend less time with their patients. These two groups must be "matched" so that they are comparable. In particular, make sure that both groups contain the same proportion of doctors who were sued in the past. Now, wait a few years and compare the rate at which each group got served with malpractice lawsuits.

Apart from the hindsight problem, this analysis also suffers from myopia. The researcher decided from day one, without justification as far as I can tell, that the only useful predictors of malpractice lawsuits arise from doctor-patient conversations. Apparently, no other factors, like experience, age, height, dexterity, education and so on, matter.

I can't say that the research is definitely wrong but I'll not take it at face value. Unfortunately, quite a few examples in Blink follow a similar pattern.

***

Another incarnation of the hindsight problem is best shown by the Tom Hanks example (p.45). Hollywood super-producer Brian Glazer tells us that he knew Hanks would be a megastar in "that first instant" when Hanks went to him and read a script. But in statistics, you have to invert the situation and ask this question: how many other actors have Glazer loved at first sight, and what proportion of those have become stars? It is all too easy to pick a superstar and trace back to that first instant; the challenge is to find all those first instants, and track them to their natural conclusions (star or bust).

***

We can classify Gladwell's examples into two groups. The first group, such as predicting superstars or malpractice lawsuits, is simpler to comprehend because the predicted outcome is countable. The second group of cases involve quantities being predicted that are ethereal things that do not have real values.

For example, psychologist Samuel Gosling (p. 36) showed that "strangers with clipboards came out on top... at measuring conscientiousness, ... emotional stability and their openness to new experiences" of other people, when compared to these people's friends. Traits like emotional stability do not have objective values. So, in fact, Gosling is comparing the friends' perception of the subject to the subject's self-perception (or, the impression the subject desires to leave upon others.)

This isn't a criticism of this type of study, just something to bear in mind when reading about them.

***

At various moments, I scratch my head over the practical value of the predictions. For instance, the tennis coach Vic Braden claims to have a supernatural ability to predict when a player will double-fault on the next service. So what? 

There is a long section on face-reading (Chapter 6) in which Gladwell follows some researchers who have a system to describe in words people's facial expressions. They narrated Kato Kaelin's testimony during the OJ Simpson trial. "It's almost totally A.U. nine. It's disgust, with anger there as well, and the clue to that is that when your eyebrows go down, typically your eyes are not as open as they are here. The raised upper eyelid is a component of anger, not disust. It's very quick. You know, he looks like a snarling dog." (p.211). I must confess I don't get the point of this story because none of these "predictions" could be checked.

***

The rest are some sentences that don't make much sense to me:

p.40 "Analyses of malpractice lawsuits show that there are highly skiilled doctors who get sued a lot and doctors who make lots of mistakes and never get sued.": Presumably, doctors who never get sued are not involved in the malpractice lawsuits being analyzed. This relates to a difficult issue in evaluating statistical predictions: if an action is taken based on a prediction, then it is often tricky to measure the counterfactual (what would have happened if the action were not taken).

p.41 "Roughly half of the doctors had never been sued. The other half had been sued at least twice.": So, none of the doctors were sued exactly once? This kind of strange experimental design should be explained. One would also like to know if the two groups of doctors are comparable in any way: say, if young doctors are more likely to get sued, then the sued-group of doctors will likely have a youth bias relative to the other group.

p.56 "... a study in which they had two groups of students answer forty-two fairly demanding questions from the board game Trivial Pursuit.... the difference between 55.6 and 42.6 percent.. is enormous. That can be the difference between passing and failing." It makes sense to explain why the 13 percent difference between the two groups of students is a large difference but it has nothing to do with the passing threshold (of 50 percent presumably). If one group scores 49.5 percent and the other, 50.5 percent, we have a tiny difference but still "a difference between passing and failing"! Typically, statisticians will use standard errors to put this difference into context.

p.93 "all other things being absolutely equal, how does skin color or gender affect the price that a salesman in a car dealership [sic] offers?" Alarm bells ring loudly whenever we read the qualifier of "all things being equal", here underlined as "absolutely equal" which adds nothing, because in social science, you cannot possibly change only one factor and fix everything else. You can't find a white guy who is the same in all aspects as a black guy except for his skin color.

pp.92-3 I felt I misread something in Gladwell's recounting of Ian Ayres's experiment. Ayres had 38 students participating, including five black men and seven white women. They "were instructed to go to a total of 242 car dealerships [sic]".  At each dealer they bargained. And the study had conclusions such as "Even after forty minutes of bargaining, the black men could get the price, on average, down to only $1,551 above invoice." Since he had an average for black men only, then it would seem that each student had to visit 242 dealers because you can't send the white guys to some of the dealers and the black guys to others. But it would be a rather lengthy study if each student had to spend up to forty minutes at each of 242 dealers. I also noted that the average price for black men was an average of five samples.

 

 

 

Typepad's complacency towards spam comments

It appears that the IT world today considers spam detection to be a solved problem. One doesn't hear many complaints about spam. Spam used to only infiltrate our emails; they now appear in many formats, including spam websites that adulterate search engine results, and spam comments that pollute blogging communities.

Many types of methods have been deployed to fight spam. Fighting spam is a statistical problem: algorithms try to predict whether each incoming email or comment is spam or not.

Bayesian spam filters, which determines what features of a piece of text are predictive of spam, are popular - the buttons that allow users to "report spam" or "report not spam" are integral to such algorithms, enabling them to learn by experience. Blacklists that block IP addresses are another method. The blacklist is a type of heuristic, essentially fixed rules.

***

I have been bothered by Typepad's seeming ineptitude in spam filtering for a long time. Certain comments that are obviously spam do not get filtered automatically. These are comments that appear to be extremely easy to identify via some simple heuristics. Namely, if the name of the person writing the comment is the name of a pharmaceutical drug or a brand of shoes or fashion, classify the comment as spam.

Imposing such rules will get rid of the following types of spam comments that recently have been "false negatives" under Typepad's so-called spam filtering product:

Cheap Jordans said:

If a thing is worth doing it is worth doing well.

Tiffany & Co Outlet Jewellery said:

There is someone that is coming or passing away in your life around the clock, so you may lose sight of those seen, and forget those remembered. There is gain and loss in your life, so you may catch sight of those unseen, and remember those forgotten. Nevertheless, doesn‘t the unseen exist for sure? Will the remembered remain for ever?

bike clothing said:

reaction of officials in Boston who advised residents to stop drinking tap water for fear of possible contamination which has not been verified.

louboutin shoes said:

My readers will note that I disapprove of how most statistics textbooks discuss the correlation/causation issue: you are told what not to do, but left with no idea of what to do.

The blog recently got flooded with 55 spams on a single day, coming from "louboutin", "louboutin shoes", and "christian louboutin sale". I just don't understand how any spam filtering algorithm worth its name can miss such obvious cases. It is extremely unusual to have someone whose first name is louboutin and last name is shoes. If someone has such a name, I'd argue (and sorry to be mean) that your comment deserves being expunged.

***

Every time I provide feedback to Typepad, their IT staff recommends that I manually block those IP addresses (the blacklist method). It appears that they don't see this as an obvious failure. They certainly don't realize that the failure to identify the spam already occurred. So I'm mystified. If you know anything about spam filters, please address these questions:

• Is it operationally difficult to add some heuristics to the algorithm?
• Is there something wrong with rules that assume no one is named "viagra" or "cheap viagra" or "cheap jordans"?
• Are there any new developments in spam filtering worth our attention?

 

A statistician reads Blink

Blink by Malcolm Gladwell is a difficult book to review -- only because its theme defies being nailed down. Superficially, the book glorifies the snap judgment (blink), the elevation of quick, instinctive, possibly unconscious, or involuntary decision-making over slower, deliberate, careful consideration. Viewed in this light, Blink is the anti-Moneyball: there is even a section about a tennis coach Vic Braden who, we are told, has the surprising instinctive ability to predict when a player will serve a double fault, a task even computers couldn't master.

As the book continues, haze starts to surround this message. As Gladwell introduces the police officers who mistakenly thought a suspect was holding a gun (it was a wallet), and the market researchers who did not trust a musician called Kenna based on focus-group response, and others, the reader senses that Gladwell thinks snap judgments are sometimes right, and sometimes wrong. On that, the reader would prove correct: turning back to the Introduction, she finds, perhaps unexpectedly, the message that "decisions made very quickly can be every bit as good as decisions made cautiously and deliberately." (p.14) So, Gladwell does not argue that all blink decisions are better, only that some are better; strictly speaking, he does not even claim that blink is better, only that it is not worse!

At times, Gladwell seems to share with Freakonomics a condescending attitude toward "experts". In the book's opening pages, he pitted intuition against a geologist whose scientific analysis led to the erroneous authentication of a fake kouros (a type of Greek statue). Some art historians were said to have an inkling that the statue was a fake from the first moment they set sight on it, so why, Gladwell asks on p.14, "didn't the experts at the Getty (Museum) also have a feeling of intuitive repulsion during the fourtheen months they were studying the piece?" The rhetorical no then burdens such experts with a costly, wrong decision.

At some point, the reader realizes that the heroes of the book -- be they the art historians, the tennis coach, the Herman Miller chair designer, or others -- are also experts in their own right. Just like in Freakonomics, those busting "conventional wisdom" are often themselves "experts". Gladwell goes one step further, and freely admits this. One of his side arguments is that blink can be trained. The key is to identify the small number of variables that determine the outcome, reducing complex data to simple terms.

This, too, eventually gets shrouded in smoke. Scientists at Coca Cola who rely on the snap-judgment taste test created a New Coke that became an expensive debacle. It becomes apparent that in the Gladwell universe, not all experts are bad. Some should be trusted and some not. On p.15, Gladwell asks "when should we trust our instincts, and when should we be wary of them? Answering that question is the second task of Blink." It is this promise that Gladwell ultimately fell short of. By the end of the book, the reader is likely to have a few vague ideas and not much more.

For example, in Chapter 3, Gladwell introduces a car salesman Bob Golomb who, he explains, has learnt not to trust his first impression, i.e. the previously praised snap judgment. The message seems to be that one ought to recognize the types of problems that blink cannot solve. And yet, how did Golomb figure out that he should treat all customers as potential buyers and not prejudge them based on physical appearance? From stories of previous mistakes. So, recognizing when not to use blink requires that one first makes mistakes using blink. I, for one, find this advice difficult to follow.

Perhaps Gladwell does not intend to provide definitive answers to these questions. He has hit on a significant area of research in decision making, and has raised (self-)awareness of some important issues. Many of the anecdotes contribute to a lively discussion of these topics. I read the book enthusiastically and would recommend it to others. In particular, Blink holds considerable interest to those who practice statistical decision-making, a topic I will turn to in a future post as I've now exhausted your patience for one post.

 

 

Some work till they die, some retire after they die: the discontent of being averaged

Flowing Data has a post about early retirement which provides an opportunity to highlight, yet again, the discontent of being averaged (Chapter 1 of Numbers Rule Your World).

The post comes with this chart. For each country and each gender, he compares the life expectancy and the average retirment age, the difference between which yielding the "number of years in retirement".

Unfortunately, one should be careful when taking the difference between two averages. This is because the population that underlies one average may not be the same one that underlies the other average.

The life expectancy is clearly an average over the entire population. Average retirement age, however, is almost certainly an average over people who live to at least retirement age. Thus, taking their difference doesn't make sense. What we need is the life expectancy of those people who at least live to see retirement.

The (somewhat) funny result of taking the difference of those averages can be seen in the case of Mexican men (the bottom blue bar). By this metric, the average years in retirement for Mexican males is essentially zero: they work till they die.

Another way to see that this metric can't work: if the average years in retirement is zero, then some will die after they retire while others will retire after they die! Retiring after death is quite literally captured when you substract the average retirement age from the average life expectancy!

***

By locating the right data, one can improve this analysis. What is needed is a life table. (one source is here). From this table, we can find for every age, the expected remaining life.

Take all the age groups above the retirement age, compute weights for each age group as a proportion of all remaining age groups, use these weights to compute the weighted average remaining life.

For Mexican males, this yields about 7 years, assuming the retirement age is indeed 75 as shown on the chart above.

 

P.S. See also the comments from the Flowing Data readers.

 

 

Grouponomics, and the power of counterfactual thinking

Felix Salmon wrote thousands of words to convince himself that Groupon has a viable business model. I have a lot of respect for Felix and love his blog, but he missed a key issue here. Maybe he drank the Koolaid, or perhaps I am an even bigger skeptic than Felix is. I don't have a rebuttal as much as an alternative narrative.

***

Let's start with his neighborhood restaurant example:

At Giorgio's, for instance, diners paid $15 for their Groupon -- which gave them $30 of food. But dinner for two at Giorgio's, with some kind of alcohol, can easily run to $100 or more. So even after knocking $22.50 off the bill (remember that Giorgio's kept $7.50 of the proceeds of Groupon), the restaurant would often still make money.

This is a bit complicated. We can trace how the cash flows. For Groupon, diners pay them $15, and they keep half of that, $7.50. For the diners, they paid Groupon $15 (now worth $30 spending), and so they pay Giorgio's $70; in other words, they paid $85 out of pocket for a meal worth $100 without Groupon. Giorgio's take in $70 from the diners plus $7.50 coming from Groupon for a meal worth $100.

Where I think Felix missed the mark is when he reasoned:

If you're already a regular somewhere, of course, then buying its Groupon is a no-brainer. And the restaurateur won't begrudge you the savings, either: all restaurant owners want to treat their regulars as well as they can.

I violently agree with the first sentence -- that Groupon-type deals are a gold mine for regular customers. The second sentence, however, doesn't make sense. Go back to the example. For a new customer, Giorgio's makes $77.50 that it would otherwise not make without Groupon. For a regular customer, Giorgio gives up $22.50 that it would otherwise have taken in without Groupon. So, for every 3.4 regulars who use the coupon, Groupon has to generate at least 1 new customer for Giorgio's to break even.

If Giorgio's does not achieve that ratio of new to regular Groupon-wielding customers, then Giorgio's could end up earning less total revenues despite having served more diners/meals.

So, I don't think the Groupon model is the kind of slam dunk Felix seems to think it is. Only if certain conditions are met will the merchants gain anything from Groupon:

• the value of the coupon has to be a fraction of the total spending at the merchant; in this example, the diners spent more than 3 times the face value of the coupon. What if the diners spend exactly $30? Then Giorgio’s loses $22.50 on each regular customer and earns $7.50 on each new customer, meaning that every 3 new customers pay for each regular’s discount. Not very attractive numbers at all.
• the lifetime value of the new customer must be high. Felix also mentions this. The business case is better if the new customers keep coming back, and pay undiscounted rates.
• the coupons are available in limited quantities and at limited times. The more times regulars use these coupons, the bigger the revenue hole the merchant finds itself facing.
• the merchant has no other means of attracting new customers. Our computation assumed that the new customer would never contribute any revenues unless Groupon is in the picture.

This analysis deals with the business model of a merchant using Groupon; even when the merchant loses, Groupon will still make money.

***

How is this related to statistical thinking, you might ask? Why am I wasting ink talking about this?

What is missing in the typical narrative about Groupon – the one promoted by the company – is the “counterfactual”. The counterfactual is a powerful concept embedded in statistical theory. In order to evaluate the data in front of us, we must imagine an alternative world (the counterfactual) in which we allow the data to present themselves differently. To understand how medicine X might affect you, you can't just measure what happened after you took medicine X; you must also consider what might have happened if you took medicine Y and/or nothing at all.

So, instead of just thinking about the new customers brought in by Groupon, we must also consider the world without Groupon.  In the world without Groupon, the regulars pay $100 for their meals, and the new customers pay $0 or some other amount (if Giorgio’s has other ways to attract them). This allows us to realize that the insertion of Groupon into that world would lower the intake from regulars while simultaneously raising the intake from new customers. For the merchant, whether Groupon is a net benefit depends on the balance between those two numbers.

This is related to the concept of opportunity cost in economics. When evaluating the value of an investment, we can’t just tally up the returns of said investment; we have to compare those returns to the returns of doing something else such as keeping the money in the bank.