Big Data, Plainly Spoken (aka Numbers Rule Your World)

The cost of bad advice

In the last week, I kept being prompted by the recommendation engines to read the article in Slate titled "Don’t Just Look at the Odds: The Math Says You Should Buy a Mega Millions Ticket." It's obviously false but eventually I succumbed and clicked on the link.

I wanted to stop reading after I encountered this sentence, about half way through the article: "One assumption in this model is that you’ll take the annual payment option and receive the full $1.6 billion."

The pertinent assumption that they made isn't whether the winner should take the lump sum or annual payment options. The blockbuster is to assume that there would only be a single winner.

Earlier in the article, they stated a true but irrelevant point: that the odds of a winning ticket and the price of each ticket are still th same no matter how many tickets are sold.

I persisted in reading the entire article for the sake of writing this post. I wanted to know whether they will acknowledge the biggest factor here, which is that the more tickets are purchased, the more likely there will be more than one winners. Even having two winners can be devastating. The $1.6 billion suddenly turned into $0.8 billion. With three winners, it's $0.53 billion. If the ticket is purchased as part of an office pool with, say 10, participants, then $1.6 billion becomes $160 million. Not a bad payday but much, much worse than projected.

 

PS. They updated the post because too many people are pointing out the obvious about multiple winners. They continue to deny its importance.

Their math is wrong. Here is a good webpage that explains the math from a previous situation 10 years ago. The number of winners follows what is called a Poisson distribution. In the example given, there is a 60% chance of a single winner, but 30% chance of double winners and 10% chance of more than two winners. How can this be called "incredibly low odds"?

Another issue is that there are certain sets of numbers that attract much more tickets than usual. If one of those sets is the winner, you're going to have lots of winners.

They continue to completely miss the point.

 

 

 

 

Some statistics about nutrition statistics

I only read nutrition studies in the service of this blog but otherwise, I don't trust them or care. Nevertheless, the health beat of most media outlets is obsessed with printing the latest research on coffee or eggs or fats or alcohol or what have you.

Now, the estimable John Ioannidis has published an editorial in BMJ titled "Implausible Results in Human Nutrition Research". John previously told us about the crisis of false positives in medical research.

Oops, here are some statistics on nuitrition "science":

• In 52 attempts at using randomized experiments to validate findings from observational studies, the number of times the findings were replicated: 0
• In the NHANES questionnaire (the basis of all those findings), two-thirds of the participants provided answers that imply an energy intake that is "incompatible with life". I haven't read this paper; seems like worthwhile reading.
• There are at least 34,000 papers on PubMed with keywords "coffee OR caffeine" which means this one nutrient has been associated with almost any interesting outcome.
• Almost every single nutrient imaginable has peer reviewed publications associating it with almost any outcome. A statistician should never give the advice "If at first you don't succeed,..."
• Many findings are entirely implausible (and still get published in top journals)... for example, the idea that a couple of servings a day of a single nutrient will halve the burden of cancer is clearly "too good to be true," even more so for anyone who is familiar with this literature
• "Big datasets just confer spurious precision status to noise"
• Randomized experiments offer hope but are woefully undersized (like requiring 10 times the current sample).

Just to nail home the point, John concludes: "Definitive solutions will not come from another million observational papers or a few small randomized trials."

 

The tricky business of thinking about uncertainty, and why banks can never repay us enough

Andrew Gelman alerted us to a classical probability error that showed up on the Freakonomics blog recently. (by way of Xian).

Probabilistic thinking is not very intuitive. One wishes the Freakonomics team would attack conventional wisdom, instead of embracing it. Here, they trumpeted the "very long odds in the Israeli Lottery": the fact that the same six numbers won the same lottery three weeks apart. This feels like an extremely rare event but it is a case where we tend to underestimate severely its rarity.

My readers may want a refresher of what the mistake is.

***

Let's start with the much simpler "Birthday Problem". Say you're in the jury selection room with a hundred people. What's the chance that you can find two people born on the same day? The chance is much higher than you would imagine! In fact, the math shows that it is more or less 100%! Even if you have 60 people in the room, it is almost certain. With only 30 people in the room, there's about a 65% chance of finding at least two with the same birthdays.

The key to understanding this puzzle is to realize that we did not fix the day of birth; we are not looking for two people born on January 1st. The two people born on the same day could have been born on any one of 365 days (excluding Feb 29 for convenience). The chance that we can find two people born on exactly Jan 1 is much less likely than the chance of finding two people born on the same day.

Similarly, there are 4,950 distinct pairs of people in the room, any one of which can be the pair of people with the same birthday. So, there are many ways in which we can find two with the same birthday and that's why it is much more likely than we think it is.

The point being: If you take enough drives in the safari, you will find a leopard.

***

Turning to the Israeli lottery. The "rare" event in question can also happen in many ways: the numbers could reappear one, two, three, ..., ten weeks apart (assuming it's a weekly lottery); and the reappearance could happen in any one of I'd think thousands of lotteries all over the world. In fact, every single time a lottery is drawn anywhere in the world, there is a chance that the outcome will prompt some journalist to write up the same story: in X lottery, the same Y numbers won Z weeks apart, what long odds!

The point being: The more drives you take in the safari, the more likely you will eventually find a leopard.

***

Perhaps the difficult part of taking this argument is: why should we be thinking about these imaginary scenarios if we already know that it happened in the Israeli lottery and it happened three weeks apart? Doesn't the reality make the imaginary scenarios irrelevant?

Here is how I'd think about this: if you already know that the Israeli lottery drew the same numbers six weeks apart, then the question of what are the odds is silly because the odds are 100% since we know for sure that it happened! 

The question of "what are the odds?" makes sense only at the moment when we don't know if something would happen or not. That means one of many possible scenarios could happen.

This need to place ourselves in a state of uncertainty is what I think trips up a lot of people. We do not like uncertainty.

***

Because of the same reason, some people (most mainstream journalists, it seems) buy the government line that we "made money" or "did not lose money" from the TARP rescue of failed too-big-to-fail banks.

This analysis is done from a state of certainty, from hindsight, taking into account the banks thus far survived the danger. But evaluating TARP from this view is like celebrating a lottery win after you found out you are the winner. The fact that you won the lottery does not change the fact that economically, it was silly to play the lottery in the first place.

When a decision is made in a state of uncertainty, it has to be evaluated in that state of uncertainty. For it is the uncertainty that imposes the greatest cost. In the case of TARP, the key question is to determine whether the government negotiated the right terms on behalf of the people at the time of the decision, when the banks faced enormous existential uncertainty. There is little question that the government did not get a good deal. Just take a look at the kinds of deals that the banks themselves get when they save failing companies from bankruptcy! (This often involves the banks owning a vast majority of the shares of said companies in addition to multifarious onerous conditions.) Or, how banks act when we don't pay our credit card or loan bills: is it okay for us to stop paying for a year, repay the amount in full, and declare no harm done?

Of course, there were a zillion other policies not called TARP that directly benefited banks at a cost to the people, and thus a variety of arguments why the bank rescue was not costless. My favorite is the suspension of mark-to-market accounting so banks can list their assets at fictitious prices (some of these assets are cash flows from "liar loans" and so on). I will outsource the big picture to economist Dean Baker who does a great job "beating the press" every day (here, and here).