Big Data, Plainly Spoken (aka Numbers Rule Your World)

No, selfies are not more deadly than sharks

You may have seen the clickbait of the week: "Selfies have killed more people than sharks this year!"

There are three problems with this statement:

1. Almost everyone takes selfies but many, many fewer of us are exposed to shark attacks. Thus even if the absolute number of deaths from selfies is higher than that of deaths by sharks, the risk of dying from taking selfies is still much lower.

2. The headline did not lie. It is worse than lying. It misleads readers into thinking that selfies are more dangerous.

3. Your risk of dying from selfies is ridiculously low. (only 12 deaths in 9 months, and countless exposures-in both sense of the word)

Now, if someone died from taking a selfie in front of a shark, that would be something.

 

PS. [9/24/2015] A dialogue on Twitter reveals problem #4: the deaths while taking selfies were attributed to taking selfies but could have been attributed to other more direct causes such as climbing on top of a moving train. Meanwhile, sharks do directly attack and kill people.

 

Mass media revisionism

To borrow from probability theory, one can apply the "memoryless" property to our news media: it appears that current news reports do not need to retain memory of past news reports.

The other day, I commented on new reports about doping that suggest that there is widespread belief that steroids are not abused by small-time athletes because they have no chance of winning--when for the longest time, the belief was that steroids were not abused by big-time athletes because they had too much talent or worked too hard to require extra help.

So now there is an article on ZDNet about Groupon that states with a straight face:

There was long speculation that the daily deals craze would eventually die out, with many analysts deeming the model unsustainable because of its contentious relationship with merchants. Seemingly realizing its shaky future, Groupon attempted to refocus its business more around e-commerce, but the company's best efforts appear to have fallen short.

As I documented in Numbersense, the mass media was in euphoric consensus about Groupon's world-conquering prospects... to the tune that the company was able to raise about $1 billion in the public market (most of which immediately went to pay off initial investors).

As a reminder, this was a post I wrote before Groupon's IPO pointing out the flaws in their business model.

The majority of the business and technology press was not speculating that the "daily deals craze would eventually die out". In fact, we were told Groupon would become the next eBay or Amazon.

Beware of mass media memoryless revisionism.

 

 

 

Propublica is on a roll

Really enjoying Propublica pieces lately. There are several articles about topics of great interest to me, and those who read my books will be familiar with these themes.

My favorite is an article that speaks a truth about data projects -- much as we sweat about data collection, data integrity and statistical models, the true challenge is in persuading the rest of the world to adopt our endproducts. The title of this piece says it all "The FBI built a database that can catch rapists--and almost nobody uses it. " (link).

The data project in question is an early effort to link data from multiple sources to leverage correlations to solve the problem of identifying serial offenders. However, less than 10% of local police departments contribute data to the system, rendering it toothless. In my experience, it is common to find data projects stuck in first gear, and failing to make any real-world impact.

Kudos to the authors for asking the dirty question of the return on investment of such a system. It is believed that in 12 years, the system may have helped solve 33 crimes. It costs $800,000 per year to maintain (most likely, contractor expenses). You do the math!

For managers, the key is to diagnose properly the reasons for inaction. Lack of adoption is frequently blamed on technology but the reality is much more complicated.

A must-read article.

***

David Epstein reports on raids on steriods labs. (link) Law enforcement is the most effective way to catch cheaters in sports. In Numbers Rule Your World, I explained why anti-doping tests are ineffective, in the sense that false negatives are rampant, letting lots of dopers off the hook. This conclusion comes from a simple statistical calculation. In the chapter on a lottery cheat, I described how statistics can be used to prove that "someone has beyond reasonable doubt cheated" but physical evidence is required to nail the perpetrator.

Epstein then expanded the conversation: "World-class athletes are merely the fine layer of frost atop the iceberg’s tip when it comes to the steroid economy." The headline of the piece is "Everyone's Juicing".

I find it interesting that Epstein said "In years of reporting on performance-enhancing drugs, I’ve frequently been asked why athletes in smaller sports or facing lower stakes would dope, given that there’s little money in it for them." This feels odd because when I was researching my book five or six years ago, I heard the opposite claim, that elite athletes couldn't possibly be doping because they don't need steroids. (Think Barry Bonds back in the days.) This tells me that (a) public opinion has shifted due to the Armstrong revelations and (b) the human mind will rationalize any story even if the story flips.

Epstein has another article in August about false negatives, which should be familiar territory for my readers (link).

***

Joaquin Sapien reports on the case of one Ruddy Quezada, who was released after spending 24 years in prison for murder. This case reminds me of the Innocence Project, whose amazing work I featured in Numbers Rule Your World. In the current scenario, though, we don't know if Quezada was innocent, only that the prosecution lied about how they coerced the witness to testify. The witness testimony was the only piece of evidence in that case, which means that the prosection is left with no avenue to re-try the case.

The case I used in my book concerns false confessions so both cases deal with coerced evidence.

 

 

 

Statbusters: What the experiments on rigging elections via Google tell us

For this week's Statbusters (link), we opine on that astounding report from a few weeks ago about how Google could manipulate the next elections by biasing search results. We walk you through our vetting process, starting with face validity ("the magnitude of the reported effect is too large to be believed!").

The crux of the article is about the experimental design. You start with a group of people who have no prior opinion of the candidates (e.g. showing Americans Australian candidates), then give them only one source of data about those candidates, then check to see if the data affected their opinions. Besides, the manipulation was rigged for maximum impact - the base setting is that the entire first page of results favors one candidate.

As we said, the result of these experiments is still interesting but not as interesting as the breathless headlines proclaim.

One issue that didn't make it to the article but I should mention here is the ethics of one of the experiments, which sought to manipulate a real-life election in India. Recall the outcry surrounding the Facebook experiments that supposedly manipulated the emotions of people via feed messages. I didn't think much of that controversy (link). However, I'm surprised there isn't a bigger outcry surrounding this Indian experiment, which manipulated political voting behavior. The researchers said the IRB accepted their argument that their sample size is tiny relative to the size of the Indian electorate. I am not comfortable with the concept that ethics is relative to sample size.

Read our Statbusters column here.

Making stuff up to get published in NYTimes

Reader/friend Tom B. knows about my interest in grade "deflation" policies, and proceeds to ruin my breakfast by sending me a link to this ludicrous "letter to the editor" by a high-school counsellor (link).

It starts with a made-up assertion:

As the new academic term starts, I’m rooting for this to be the year when students start getting the grades they rightfully earn without high schools and colleges manipulating numbers to protect their institutions from accusations of grade inflation.

Which quickly leads to the conclusion:

Eliminating grade deflation would vastly improve education.

I suppose the Times does not think there is a need to fact-check a "letter to the editor" which doubles as a soft marketing stunt. But anyone who works in education, including many of the readers who responded to this stunt, immediately notices a problem. There is no outcry or movement to deflate student grades. Look at the chart in my post here: grades have been climbing up not down.

This chart says it loudest (the red line are the proportion of As given out):

[The chart is from this National Journal article, via a Washington Monthly article. It is not clear who is the original source, most likely from Stu­art Rojstaczer, a Duke professor.]

At the high school level, the upward trend is also unmistakable:

 ***

Later on, the author cites the Princeton case, which I discussed here. I winced when she said "Princeton’s method ... was copied throughout the nation." In fact, when Princeton's new president explained to alumni (of which I am one) why he did away with the grade deflation policy, he explicitly said that its peer institutions did not follow suit, causing students and parents to feel that graduates were at a disadvantage in the job market!

Here is a direct quote from Princeton President Eisgruber (with my bolding):

Almost 10 years after its enactment, the policy remained a lightning rod of controversy and a considerable source of stress for many students, parents, alumni, and faculty members. And regrettably, none of our immediate peer institutions followed our example in taking tough measures to address grade inflation. As a result, Princeton, which ought to be renowned for the unsurpassed quality of its teaching, was attracting more attention for the severity of its curve.

In my prior post, I described this situation as a prisoner's dilemma, in which the stable states are that all schools give out more meaningful grades (not happening) or that all schools give out inflated grades.

Richard Grayson, the first printed response to the Times article, hits the nail on its head - the replacement of full-time faculty with adjunct faculty is a major cause of grade inflation.

***

It appears that in order to get published these days, you have to make up your own facts and generate controversy around them.

 

No one in Gene Woolsey's class made spelling errors

I wished I had a chance to take Gene Woolsey's class. Woolsey was a major personality in Operations Research, and from reading a set of remembrances in ORMS Today (June 2015), I learned that he was also an amazing teacher. And no one in his class made spelling errors! How cool is that. Well, I don't know if no one ever made a spelling error but I bet that the frequency of typos was much lower than usual. The INFORMS obituary is here.

The ORMS Today article contains an amusing passage about his Statistical Consulting class, starting with:

The class had a dress code: a suit or equivalent attire for presenting a project to the CEO of an organization or a three-star general (or higher).

and my favorite (my bolding):

To emphasize the importance of presenting your work professionally, each spelling or grammatical error dropped your assignment mark by one letter grade.

In case the reader does not feel the gravity of the situation, the author, a former student, elaborates:

It really drove the lesson home when you saw an "A" on your paper scratched out and replaced with a "C" because of two spelling errors.

Next time I mark a paper that contains multiple typos and errors, I might need to pull a Woolsey!

 

Statbusters: Were you drunk on Labor Day?

On Labor Day, our new Statbusters column appeared. This one concerns a popular news story from some weeks ago, saying science has proven that there are four types of drunks. The four refers to four "clusters" formed by running a cluster analysis algorithm. But four is decided by the analyst. Some algorithms won't run unless the analyst specifies the number of clusters; other algorithms generate the best structure for every number of clusters. This method is great for exploring and understanding the data but cannot confirm that there are precisely four types of drunks!

The media often removes the uncertainty of science in the name of "popularizing." The entire article is here.