Big Data, Plainly Spoken (aka Numbers Rule Your World)

Propublica is on a roll

Really enjoying Propublica pieces lately. There are several articles about topics of great interest to me, and those who read my books will be familiar with these themes.

My favorite is an article that speaks a truth about data projects -- much as we sweat about data collection, data integrity and statistical models, the true challenge is in persuading the rest of the world to adopt our endproducts. The title of this piece says it all "The FBI built a database that can catch rapists--and almost nobody uses it. " (link).

The data project in question is an early effort to link data from multiple sources to leverage correlations to solve the problem of identifying serial offenders. However, less than 10% of local police departments contribute data to the system, rendering it toothless. In my experience, it is common to find data projects stuck in first gear, and failing to make any real-world impact.

Kudos to the authors for asking the dirty question of the return on investment of such a system. It is believed that in 12 years, the system may have helped solve 33 crimes. It costs $800,000 per year to maintain (most likely, contractor expenses). You do the math!

For managers, the key is to diagnose properly the reasons for inaction. Lack of adoption is frequently blamed on technology but the reality is much more complicated.

A must-read article.

***

David Epstein reports on raids on steriods labs. (link) Law enforcement is the most effective way to catch cheaters in sports. In Numbers Rule Your World, I explained why anti-doping tests are ineffective, in the sense that false negatives are rampant, letting lots of dopers off the hook. This conclusion comes from a simple statistical calculation. In the chapter on a lottery cheat, I described how statistics can be used to prove that "someone has beyond reasonable doubt cheated" but physical evidence is required to nail the perpetrator.

Epstein then expanded the conversation: "World-class athletes are merely the fine layer of frost atop the iceberg’s tip when it comes to the steroid economy." The headline of the piece is "Everyone's Juicing".

I find it interesting that Epstein said "In years of reporting on performance-enhancing drugs, I’ve frequently been asked why athletes in smaller sports or facing lower stakes would dope, given that there’s little money in it for them." This feels odd because when I was researching my book five or six years ago, I heard the opposite claim, that elite athletes couldn't possibly be doping because they don't need steroids. (Think Barry Bonds back in the days.) This tells me that (a) public opinion has shifted due to the Armstrong revelations and (b) the human mind will rationalize any story even if the story flips.

Epstein has another article in August about false negatives, which should be familiar territory for my readers (link).

***

Joaquin Sapien reports on the case of one Ruddy Quezada, who was released after spending 24 years in prison for murder. This case reminds me of the Innocence Project, whose amazing work I featured in Numbers Rule Your World. In the current scenario, though, we don't know if Quezada was innocent, only that the prosecution lied about how they coerced the witness to testify. The witness testimony was the only piece of evidence in that case, which means that the prosection is left with no avenue to re-try the case.

The case I used in my book concerns false confessions so both cases deal with coerced evidence.

 

 

 

Innocence makes one more likely to make a false confession

That was one of the revelations while I was researching for Chapter 4 of the book, dealing with unavoidable errors in statistical decision making.

The New York Times took up this very important issue in a recent article. Typically, we think that if someone confesses to a crime, the confession is the strongest evidence possible, and guilt is  assured. However, criminologists who have studied this issue discover that innocent people do confess to crimes that they did not commit, and not just mentally challenged people. They believe that other evidence will surface to prove their innocence but juries tend to take confession evidence at face value.

The other shocking thing I learned was that it is legal for the police to tell you you failed a lie detector test even if you have passed it. The police also consider lie detector tests as a tactic to extract a confession (because in most courts, lie detector test results themselves are inadmissible.)

The trouble, as the article points out, is that very rarely are such "false positive" errors uncovered and only after extraordinary effort, and therefore the cost to the decision-maker (law enforcers) of this type of error is quite small. That's why the work of non-profit groups like the Innocence Project is absolutely invaluable.

(Thanks to reader Stephen N. for bringing this article to my attention.)