Reading SuperFreakonomics 3

The third chapter of SuperFreakonomics is simpler in structure than the other two chapters, containing just two parts, one dealing with the Kitty Genovese murder and the other with the research of Chicago economist John List, a colleague of Levitt.

The chapter touches upon a number of keystone experimental results from psychology, exposed to anyone who have taken PSY 101. The Kitty Genovese case is a real-life example of the "bystander effect", the tendency of human beings to offer no help to strangers because they are expecting others to offer help. 

The Ultimatum and Dictator games are used to study human rationality and the attitude toward inequity.

The Stanford prison experiment and the Milgram experiment at Yale were also briefly touched upon, these being studies of "obedience to authority", used to show that people have a capacity to do really bad things when given roles as authority figures (Stanford), or subject to authority figures (Yale).

***

Here are, again, my thoughts on statistical topics as I read the chapter:

pp.100-1: Raises an important point that randomized controlled experiments in criminology that are also ethical and politically acceptable are very difficult to design. The thought experiment of a randomized experiment they concocted to "know whether putting more people in prison really lowers the crime rate" doesn't make much sense to me:

Pretend you could randomly select a group of states and command each of them to release 10,000 prisoners. At the same time, you could randomly select a different group of states and have them lock up 10,000 people, misdemeanor offenders perhaps, who otherwise wouldn't have gone to prison. Now sit back, wait a few years, and measure the crime rate in those two sets of states. Voila!

In case you are interested in running randomized experiments, here are a few pointers:

• Even though there are two groups of states, there should be only one randomization that puts states into two groups. They did use the phrase "at the same time" but I find the sentences a bit confusing.
• Think clearly about the unit of analysis (also the sampling unit): for this example, it could be "state", "locale", "prison", "prisoner".
• Think about the nature of the difference in treatments applied to the groups: what is really being measured when one group releases 10,000 guilty while the other group locks up 10,000 "innocents"?

p.115: L&D wrote: "In Virginia, List cruised the trading floor and randomly recruited customers and dealers, asking them to step into a back room for an economics experiment."

This hits on one of my pet peeves. Whenever I see an assertion of "random selection" in an observational study, I want to ask what mechanism was used to ensure randomness. Think, for example, when the NYC subway police tell us they "randomly" inspect bags, how is it precisely that they enforce such "randomness"? Are they picking on every Nth passenger? Do they carry a pseudo-random generator?

Not knowing the precise selection rules is not an excuse for assuming random selection. In this example, I'd like to know how List "randomized" the recruiting process.

pp. 118, 122: would love to see sample sizes cited alongside the results from the Ultimatum/ Dictator experiments, all reporting fairly large differences between groups. I'm sure they were large enough, but statisticians always worry about the so-called "law of small numbers" (attributed to Kahneman and Tversky). When sample size is too small, even very large differences may occur by chance.

pp.120-3: Discusses the shortcomings of the experimental economics field, which has become quite influential (see Dan Ariely's book). Many of these issues are exactly the things statisticians worry about... selection bias, nonresponse bias, generalizability, observer effect, etc. Well worth reading and pondering.

What I don't get about this section is how they can be so negative on these "lab studies" while simultaneously so admiring of John List's research. As far as I can tell, List also ran "lab studies". The main difference is that in some cases, his subjects were not students but traders taken from baseball-card trading floors. These subjects knew they were part of an experiment and were instructed to do certain things.

But I agree with their general points about lab research, and also think List's research is useful. I hope List and other researchers will reach out to the market research and political polling communities because many of the problems they face are not new problems; they have been studied for a long time, and these communities can help each other.

pp.122-3: They continue the takedown of lab research, citing a researcher saying "lab experiments have the power to turn a person into a 'stupid automaton' who may exhibit a 'cheerful willingness to assist the investigator in every possible way by reporting to him those very things which he is most eager to find'." This is where they introduce the Stanford and Yale experiments as proof of "forced cooperation".

When I was taught about these experiments (admittedly many years ago), they were evidence of the human capacity to do bad things but L&D's point here is contrarian: they say these bad things could only happen in labs but not in real life. This is a new interpretation to me.

p.124: am glad to hear about "warm-glow altruism" research, would love to learn more.

pp.125-: these pages are a take-down of the New York Times reporting on the Genovese murder. Interesting story, looking for a rejoinder.

Comments

[A nonnimous]

I am not going to bother reading Freak II, given the sloppiness you have already documented. But I wonder whether Levitt and Dubner portray the "38 silent witnesses" to Genovese's murder as fact. This would be more sloppiness, since the standard version of the story, due to the NY Times, is false. See http://riverdaughter.wordpress.com/2009/08/02/a-fascinating-intersection-of-true-crime-psychology-and-media-misinformation/

for example.

[Kaiser]

A nonnimous: The point of these posts is to encourage reading of SuperFreakonomics but with a critical mindset. They raise a host of interesting issues, but will not convince every reader.

Regarding Kitty Genovese, they approached it in the usual Freakonomics style. At the start, they presented the original New York Times report of the story, setting up the "bystander effect" theme. At the end, they came back to this story, and presented the contrarian viewpoint. This latter viewpoint was presented as another person's opinion; I don't believe they presented it as the definitive version. Given the nature of these eyewitness accounts, it would be difficult to designate a definitive version, I'd think (but I am not so familiar with this case.)

[A Professor]

I am assigning "Numbers Rule Your World" as required reading for my intro statistics course.

This is how I plan to use the book.

Suppose the students are to have read chapter 1 by Wednesday. On Wed or Fri I will stop during the lecture and write two or three pointed questions on the board, the answers to which are to be found in Chapter 1 of Fung. But I will not tell the students that the answers are in Fung.

Some of these questions will appear on the exam. Students who have read the book will find it very easy to answer them.

A Professor