My letter to the New York Times is published

Some of you might be wondering why I haven’t commented on the front-page feature in New York Times Magazine two weeks ago, titled “When the Revolution Came for Amy Cuddy.” Susan Dominus, the author of the front-page feature, cited an article about Cuddy's "power pose" research that Andrew Gelman and I co-authored in 2016 as an example of a vicious personal criticism on Cuddy, in fact, one of the very few actual examples of vicious personal criticism that Dominus repeatedly told readers has destroyed civility in the field of social psychology.

You can now read my response to the NYT Magazine article in the now-published Letter to the Editor. (Scroll to the second entry.) I didn’t say much before because I didn’t want to front-run the letter but I do have lots to say. Since the NYT letter is limited to 200 words, it is a much abridged version of what I had in mind.

I find Susan’s interpretation of the events to be unbalanced. If you are a curious, or serious, reader, I urge you to follow the links below (in addition to reading Dominus's feature article), and form your independent judgment of the movement to reform the use of data in psychological experiments.

1. Susan Fiske, who is the academic advisor of Amy Cuddy at Princeton, is mentioned in the NYT Magazine. An important contribution of hers to this debate is a "Presidential Guest Column" in APS Observer, titled "Mob Rule or Wisdom of Crowds?," in which she coined the term "methodological terrorism." Prof. Fiske is a past president of Association for Psychological Science (APS). The entire column can be read here.
   
   
2. The NYTM article mentioned a “combative” response by John Bargh, Yale professor and father of the seminal priming study, to the failed replication study. He published his response on Psychology Today, a blogging platform. The blog post was subsequently deleted but you can find the post titled “Nothing in Their Heads” archived here.
   
   
3. Daniel Kahneman's letter to priming researchers is cited by Dominus, and deserves to be read in full. Dan Goldstein has a copy of it here. There are many good quotes, including "Your problem is not with the few people who have actively challenged the validity of some priming results. It is with the much larger population of colleagues who in the past accepted your surprising results as facts when they were published."
   
   
4. Andrew Gelman, professor of statistics at Columbia, regularly blogs about the validity of social science research at his popular blog. As Dominus noted, the power pose research started by Dana Carney, Amy Cuddy and Andy Yap is one recurring topic on the blog.  Gelman has many other favorite, recurring targets, for example, Daryl Bem (ESP research, Cornell), Brian Wansink (Food and Brand Lab @ Cornell), Richard Tol (climate economics), Roy Baumeister (ego depletion), Satoshi Kanazawa (evolutionary psychology, LSE), etc. I can't help but notice that almost everyone on this list is male.
   
   
5. The Data Colada website, run by Uri Simonsohn and Joe Simmons, is only one click away. Given that a whole meal was made of the inner workings of a single blog post that criticized the Carney, et. al. study, the reader should absolutely check out this website where serious scientists are discussing how to reform science.
   
   
6. The original paper by Carney, Cuddy and Yap (2010) that launched this power pose research program is found here. If you don't have time to read the whole paper, at least read the abstract, for what their original scientific claim was:
   
   Humans and other animals express power through open, expansive postures, and they express powerlessness through closed, contractive postures. But can these postures actually cause power? The results of this study confirmed our prediction that posing in high-power nonverbal displays (as opposed to low-power nonverbal displays) would cause neuroendocrine and behavioral changes for both male and female participants: High-power posers experienced elevations in testosterone, decreases in cortisol, and increased feelings of power and tolerance for risk; low-power posers exhibited the opposite pattern. In short, posing in displays of power caused advantaged and adaptive psychological, physiological, and behavioral changes, and these findings suggest that embodiment extends beyond mere thinking and feeling, to physiology and subsequent behavioral choices. That a person can, by assuming two simple 1-min poses, embody power and instantly become more powerful has real-world, actionable implications.
   
   
7. Here is the Ranehill, et. al. replication study. Here is the full text of Dana Carney's renunciation of power pose research, in which she disclosed how the steps taken during the data analysis could have led to a false-positive finding. Here is the full press release by Amy Cuddy's publicist to New York Magazine, which is Cuddy's first public response to the Ranehill et. al. study.
   
   
8. Andrew Gelman has given a series of talks to academic conferences and academic departments on how to reform the use of data evidence in psychology and/or social science. For example, ESRC Research Methods Festival at University of Bath, Columbia Computational Social Science, Centre for Research in Statistical Methodology at Warwick, Department of Psychology at UT Austin, Psych Colloq at Harvard, Psychology/Statistics at Ohio State University, etc. etc. This is not a comprehensive list - I used Bing and Gelman's blog and all these showed up just in the first page of results.
   
   
9. Here is the article Andrew and I co-authored about our general concern about popular press and social media promulgating peer-reviewed studies that have weak statistical foundations, using the power pose research as an example.

****

One thing to which the NYTM article did not do justice is the underlying, deep issues related to how evidence from data is used in social science. In November 2016, I wrote a series of blog posts to explain the statistical issues being discussed. The agenda:

Key Idea 1: Peer Review, Manuscripts, Pop Science and TED Talks (link)

Key Idea 2: P < 0.05, P-hacking, Replication Studies, Pre-registration (link)

Key Idea 3: Negative Studies, and the File Drawer (link)

Key Idea 4: Degrees of Freedom, and the Garden of Forking Paths (link)

Key Idea 5: Sample Size (link)

Comments

[Ken]

One of the things that behavioural researchers should start doing is to write protocols, and those would require registration for publishing in journals, the same as is done for medical research. Then it would be obvious how many outcomes they actually used, which hopefully would make them limit themselves to one. Then they would only be troubled by odd results, which could be checked by replication.

[Kaiser]

Ken: there is indeed a debate about pre-registering studies. The concern may be that there are sometimes things you discover during the studies that may be real effects that weren't anticipated. So there is pushback.