Are science journalists required to take one good statistics course? That is the question in my head when I read this Science Times article, titled "One Cup of Coffee Could Offset Three Drinks a Day" (link).
We are used to seeing rather tenuous conclusions such as "Four Cups of Coffee Reduces Your Risk of X". This headline takes it up another notch. A result is claimed about the substitution effect of two beverages. Such a result is highly unlikely to be obtained in the kind of observational studies used in nutrition research. And indeed, a glance at the source materials published by the World Cancer Research Fund (WCRF) confirms that they made no such claim.
The headline effect is pure imagination by the reporter, and a horrible misinterpretation of the report's conclusions. Here is a key table from the report:
The conclusion on alcoholic drinks and on coffee comes from different underlying studies. Even if they had come from the same study, you cannot take different regression effects and stack them up. The effect of coffee is estimated for someone who is average on all other variables. The effect of alcohol is estimated for someone who is average on all other variables. The average person in the former case is not identical to the average person in the latter case. So if you add (or multiply, depending on your scale) the effects, the total effect is not well-defined.
In addition, you can only add (or multiply) effects if you first demonstrate that the two factors do not interact. If there is interaction, the effect of alcohol is different for people who drink less coffee relative to those who drink more. The alcohol effect stated in the table above, as I already pointed out, is for an average coffee drinker. Conversely, the protective effect of coffee may well vary with alcohol consumption.
***
The reporter also misrepresented the nature of the analysis. We are told: "In the study of 8 million people, cancer risk increased when they consumed three drinks per day. However, the study also found that people who also drank coffee, offset some of the negative effects of alcohol."
The reporter made it sound like a gigantic randomized controlled study was conducted. This is a horrible misjudgment. WCRF did not do any study at all, and certainly no researcher asked anyone to drink specific amounts of alcohol or coffee. The worst is the comment on people who drank coffee as well as alcohol. I can't find a statement in the WCRF report about such people. It's simply made up based on the false logic described above.
***
At one level, the journalist misquoted a scientific report. At another level, the WCRF report is rather disappointing.
The authors of the executive summary repeatedly use the language of causation. For example, "There is strong evidence that being overweight or obese is a cause of liver cancer." Really? Show me which study shows obesity "causes" liver cancer?
Take one of their most "convincing" findings: "Aflatoxins: Higher exposure to aflatoxins and consumption of aflatoxin-contaminated foods are convincing causes of liver cancer." The causation is purely an assumption of the panel who reviewed prior studies. In Section 7.1, readers learn that this cause-effect conclusion comes from "four nested case-control studies and cohort studies" for which "meta-analyses were not possible". So not a single randomized trial and no estimation of the pooled effect.
What is nicely done in the report is the inclusion of "mechanisms" which are speculative explanations for the claimed causal effects. It's great to have thought carefully about the biological mechanisms. Nevertheless, these sections are basically "story time" unless researchers succeed in establishing those unproven links.
Comments