Previously, we looked at the claim that 7 percent of American adults believe that chocolate milk comes from brown cows. The media, including mainstream outlets such as the Washington Post, demonstrated incompetence in reading survey results.
That was before I investigated the other survey covered by the Washington Post in that article about brown cows (link). The Post should immediately apologize for their horrible reporting. For this study, the data are published, and so no excuses. The reporter used this study to make a case for why the brown-cow study is credible:
One Department of Agriculture study, commissioned in the early ’90s, found that nearly 1 in 5 adults did not know that hamburgers are made from beef.
Turns out they just made this up.
I found this right there in the abstract:
Data were collected from 2,005 respondents representing the following groups: purposely selected primarily white Indiana high school students and primarily black Michigan high school students, randomly selected rural Missouri adults attending one of several town meetings, and randomly selected urban Missouri adults contacted in various settings (including churches, libraries, and grocery stores).
Searching for the word "hamburger," I learned that respondents were asked to agree or disagree with the statement "hamburger is made from the meat of pigs". The results were presented in multiple tables, covering the aggregate sample population and subgroups. For each statement, the researchers listed "% correct", "% incorrect" and "% don't know."
The next time "hamburgers" appeared was on page 37 where the statement about hamburgers was highlighted as one of the top five statements that respondents were most likely to get right (!) To the extent that this report demonstrated illiteracy of agricultural facts, the hamburger example is one of the least powerful evidence.
Moving on to the two adult subgroups. As shown in Table 19, rural Missouri adults answered the hamburger question correctly 94 percent of the time (!!) And in Table 20, urban Missouri adults were correct 80 percent of the time (roughly the same proportion as the aggregate sample).
***
So, let's count the ways the Washington Post messed up:
1. What they reported applied to urban Missouri adults only, and definitely not "adults" nationwide.
2. The urban Missouri adults in the survey were not randomly selected from all urban Missouri adults, and so they might not even represent the average Missouri urban adult.
3. Eighty percent correct is not the same as 20 percent (1 in 5) "did not know," since the survey had a third category called "Don't Know".
4. The statement respondents reacted to was "hamburger is made from the meat of pigs". If someone said this statement is false, we still do not know if this person knows if "hamburger is made from beef".
5. We don't even know if hamburgers are made with 100% beef. In fact, there were reports in recent years that horse meat and other meat might have been mixed in.
Comments