Reader Chris P. alerted us to a fascinating post from Errol Morris' blog, which presents results in graphical form from a readers' poll related to this other post. This other post deals with a pair of photographs taken during wartime, previously discussed by Susan Sontag and others. Sontag believed the pair documented a before-and-after setting: it was alleged that the photojournalist shifted some cannon balls from their natural position between takes.
Morris polled his readers asking them in which order they thought the photos were taken ("on before off", "off before on", "undecided"), and which factors were used to make the decision. He presented results in two formats, first plotting frequencies in bar charts and then plotting proportions in pie charts. He preferred the pie chart construct.
Most here would share Chris' reaction: "Oh my. What people do with Excel."
The biggest problem with these pie charts is the unreasonable baseline. This is one of those polls that allow respondents to pick any number of factors and clearly, the pie chart creator used the 1,151 responses as the baseline, as opposed to 910 people who voted. Consider these two statements:
- 52% of respondents who decided "on before off" listed "sun shadow" as a decision factor
- 30% of the decision factors submitted by respondents who decided "on before off" were "sun shadow"
It is tough to figure out what the second statement means. It is as if the respondent who selects more than one factors gets more than one votes in the final tally. To put it differently, the 30% is meaningless unless one also knows how many decision factors were selected by each respondent, on average and in distribution. The 52% is independent of such consideration.
Combining the data given in the bar charts and pie charts, one discovered that 469 out of 910 respondents could not decide which photo was taken before the other; besides, these respondents on average expressed 0.9 opinions on the decision factors whereas the respondents who made a decision expressed 1.6 opinions.
A simple illustration to show the key decision variables by type of respondents is shown below. From this chart, one sees that the number and position of the cannon balls were crucial to at least 50% of those who came to a conclusion. Sun shadow were much more important to those who decided "on before off" while those who decided "off before on" noticed character artistic, shelling and rocks. Most other factors did not differentiate the three groups.
Source: "Not Your Mum's Apple Pie Chart", Errol Morris, Dec 18, 2007.