When nothing works
Sep 13, 2005
Here's a chart to delight those seeking teaching materials. The chart summarizes responses to two questions in a marketing survey about "storage virtualization"; it came from an IT publication, for those of you wondering.
It manages to violate almost every rule of good chart-making. Lets make a list:
1. "Hide the message": it would appear that these two questions were posed in order to understand which benefits/pitfalls potential customers would consider "most important", "important" and "not so important". This information is squeezed to the side of the page, turned vertical, and written in black font that is totally overshadowed by the brightly-lit bars.
2. Chartjunk: Tufte would have a field day with this chart. Those vertical gridlines are utterly useless as the value for each bar is provided. Drawing in gridlines without a scale compounds the problem. The reader needs to guess that the survey probably asked respondents to rank each benefit/pitfall from least important (1) to most important (5). Besides, the choice of color is inexplicable, taking attention away from the key message.
3. Carelessness: the bottom chart is unrecyclable junk: the printed numbers do not match the lengths of the bars! Take the last set "Technology too immature": the red bar (3.34) is shorter than the orange (3.19). Thus, we have to throw out the bottom chart. Even then, a careless mistake like this destroys the graph's credibility completely.
4. "More is Less": this often happens when data get divided into segments without revealing much insight. If we are to be shown large, medium and small businesses separately, we should be able to easily pick up differences between the three segments. Looking at the top chart, I observe that almost every rating is between 3 and 4, regardless of size of business. The key message can be reinforced if the three segments are combined into one.
When studied more carefully, one can see differences in ranking of the benefits across segments. But these bar charts obscure rank information. In fact, for this data, the relative ranking of the benefits is probably more meaningful than the actual ratings and so even the metric has not been chosen judiciously.
This pair of charts should have been left on the cutting room floor. Ironically, they appeared in the inaugural edition of what was heralded as a new beginning for a magazine. This was a magazine I thought was the best in the computer networking industry. They need a good graphics editor, presto!
Reference: "Virtual Necessity ", IT Architect, Sept 2005.
PS. It appears that the mistake in the bottom chart has been caught, after the paper version went to print, as the on-line version of the graph does not contain the error.
For the first question, it seems that large organizations rate the attributes on a different scale. I wonder what the chart would look like if you normalized.
Posted by: John | Sep 14, 2005 at 10:22 AM
As you said, it is clear that the large organizations tend to give higher ratings for all categories so normalizing can help.
This actually raises another troubling issue, concerning how they decided which categories are "most important", "important" and "not so important". In the top chart, this appears to be solely determined by ranking from the large businesses. Was that the rule they used or was it a coincidence, it's hard to tell.
Posted by: Kaiser | Sep 14, 2005 at 05:21 PM