Darin M. points us to this speedometer chart, produced by IBM (larger version here). They call it the "Commuter Pain Index". I call it a prickly eyebrow eyelashes chart. You be the judge.
The "eyebrows" on this chart are purely ornaments. The only way to read this chart is to read the data labels, so it is a great example of failing the self-sufficiency test.
The simplest way to fix this chart is to unwrap the arc, turning this into a bar chart. The speedometer is a cute idea but very difficult to pull off because the city names are long text fields, and variable in length.
First, we must fix the vertical scale. For column charts, one must start at zero, without exceptions. The effect of not starting at zero is to chop off an equal length piece from the bottom of each column, and in doing so, the relative lengths/areas of the columns are distorted. The amount of distortion can be very severe. For example look at the fourth set of columns as shown below:
In both charts, I made the length of the first column the same so we are staring at comparative charts. The data plotted is exactly the same; the only difference is that the left chart starts the axis at zero. Notice that the huge difference seen on the right chart for the 4th pair of columns does not appear as extraordinary when the proper scale is used.
A multitude of other problems exist, not the least this is a chart that is highly redundant. The same data (10 numbers) show up three times, once as data labels, once as column lengths (distorted), and once as levels on the vertical scale.
An alternative way to look at this data is the Bumps chart. Like this:
What this chart brings out is the variability of the estimated vehicle densities. In theory, the density estimate should be quite accurate for the "today" numbers. You'd think that in surveying 2,000+ people about how many vehicles they currently own, most people should be able to provide accurate counts.
The data paint a different picture. From quarter to quarter, the estimated "today" density shows a range of 1.90x to 2.00x in the 5 periods analyzed, which is roughly 5%, a difference which, according to the analyst, equates to 5 million vehicles! Given current vehicle sales of about 13 million per year, 5 million is almost 40% of the market.
So, one wonders how this survey was done, and one wants to know how large is the margin of error of this estimate. I also want to know if the survey produces estimates of number of households as well since the vehicle per household metric has two variable components.
Craig N. sent us to this infographic from Fast Company about MTV's 30th anniversary, nominating it as the worst infographic ever.
Apply the self-sufficiency test to this chart. Wish away the printed data. Now, does the chart convey any message? Where is the data embedded? Is it in the white dot, the black dot, the gold ring, the gold disc, the black ring, the eye-white? All of the above?
Now, do the same test on this chart (I removed the sales data, replacing it with years):
How would one compare the white to the orange? If one measures the lengths of the sides, the ratio of white to orange is about 1.32. If one compares areas of the squares, then the ratio is 1.73. Note that this requires the reader to see through the orange area to size up the area of the large white square. Alternatively, we can compute the ratio of the white area as observed to the orange square, and that ratio is 0.73.
The real ratio between 1980 and 2010 sales is given as 3.9/2.7 = 1.44. Given rounding errors, it seems like the designer may have used a ratio of lengths of the sides.
The problem is the same whether sides or areas are used. Can the reader figure out that the 1980 sales is about 40% higher than the 2010 sales?
I suspect that most of us react primarily to the visible areas, which means that we'd have gotten the direction of the change wrong, let alone the magnitude.
Craig really dislikes this one. It's a variant of the racetrack chart. As any athlete knows, inner tracks are shorter than outer tracks. Could it be that days have gotten longer in the last 30 years? Apparently, the editors at Fast Company think so.
The New York Times Magazine published an article about marriage infidelities, which I didn't read, but it was popular enough that they did an online poll to obtain some instant feedback from readers. The result was shown in this cutesy graphic:
Note that they plotted the number of responses rather than proportion of responders even though all the numbers are between 0 and 100 and could easily have been misread as percentages.
This chart is another good illustration of the self-sufficiency principle. There is no need to create a chart if all the data are printed onto the chart, and readers must look at the data to learn anything from it. Imagine the above chart without the data, and you'll see why the data labels are critical to this chart.
Below is a version in which I removed all the data labels, replacing them with an axis:
The two pink slabs were thrown in for a little chart-check. According to the designer, 6+6+6 is larger than 20. How is this so? Look at a blow-up of the "God says otherwise" bar of hearts:
The one whole heart in each bar ruins the string of half hearts. Little things can introduce infidelities into charts.
Reader Chris P. found this chart on Visualizing.org, which is one of those sites that invite anyone to contribute graphics to it:
It looks like the designer has taken Tufte's advice of maximizing data-to-ink ratio too literally. There are many, many things going on in a tight space, which leaves the reader feeling drugged-up and cloudy.
From a cosmetic standpoint, fixing the following would help a lot:
Make fonts 1-2 points larger in all cases, especially the text on the left hand side
Use colors judiciously to stress the key data. In this version, the trends, which are more interesting, are shown in pale gray while the raw data, which are not very exciting, are shown in loud red. Just flip the gray with the red.
Rethink the American flag motive: is drug abuse a uniquely American phenomenon? Should data about the American people always be accompanied by the American flag?
Separately present in two charts the time-series data on total arrests, and the cross-sectional data (2008)
Also, realize that by forcing the data into the 50-star configuration, one arbitrarily decides that the data should be rounded to 2-percent buckets. (see right).
And always ask the fundamental question: what makes this data tick?
As I explored the data, I noticed various arithmetic problems. For example, the arrests by race analysis is itself split into two parts: White/black/Indian/Asian add up to 100 percent and then Hispanic Latino and Latino non Hispanic add up to 100 percent. In some surveys, Hispanics are counted within whites but that doesn't seem to be the case here. The numbers just don't add up.
Also, adding the types of drugs involved does not yield the total number of arrests. Perhaps the category of "others" has been omitted without comment. Now I closed my eyes and proceeded to make a chart out of this.
The new version focuses on one insight: that certain races seem to get arrested for certain drugs. The relative incidence for arrests are not similar among the races for any given drug. Asians and Native Americans appear to have higher proportions of people arrested for marijuana or meth while blacks are much more likely to be arrested for crack.
You're going to need to click on the chart for the large version to see the text.
Doing this chart gives me another chance to plug the Profile chart. We deliberately connect with lines the categorical data. The lines are meant to mean anything; they are meant to guide our eyes towards the important features of the chart.
One can sometimes superimpose all the lines onto the same plot but the canvass clogs up quickly with more lines, and then a small-multiples presentation like this one is preferred.
We have a temptation to generalize arrest data to talk about drug habits by race but if you intend to do so, bear in mind that arrests need not correlate strongly with usage.
This chart, found in Princeton Alumni Weekly, only partially scanned here, supposedly gave reasons for "Princeton's top-rated [Ph.D.] programs" "to celebrate". My alma mater has outstanding academic departments, but it would be difficult to know from this chart!
Due to the color scheme, the numbers that jump out at you are the ones in the bright orange background, which refers to how many other departments are ranked equal to Princeton's in those subjects. It takes some effort to realize that the more zeroes there are in the top buckets (fading orange), the better.
The editor started with a nice idea, which is to convert raw rankings into clusters of rankings. She recognized that in this type of rankings (see a related post on my book blog here), it is meaningless to gloat about #1 versus #2 because they are probably statistically the same. For instance, in the ranking of Architecture departments (ARC), 37 schools (including Princeton) all belonged to the same cluster as Princeton, judged to be a statistical tie.
One of the main reasons why this chart looks so confusing is its failing the self-sufficiency test. It really is a disguised data table, with some colorful background and shadows; the graphical elements add nothing to the data at all. If one covered up all the data, there is nothing left to see!
In the following rework, I emphasize the cluster structure. Each subject has three possible clusters, schools ranked above, equal to, and below Princeton. Instead of plotting raw numbers, the chart shows proportions of schools in each category. The order is roughly such that the departments with the relatively higher standing float to the top. Because a bar chart is used, the department names could be spelt out in their entirety and placed horizontally.
If one has access to the raw data, it would be even better to reveal the entire cluster structure. It is quite possible that the clusters above and below Princeton can be further subdivided into more clusters. This will allow readers to understand better what the cluster ranks mean.
Guess what the designer at Nielsen wanted to tell you with this chart:
Reader Steven S. couldn't figure it out, and chances are neither can you.
The smartphone (OS) market is dominated by three top players (Android, Apple and Blackberry) each having roughly 30% share, while others split the remaining 10%.
The age-group mix for each competitor is similar (or are they?)
Maybe those are the messages; if so, there is no need to present a bivariate plot (the so-called "mosaic" plot, or in consulting circles, the Marimekko). Having two charts carrying one message each would accomplish the job cleanly.
Trying to do too much in one chart is a disease; witness the side effects.
The two columns, counting from the right, contain rectangles that appear to be of different sizes, and yet the data labels claim each piece represents 1%, and in some cases "< 1%". The simultaneous manipulation of both the height and the width plays mind tricks.
Also, while one would ordinarily applaud the dropping of decimals from a chart like this, doing so actually creates the ugly problem that the five pieces of 1% (on the left column shown here) have the same width but clearly varying heights!
What about this section of the plot shown on the left? Does the smaller green box look like it's less than 1/3 the size of the longer green box? This chart is clearly not self-sufficient, and as such one might prefer a simple data table.
The downfall of the mosaic plot is that it gives the illusion of having two dimensions but only an illusion: in fact, the chart is dominated by one dimension, as all proportions are relative to the grand total.
For instance, the chart says that 6% of all smartphone users are between the ages of 18 and 24 AND uses an Android phone. It also tells us that 2% of all smartphone users are between 35 and 44 AND uses a Palm phone. Those are not two numbers anyone would desire to compare. There are hardly any practical questions that require comparing them.
Sometimes, the best way to handle two dimensions is not to use two dimensions.
The original article notes that "Of the three most popular smartphone operating systems, Android seems to attract more young consumers." In the chart shown below, we assume that the business question is the relative popularity of phone operating systems across age groups.
The right metric for comparison is the market share of each OS within an age group.
For example, tracing the black line labeled "Android", this chart tells us that Android has 37% of the 18-24 market while it has about 20% of the 65 and up market.
Android has an overall market share of about 30%, and that average obscures a youth bias that is linear with age.
On the other hand, the iPhone (green line) has also an average market share of about 30% but its profile is pretty flat in all age groups except 65 and up where it has considerable strength.
Further, the gap between Android and iPhone at the older age group actually opens up at 55 years and up. In the 55-64 age group, the iPhone holds a market share that is similar to its overall average while the Android performs quite a bit worse than its average. We note that Palm OS has some strength in the older age groups as well while the Blackberry also significantly underperforms in 65 and over.
Why aren't all these insights visible in the mosaic chart? It all because the chosen denominator of the entire market (as opposed to each age group) makes a lot of segments very small, and then the differences between small segments become invisible when placed beside much larger segments.
Now, the reconstituted chart gives no information about the relative sizes of the age groups. The market size for the older groups is quite a bit smaller than the younger groups. This information should be provided in a separate chart, or as a little histogram tucked under the age-group axis.
Reader Tyson A. serves dessert for dinner, and stacked pancakes are on the menu!
According to the St. Louis Beacons that published these charts (and more):
These pie charts take the individual states' percentages, split them up and then stack them. In this way, you can see how the proportion of taxes in each category collected by each state compares with the states around it.
This presentation fails our self-sufficiency test: one is completely lost if the entire data set was not printed on the chart itself.
The pie pieces apparently lost shape as they got stacked on top of each other. The top green slice labeled Tennessee represents 2.1% but look at the difference between the green Nebraska (40%) and the green Kansas (40.8%), for example.
Also, the red pieces and the green pieces are ordered on their own so that the Tennessee red is near the bottom of the stack while the Tennessee green is at the top.
This data can be shown clearly in a pair of line charts.
To really learn something about the data, we can create a scatter plot.
From this plot, we see that most of these states (clustered in the middle) have similar taxation policies.
The exceptions are Illinois and Tennessee, and to a lesser extent, Missouri.
Reader Brian R. could not believe the Atlantic magazine would print a pile of chartjunk like this, and neither do we.
Pretty much every chart deserves its own entry, and they all fail our self-sufficiency test: when the actual data is removed from each chart, the failure is exposed, as one realizes that the graphical constructs do not add to the readers’ experience, and frequently subtract from it.
We'll focus on three examples where they tried to innovate, badly. The data has been stripped from each chart.
The chart shown on the right compares the amount of time spent reading by 15 to 19 year olds in 2007 and in 2009. We definitely see the severe drop in time spent but how many times higher were the average minutes in 2007?
(Amusingly, these books have 13 lines per page, not 12 lines, not 10 lines, not 15 lines.)
The next chart is similar, but comparing the minutes spent playing games. It’s a pie chart! Did our kids spend 100% of their weekend days in 2009 playing games?
No, it can’t be a pie chart. The caption said “average minutes”, not a proportion of a total; it’s a clock. Is it a 60-minute clock? But it’s a weekend day so maybe it’s a 24-hour clock. That can’t be, since the kids won’t be spending every hour of each weekend day playing games, would they? They do need to sleep, don’t they?
So we cheat and look at the data. Average minutes in 2007: 46.8 minutes; in 2009, 61.2 minutes. Oh, it’s a malfunctioning clock. In the 2007 version, it’s about a minute too fast, and in the 2009 version, it’s a minute too slow. But who can blame the 2009 clock? You can’t show 61.2 minutes in a 60-minute clock.
With just two pieces of data, it's often the case that graphics are superfluous. Even if "entertainment" is desired, one ought to keep that from obscuring the data. Perhaps like this:
OK, just one more. Not surprisingly, US book sales are shown using stacks of books except that the data was not encoded in the height of the stacks, the thickness of the books, the number of books, or other usual suspects. The data is embedded into the width of a page plus the thickness of a book, assuming every book is identical in design.
Evidently, book sales declined from 2007 to 2009. By how much? It would be impossible to know without reading the actual data (which I have stripped away).
Since the data is given, we can use a little bit of algebra to figure out how many units are represented by the long side (L) and the short side (S) respectively:
The Atlantic has invented some new math. The long line represents 1.1 billion units while the short line, 3.68 billion units!
What this means is that the difference shown in the picture of one long line is vastly exaggerated; the same difference in units would have been equivalent to one-third of the short line.
Other problems noticed by Brian:
Use of what looks like a Gaussian distribution instead of a bar.
The piggy bank graphic that distorts the saving rate.
The redundancy of pie charts next to simple percentages.
Also the presentation of statistics without any apparent relationship between the theme being presented. For example, what does the increase in 3-D movies being produced have to do with the recession? My guess more 3-D movies is more due to technology advances and implementation than recession economics.