Junk Charts

Back to basics

Today, we review one of the basic principles Ed Tufte very effectively advocated in his famous book: use gridlines and data labels only if absolutely necessary. The enemy is redundancy.

Here is a chart that appeared in the New York Times Real Estate pages: (with this article)

The gridlines serve no purpose. Between the axis labels and the data labels, the designer should pick one. If the data labels are used, then the vertical axis can be removed entirely without affecting our ability to understand the data. One can also argue that the data labels do not convey any real information since the average person is unlikely to be able to process 1004 feet versus 1250 feet. Why not remove the data labels and retain only the axis labels?

I'd be willing to go so far as to remove all data from the chart itself. This is because the Empire State Building has been chosen as the reference point. The assumption behind this choice is that the readers have a sense of "tallness" of the Empire State Building. It is then sufficient to just place columns of different heights next to the Empire State Building. To make the comparison a little easier, one can draw a reference line from the top of the Empire State, like this:

 

 

Spin, spin, spin away

From a purely graphical perspective, the following NYT chart (link) is well executed:

Labeling is always a challenge with scatter plots. Here, they have 54 points, and the chart still doesn't look too crammed. I like the axis labels, and the clear labeling of the four quadrants.

I also like the vertical scale that goes from 4 to 8, despite the scoring range going from 1 to 10. This trims unneeded whitespace and magnifies the differences between nations.

***

In the Trifecta checkup, we also care about what the key question the chart is designed to answer, and how it relates to the graphical element. According to the subtitle, this chart showed that "the nations with more progressive tax rates had happier citizens."

This conclusion certainly does not jump off the page. Reader Christopher L. who submitted this chart found "no obvious trend." (Given the source, I suspect it's the researchers who drew that conclusion.)

There are lots of unanswered questions in an international comparison of subjective results of this type:

• How were the 54 nations chosen?
• Is the year 2007 representative of the recent situation in every one of these countries? Were there any tax reforms in 2007 in any of these countries?
• How reliable is the Gallup poll in each of these countries? How large are the sample sizes? Is it the same survey?
• Why is the difference between the highest and lowest tax burdens the right measure of progressiveness? And are they using the marginal tax rates or the average tax rates? (Judging from the Wikipedia page, there is a lot of arbitrariness in determining a country's tax rate.)
• Are the two data sources comparable? Happiness is a personal question while the range of tax rate is an aggregate metric, with each individual only experiencing one tax rate.

These are not trivial questions. If the data is bad, no amount of graphical magic can save it.

Most vexing for a display like this is that it forces the reader to look for the impact of tax burden on happiness. That's how the question is framed. There is nothing in this chart, though, that suggests that tax rates can explain happiness, and certainly nothing to suggest that low tax rates cause more happiness.

I call this story time. Put up some data, then spin stories and spin away.

 

 

 

Highway Safety Agency goes rogue

A reader sends me to Adam Obeng, who did the dirty work deconstructing a set of charts by the U.S. National Highway Traffic Safety Administration on his blog. Here's an example of these charts:

Aside from the sneaker chart, they concocted a pop stick, a pencil, a tower of Hanoi, etc. These objects are ones I think should be evaluated as art. Adam gamely tells us that the proportions are totally off, and they are both internally and externally inconsitent.

***

I'll add two small points to Adam's post.

First, these charts pass my self-sufficiency test, that is to say, they did not print the entire data set (just one number here) on the page. Alas, given the distortion identified by Adam, not printing the data means everyone is free to create their own data. Herein lies the problem: there is an argument for allowing a small degree of distortion in exchange for "beauty" but these charts without any data have gone too far.

Second, see Adam's last point (the footnote). The original data is something quite convoluted: “3 out of 4 kids are not as secure in the car as they should be because their car seats are not being used correctly.” (How would they know this, I wonder.) This is a statistic about kids while the picture shows a statistic about their parents (or drivers).

 

 

Which software is responsible for this?

@guitarzan wants us to see this chart from north of the border, and read the comments. Please hold your nose first.

 

Here's one insightful comment: "I think it's insane to debate the ages 18 or 19. Why not cap it off at the much more rounded and sensible numbers 18.2 or 19.4??"

Reminds me of signs that say this elevator holds 13 people, or this auditorium holds 147 people safely.

***

I mean, which software package enables this chart?

For the vertical axis, it appears that the major gridlines are specified to 0.4 with minor gridlines at 0.2 apart. The lower limit of the vertical axis was specifically set to 17, which violates the start-at-zero rule for bar charts.

The software also allows the yx-axis labels to be printed twice, one in super tiny font in the expected locations, and the other turned sideways and printed into the bars.

And Canadians, please tell us why the provinces were ordered in this way.

***

This data calls for a simple map, with two colors.

 

A straight line going nowhere fast, despite tweets and likes

Ken B., another Australian reader, wasn't too proud of this effort, apparently excerpted from an HSBC report by the Sydney Morning Herald (link):

Ken: If you plot ranking by ranking it magically turns into a straight line.

***

There are a few other annoyances. Gridlines, data labels, double-edged arrow, bars all based on the same data, which can easily be conveyed with a ranked table. In fact, just turn the chart 90 degrees clockwise, get rid of everything else except the names of countries, and you have a much more readable figure.

The completely unnecessary legend is an Excel special. If only one data series is plotted, it should be automatic to suppress the legend.

The three-letter acronyms for different currencies is a futile educational lesson kind of like plotting geographical data on maps (in many cases). For most readers, the message of the chart does not require knowing the names of the currencies, nor their acronyms. For those who care about acronyms, say currency traders, they most likely already know those letters.

***

Just like I don't understand how we can define "over-rated" or "under-rated" restaurants (see this post and this), I also don't understand how we can define "over-valued" or "under-valued" currencies given the impossiblity of knowing the "true value" of any currency. 

***

I just had to point your attention to the fact that 123 people tweeted this article, and 221 liked this item on Facebook. And these actions form part of the so-called Big Data revolution.

A pretty chart hides the message

James C @annelidworm sent me to this BBC chart, which he thinks is "hard on the eyes":

 

I find a few things I like, and also a few I don't.

Unlike James, I actually find the chart quite pretty. The use of a small-multiples to compare season tickets with single tickets is also nice. For someone like me, who isn't well versed in the British map, the geography lesson is appreciated - although for a local reader, this may be superfluous. The thickness of lines used to encode the data works alright.

There are a few problems with the chart:

There is a self-sufficiency problem. This is a chart in which every data element is printed on the chart, which means the graphical pieces are merely cosmetic. If the data labels were removed, the reader would be entirely lost. However, this problem can be solved by judicious use of colors.

Consider how color is used here. Blue and yellow distinguishes between season and single tickets but the small-multiples setup already does the job well enough. The tint is used in some arbitrary manner unrelated to the data, as far as I can tell.

Instead, price increases above the rate of inflation should be differentiated from price changes below the rate of inflation by using two colors. The special case of Birmingham's season ticket which increased exactly at the rate of inflation deserves its own color.

Speaking of increases relative to inflation. The analyst helpfully explains via the legend that any number above 66 percent is ahead of inflation, and any number below is behind inflation, meaning prices have actually come down. The entire dataset can be simplified by subtracting 66% from each number to show the "real" price changes.

***

Take a step back. What is the story in this dataset? The numbers on the right side are all much higher than those on the left, with Shoeburyness being a bit of the exception. It appears that the rail company is trying to push sales of season tickets. Too bad this chart doesn't bring the story to the front.

Here is one attempt using paired bar charts.

 

 

 

 

English donuts rival Spanish donuts

On my holiday travel, I found a disguised donut chart in the Delta Sky Magazine (Dec 2010), talking about manufacturing jobs in the U.S. Then, flipping through the Spanish section at the back of the same magazine, I found the translated article, plus a translated chart. To my surprise, they look different:

Surprise No. 1: the sizes of the cog wheels are different. Even though the color is still mapped to year in the same way, somehow one of these authors decided to take liberty with the relative size. The suspect is the Spanish author who decided to make 2009 much larger and Jan to Dec 2012 much smaller.

Surprise No. 2: the use of commas within a number, and the format of dates differ by culture. That explains why the Spanish author removed the commas from the numbers, making it harder for me (English-speaking) to comprehend. Also, the swap from "01/12-09/12" to "Sep. 2012" suggests that Spanish speakers don't like the month/year formatting of dates. It also suggests that the Spanish readers have no trouble inferring that the "Sep. 2012" data point refers to "Jan. 2012 to Sep. 2012".

Surprise No. 3: The Spanish author improved the chart in one way. He grouped the annual data together via overlapping, leaving the 2012 partial-year data point by itself.

***

There are some problems with both charts. The most serious is the failure to project the 2012 jobs number. The chart seems to indicate that 2012 is a lackluster year, at best level with the previous years but in fact, the number of jobs in three quarters has already exceeded the full-year count of 2011, 2010 and 2009. Unless the fourth quarter is a particularly bad quarter for manufacturing jobs, it would seem that the message should be that 2012 is a great year of recovery. You can't tell from these charts: in particular, the Spanish author decided to shrink the 2012 cog wheel into insignificance.

The issue here is providing context for comparison. Even if the projected 2012 full-year number is provided, that may not be enough to judge whether manufacturing is healthy. Other useful context can be the growth rate of manufacturing versus other sectors of the economy; and the growth rate of jobs in relation to the population/work force growth rate.

As usual, a simple line chart displays the time-series data more clearly. (I simply linearly extrapolated the 2012 full-year number, which is probably an over-estimate. In practice, you can look up the data and figure out the ratio of Jan-Sept jobs to full-year jobs on average and inflate the number that way.)

 

 

 

Budding graphics connoisseurs from Down Under

A reader, Stephen M., who's a high school math Information Technology teacher in Australia, assigned the following chart to his class as a Junk Charts style assignment. (link to original here)

We have seen racetrack charts before (e.g. here or here), and we have dual racetracks here.

Stephen's class identified the following problems with the chart:

- The group agreed this should be better called a data visualisation than an infographic

- The purpose of the 'infographic' seems to be more on the design/form, than the function of conveying an understanding of the data

- There seems to be a bit of an optical illusion with the lower upper circle for the US appearing larger than the upper lower one (we checked, there isn't)

- There are no clear labels to assist. It is an assumption that because in the heading and the figures, population is on top of donations, that the lines are the same. The class agreed that country labels would help to the left of each line start.

- No scale on the lines and where do you measure from/to (especially as the US line is a single line for a proportion of the way

- It's too abstract and the spatial separation of the curves makes comparison difficult.

***

Wow, that's great critique from the 16-year-olds. They are working on ways to re-make this graphic. One good idea is to collapse the two dimensions into one: per-capita donations.

Another issue with this chart is that the countries are sorted in different ways from one chart to the next. It's really difficult to compare one country to another.

It is also instructive to discuss what the key message is in this data. Why those six countries? What kinds of donations are being counted? Do the counting methodology differ by country? How comparable is the data?

Finally, is this art or is this science?

P.S. [12/2/2012] Stephen noted that another deficiency identified by the students is the lack of sourcing. Indeed, where did the data come from? They think it's the CIA Factbook.

How to fail three tests in one chart

The November issue of Bloomberg Markets published the following pair of pyramid charts:

This chart fails a number of tests:

Tufte's data-ink ratio test

There are a total of six data points in the entire graphic. A mathematician would say only four data points, since the "no opinion" category is just the remainder. The designer lavishes this tiny data set with a variety of effects: colors, triangles, fonts of different tints, fonts of different sizes, solid and striped backgrounds, and legends, making something that is simple much more complex than necessary. The extra stuff impedes rather than improves understanding. In fact, there were so many parts that the designer even forgot to add little squares on the right panel beside the category labels.

Junk Charts's Self-sufficiency test

The data are encoded in the heights of the pyramids, not the areas. The shapes of the areas are inconsistent, which also makes it impossible to decipher. The way it is set up, one must compare the green, striped triangle with two trapezoids. This is when a designer realizes that he/she must print the data labels onto the chart as well. That's when self-sufficiency is violated. Cover up the data labels, and the graphical elements themselves no longer convey the data to the readers. More posts about self-sufficiency here.

Junk Charts's Trifecta checkup

The juxtaposition of two candidates' positions on two entirely different issues does not yield much insights. One is an economic issue, one is military in nature. Is this a commentary of the general credibility of the candidates? or their credibility on specific issues? or the investors' attitude toward the issues? Once the pertinent question is clarified, then the journalist needs to find the right data to address the question. More posts about the Trifecta checkup here.

Minimum Reporting Requirements for polls

Any pollster who doesn't report the sample size and/or the margin of error is not to be taken seriously. In addition, we should want to know how the sample was selected. What does it mean by "global investors"? Did the journalist randomly sample some investors? Did investors happen to fill out a survey that is served up somehow?

***

The following bar charts, while not innovative, speak louder.

Purists stay clear; many rules to be broken

This chart, from Internet Retailer (March 2012, p. 26), is okay, at least they didn't use pie charts. But it could have been much more effective.

To make it better, we have to break all the rules:

• Use lines instead of columns. The following reproduces the right side of the chart above which deals with shopping behavior by income groups. One of the issues with this chart is that the gray partition between the gender section and the income section is too incognito.

• Use rounded percentages instead of two decimal places. If we need to see two decimal places in order to tell two categories apart, then the difference is surely insignificant.

• Round up group boundaries. Notice that I committed the sin of imprecision by not specifying whether $60,000 belongs to $30-60K or $60-100K. Is that level of precision necessary? Is it worth the ugliness of printing a number like $59,999? If there is information in whether $60,000 belongs to $30-60K or $60-100K, what does that say about your analysis?

• Dare to throw away information. Look at this final version of the above charts:

I have collapsed the five income groups into two. That's because the bottom three income groups are more or less the same in terms of online/offline shopping behavior, and the top two groups are more or less the same. Bear in mind that any analysis of this type has a margin of error so differences of a few percentage points are not worth representing. It is possible that even the 10 percent or so differences between the two remaining income groups are not meaningful--we won't know unless we know the margin of error of their methodology.

(Note that you have to consult the Census to get the relative proportions of each income group in order to average the data shown in the original chart.)

In any case, the key message of the article becomes a lot clearer than in the original chart.

• Use black and white instead of color. Having simplified the presentation--but retaining all the important information--we no longer need colors.

Start breaking some rules today, and you'll make better charts.