Junk Charts

A restaurant gem

Felix Salmon spoke highly of this Wall Street Journal chart, and I agree.

Why do I like this? Although it's a basic chart, they did many little things well.

• They are brave enough to not print any of the actual data on the chart. In other words, no loss aversion.

• The legend is integrated onto the chart, not banished to some corner or border, requiring readers to stray from the graph. For added effect, the A, B, C labels imitate the actual signs posted outside the restaurants.

• Simple and effective use of colors

• Sensible scales. It's even better if they would thin out the horizontal scale for the C rating, say make it 10-point intervals instead of 5-point intervals. Although this is hard to accomplish using conventional software, an axis with different intervals in different regions is surprisingly effective.

• Using pencil-thin columns. The same chart with thicker columns would be both uglier and less effective.

(I'm not sure I like the up and right arrows on the axis titles. Is it better to remove the arrows and center the text?)

 

Tough love for chart infidelity

The New York Times Magazine published an article about marriage infidelities, which I didn't read, but it was popular enough that they did an online poll to obtain some instant feedback from readers. The result was shown in this cutesy graphic:

 Note that they plotted the number of responses rather than proportion of responders even though all the numbers are between 0 and 100 and could easily have been misread as percentages.

This chart is another good illustration of the self-sufficiency principle. There is no need to create a chart if all the data are printed onto the chart, and readers must look at the data to learn anything from it. Imagine the above chart without the data, and you'll see why the data labels are critical to this chart.

Below is a version in which I removed all the data labels, replacing them with an axis:

The two pink slabs were thrown in for a little chart-check. According to the designer, 6+6+6 is larger than 20. How is this so? Look at a blow-up of the "God says otherwise" bar of hearts:

 

The one whole heart in each bar ruins the string of half hearts. Little things can introduce infidelities into charts.

All potatoes are not born equal, says chart

Also from Consumer Strategist magazine comes the following chart about "PotatoPacks". (To their credit, the magazine uses a lot of charts, nost of which are completely harmless.)

 This is a good example of what Tufte calls a low data-to-ink ratio. There are exactly 10 pieces of data on this chart, the number of potato packs and the market share for a five-year period.

Much resources have been thrown at the problem of showing growth: it's a surround-sound treatment with loud speakers. The potatoes, the gridlines, the axis, the data labels. And yet, it's unclear what the message is.

According to the title, "both sales volume and market share have steadily increased since the introduction of the PotatoPack." It would have been a very nice touch to add a little arrow letting us know when PotatoPack was introduced. Was it in 2006 the starting point of the data set? Or was it in 2007 when the sales volume started to increase?

***
There is an unintended message. It's that all potatoes are not born equal!

Take a look at the two stacks labeled 572 and 493. How is it that 572 gets us 4 potatoes, and 493 gets us 3 potatoes? So for 2006, each potato is worth 143 packs while in 2007, it's worth 164 packs.

***

For 2010, they plotted projected data, which is exactly how it should be done. 

The following chart shows the year-to-year growth rate of the PotatoPack sales relative to the growth rate of the entire market. This may be the more interesting aspect of this data set.

 

Worst statistical graphic nominated

Phil, over at the Gelman blog, nominates this jaw-dropping graphic as the worst of the year. I have to agree:

Should we complain about the "pie chart"/4 quadrants representation with no reference to the underlying data? Or the "pie within a pie within a pie" invention, again defiantly not to scale? Or the creative liense to exaggerate the smallest numbers in the chart ($2 billion, $0.3 billion) making it disproportionate with the other pieces? Or the complete usurping of proportions (e.g. the $0.2 billion green strip on the top right quadrant compared to the $0.3 billion tiny blue arc on the top left quadrant)?

Or the random sprinkling of labels and numbers around the circle even though if one takes the time, one notices that the entire chart contains only 8 numbers, as follows:

***

Instead, we can display the data with a small multiples layout showing readers how the data is structured along two dimensions.

Or a profile chart may also work:

 

 

The best way to handle two dimensions may be to not use two dimensions

Guess what the designer at Nielsen wanted to tell you with this chart:

Reader Steven S. couldn't figure it out, and chances are neither can you.

What about...

• The smartphone (OS) market is dominated by three top players (Android, Apple and Blackberry) each having roughly 30% share, while others split the remaining 10%.
• The age-group mix for each competitor is similar (or are they?)

Maybe those are the messages; if so, there is no need to present a bivariate plot (the so-called "mosaic" plot, or in consulting circles, the Marimekko). Having two charts carrying one message each would accomplish the job cleanly.

***

Trying to do too much in one chart is a disease; witness the side effects.  

The two columns, counting from the right, contain rectangles that appear to be of different sizes, and yet the data labels claim each piece represents 1%, and in some cases "< 1%".  The simultaneous manipulation of both the height and the width plays mind tricks.

Also, while one would ordinarily applaud the dropping of decimals from a chart like this, doing so actually creates the ugly problem that the five pieces of 1% (on the left column shown here) have the same width but clearly varying heights!

What about this section of the plot shown on the left? Does the smaller green box look like it's less than 1/3 the size of the longer green box? This chart is clearly not self-sufficient, and as such one might prefer a simple data table.

The downfall of the mosaic plot is that it gives the illusion of having two dimensions but only an illusion: in fact, the chart is dominated by one dimension, as all proportions are relative to the grand total.

For instance, the chart says that 6% of all smartphone users are between the ages of 18 and 24 AND uses an Android phone. It also tells us that 2% of all smartphone users are between 35 and 44 AND uses a Palm phone. Those are not two numbers anyone would desire to compare. There are hardly any practical questions that require comparing them.

Sometimes, the best way to handle two dimensions is not to use two dimensions.

***

 The original article notes that "Of the three most popular smartphone operating systems, Android seems to attract more young consumers." In the chart shown below,  we assume that the business question is the relative popularity of phone operating systems across age groups. 

The right metric for comparison is the market share of each OS within an age group.

For example, tracing the black line labeled "Android", this chart tells us that Android has 37% of the 18-24 market while it has about 20% of the 65 and up market. 

Android has an overall market share of about 30%, and that average obscures a youth bias that is linear with age.

On the other hand, the iPhone (green line) has also an average market share of about 30% but its profile is pretty flat in all age groups except 65 and up where it has considerable strength.

Further, the gap between Android and iPhone at the older age group actually opens up at 55 years and up. In the 55-64 age group, the iPhone holds a market share that is similar to its overall average while the Android performs quite a bit worse than its average. We note that Palm OS has some strength in the older age groups as well while the Blackberry also significantly underperforms in 65 and over.

Why aren't all these insights visible in the mosaic chart? It all because the chosen denominator of the entire market (as opposed to each age group) makes a lot of segments very small, and then the differences between small segments become invisible when placed beside much larger segments.

Now, the reconstituted chart gives no information about the relative sizes of the age groups. The market size for the older groups is quite a bit smaller than the younger groups. This information should be provided in a separate chart, or as a little histogram tucked under the age-group axis.

 

 

Why should charts exist?

That sounds like a silly question. Isn't the answer self-evident? Am I suggesting that we banish the discipline of charting?

Maybe I won't go so far. But it's difficult not to have such a destructive thought when one stares at charts like this:

Now, compare the above with this version shown on the right ...  and it's clear all the squares and bubbles and colors gave us nothing. Readers have to read the fine print in order to take in the unequal distribution of income. This chart violates the notion of self-sufficiency we often speak about.

Peering back at the original chart, we find that the entire square grid edifice only serves to explain that 0.01% is one-tenth of 0.1%, which is one-tenth of 1%, etc. On the other hand, the part that has a chance of conveying the main message -- the relative size of the biggest bubble versus the smallest bubble -- is shoved off the screen. The gigantic yellow bubble being mostly off the chart, readers are essentially asked to read the data labels.

***

The same article (via Yahoo!) contains other charts that are well executed. 

This one, for instance, shows the increasing inequality very well. (The legend is on the left panel which I did not include here: the top red line is the top 1%, the other five lines are the quintiles or 20% buckets). At least four-fifths of the country is worse-off now than in 1980 in terms of their share of after-tax income.

A serious magazine seeks our attention, badly

Reader Brian R. could not believe the Atlantic magazine would print a pile of chartjunk like this, and neither do we.

Pretty much every chart deserves its own entry, and they all fail our self-sufficiency test: when the actual data is removed from each chart, the failure is exposed, as one realizes that the graphical constructs do not add to the readers’ experience, and frequently subtract from it.

We'll focus on three examples where they tried to innovate, badly. The data has been stripped from each chart.

The chart shown on the right compares the amount of time spent reading by 15 to 19 year olds in 2007 and in 2009. We definitely see the severe drop in time spent but how many times higher were the average minutes in 2007?

(Amusingly, these books have 13 lines per page, not 12 lines, not 10 lines, not 15 lines.)

The next chart is similar, but comparing the minutes spent playing games. It’s a pie chart! Did our kids spend 100% of their weekend days in 2009 playing games?  

No, it can’t be a pie chart. The caption said “average minutes”, not a proportion of a total; it’s a clock. Is it a 60-minute clock? But it’s a weekend day so maybe it’s a 24-hour clock. That can’t be, since the kids won’t be spending every hour of each weekend day playing games, would they? They do need to sleep, don’t they?

 So we cheat and look at the data. Average minutes in 2007: 46.8 minutes; in 2009, 61.2 minutes. Oh, it’s a malfunctioning clock. In the 2007 version, it’s about a minute too fast, and in the 2009 version, it’s a minute too slow. But who can blame the 2009 clock? You can’t show 61.2 minutes in a 60-minute clock.

With just two pieces of data, it's often the case that graphics are superfluous. Even if "entertainment" is desired, one ought to keep that from obscuring the data. Perhaps like this:

***

 OK, just one more. Not surprisingly, US book  sales are shown using stacks of books except that the data was not encoded in the height of the stacks, the thickness of the books, the number of books, or other usual suspects. The data is embedded into the width of a page plus the thickness of a book, assuming every book is identical in design.

Evidently, book sales declined from 2007 to 2009. By how much? It would be impossible to know without reading the actual data (which I have stripped away).

Since the data is given, we can use a little bit of algebra to figure out how many units are represented by the long side (L) and the short side (S) respectively:

The Atlantic has invented some new math. The long line represents 1.1 billion units while the short line, 3.68 billion units!

What this means is that the difference shown in the picture of one long line is vastly exaggerated; the same difference in units would have been equivalent to one-third of the short line.

***

Other problems noticed by Brian:

Use of what looks like a Gaussian distribution instead of a bar.

The piggy bank graphic that distorts the saving rate.

The redundancy of pie charts next to simple percentages.

Also the presentation of statistics without any apparent relationship between the theme being presented.  For example, what does the increase in 3-D movies being produced have to do with the recession?
My guess more 3-D movies is more due to technology advances and implementation than recession economics.

 

A shark attack on a chart

Happy New Year!

Starting 2011 with shark attacks courtesy of Julien D.

***

It seems like the good chart did not survive a shark attack if one were to judge from what's left of it.

It's a distorted pie chart with some kind of 3D hemispherical add-on, or it's a cross-sectional chart with the top of a sphere lopped off.

Charts of the USA Today variety do not usually feature here but this one has the aspiration to inform readers -- the chart appears on a web page that purports to correct some myths about shark attacks.

The biggest casualty of the shark attack is the ordering of the data labels. It is a brain teaser as to what criterion was used to order the pie slices: it's not the total number of accidents, nor the number of deaths, nor the death rate, nor alphabetical.

It's also unclear why the data labels were made vertical. The palette of colors is, however, typical of pie charts.

Since a death rate is usually defined as deaths / total accidents, not the other way round, even the numerical data labels is harder to read than necessary.

***

There are two primary questions this chart is intended to address: the prevalence of different types of shark attacks, and the death rate of each type of attack (as a proportion of reported accidents).

We try a scatter plot showing these two metrics, adding notes to point out where the interesting data is.

 

 

A not-so-wonderful Bumps chart

The Trifecta checkup requires us to align all three aspects to make a great chart. It is sometimes the case that a wise choice has been made regarding the type of chart, but the other elements are missing. Reader Parker S. sent in an example of such a chart.

This chart created by ESPN illustrates the evolution of the "power ranking" of the San Diego Chargers football team within each 18-week-long season and across multiple years.

The bumps chart is invented for this type of ranking over time data. And in fact, we are looking at a bumps chart.

But with lots of distractions: the multiple colors (instead of year labels), the dots, the legends, the year selector, no foreground (current season).

***
Parker couldn't figure out the practical question this chart is supposed to answer (the top corner of the Trifecta).

It seems to me that the more interesting question is how different teams fare from week to week within a given season, rather than how one team fared from week to week over consecutive seasons.

In fact, one of the secrets of the Bumps chart -- the reason why it feels far less cluttered than it has the right to be -- is that no two data points will overlap, that is, for any given week, only one team occupies any particular rank. This simple rule is violated when the same team's rank across multiple seasons is plotted, and thus the chart feels very busy.

***

It proves impossible to find a source of ESPN power rankings that has all teams for a given season. However, I found something similar at CBS Sportsline, a competitor. Here is their version of the ranking chart:

They got the practical question right but severely under-utilized the form. We can see how the Chargers season is going but have no ability to compare them to other teams.

We can start with the question of visualizing how Chargers and their AFC West compatriots are doing relative to the rest of the league:

The AFC West is a mediocre division this season, with all four teams in the middle of the pack, none in the top quarter of the table. The Chargers started high, plunged and are recovering while the Oakland Raiders have improved over the course of the season.

The Bumps chart is more powerful when the full set of data is plotted, and when the lines are highlighted with reference to the question being answered. Are AFC teams or NFC teams doing better?

The next one highlights the teams that earned the largest change in ranking from week 1 to week 10. The background (gray lines) consists of those teams whose rankings in Week 10 were within 5 places of their initial rankings.

The practical question might be whether Week 1 rankings are a good predictor of Week 10 rankings. The following chart shows that most teams in the top quartile remain there (except San Diego which is coming back, and Dallas which could be coming back too), the bottom-quartile teams also tend to remain there, while not surprisingly, the middle teams don't tend to stay in the middle. The color scheme should be reversed if one wants to highlight the dispersion of the rankings of these middle teams by Week 10.

 

Loss aversion and faux accuracy

Reader Bernie M. is not a fan of this Economist chart.

The chart was prepared by Aurora Flight Sciences, an aircraft manufacturer, commissioned by a professor who supports the concept of maintaining a fleet to pump sulphuric acid into the stratosphere as a way to induce artificial cooling to counteract human-induced global warming.

The chart appears to compare many different ways of shooting the acid into the skies along two dimensions: cost and altitude.

Bernie wrote:

I find the choice of axes extremely counterintuitive. Altitude one would expect on the y-axis. And mixing up the scatter chart elements with the connected line chart doesn' really help either.

The convention regarding axes is to put the outcome variable on the vertical, and the explanatory variable on the horizontal. Thus, in this case, if the cost of a particular solution is primarily determined by the "altitude" (presumably of where the acid would be released), then the designer has followed convention. It is unfortunate that "altitude" is more intuitively put on the vertical axis, but I suspect that defying convention might cause more confusion.

On the other hand, if altitude and cost are not related to each other but two different metrics to evaluate geoengineering concepts, then Bernie's point is right on - swap the axes!

***

The use of connected lines for two of the solutions but not the rest is a symptom of what I have called "loss aversion". The horror of leaving some of the data on the cutting floor.

The only mention of altitude in the article refers to Aurora's assertion that it is sufficient to use newly designed aircraft flying at 20-25 kilometers. If that is Aurora's preferred solution, there is little reason to show all the other altitude configurations that are suboptimal.

Perhaps the designer wants to make the point that the Boeing 747 solution is inferior to the Aurora solution because Aurora could design aircraft to fly at 10-15 km at a lower cost?  If so, then the chart is very misleading in not providing a comparable cost for Boeing's solution if required to fly at 20-25 km.

When comparing different entities, it is always a bad idea to treat the entities differently. Comparison is only possible on equal footing.

In fact, I think the chart would be a lot clearer if they dropped the altitude dimension on the floor. For each solution, plot the yearly cost at the optimal altitude selected by the respective engineers. Use a bar chart. With a single dimension, it is much easier to accommodate the very long data labels.

(Now, I'd defer to the geoengineers as to whether the altitude dimension is dispensable. I don't have any expertise in this science. Judging from the Aurora red line, I'm assuming that there can be feasible solutions at all altitudes, which leads me to conclude that altitude isn't all that.)

***

So what is the biggest problem with this chart? It is the faux accuracy.

Given the tremendous amount of uncertainty surrounding these projected costs, one would expect very big error bars around the cost estimates. Using single dots with no error bars is hard to stomach.