Junk Charts

A shocking failure to communicate

So said a reader, Stephen B., of the following graphic (note: pdf) in the London Times concerning Andy Murray's recent tennis triumphs.

How can we disagree?  Shocking?  Yes.  Failure?  Definitely.  Failing to communicate?  No doubt.

Let's first start with the five tennis balls at the bottom.  It fails the self-sufficiency test.  It makes no difference whether the balls (bubbles) are the same size, or different sizes.  Readers will look at the data and ignore the bubbles.

Amazingly, the caption said that "Murray has one of the best returns of serve in the game."  And yet, the graphic showed the five players who were better than Murray, and nobody worse!  For those unfamiliar with tennis statistics, it does not provide any helpful statistics like averages, medians, etc. to help us understand the data.

But that is only the beginning.

Take a look at these two donuts.

(The color scheme from light to dark: first, second, third, fourth round of tournament)

So we're told: the 75% of first-serve points won in the fourth round was 25.6% of the sum of the percentages of first-serve points won from first to fourth rounds (75%+70%+71%+76%).  What does this mean?  Why should we care?

The challenge with these two statistics is that they are correlated and have to be interpreted together.  If a first-serve is won, then there would be no second serve, etc.  Here's one attempt at it, using statistics from the Soderling-Federer match.  It's clear that Federer was better on both serves.

Reference: "Murray's march to the last eight", London Times.

 

Reading comprehension

Note: I am in the middle of a holiday and so posting will be limited.

Andrew posted a pretty chart that caught my attention.  This is the sort of sophisticated chart that rewards careful reading. 

Below is a guide to reading the chart:

• It is a small multiples chart with the components arranged in two dimensions (income levels, and a race-religion hybrid category).  The top row is a summary of voters of all race-religion grouped by income.  Note that there is no corresponding summary column for voters of all incomes grouped by race-religion.
• Source of data: 2000 poll but applied to 2008 demographic patterns.  In other words, there is an underlying assumption that opinions have stayed stable within the demographic groups.
  
• The chart is in fact three dimensional because each map gives us the geographical (state by state) breakdown.
• It is useful to figure out the smallest unit of data: in this case, this is the percentage support of federal school vouchers by voters of a given race-religion-income-and-state category.
• The color scheme is such that red represents highest support and blue lowest support, with pink and purple in the middle
• It's almost always better to start from the aggregate (that is, the average) and then study variations along different dimensions, and this is how the chart is arranged from top to bottom
  
• On the top row, the higher income groups tended to favor vouchers more than lower income groups, with a break point around $75k; even here, the regional differences are significant, with northeast and southwest hotter for vouchers at all income levels
• As we move from row to row, we realize that the aggregate data hides many disparities.  For example, white Catholics (second row) are more likely to support vouchers regardless of income level while white non-evangelical Protestants (fourth row) are much less likely than average to support vouchers at all income levels.
• Notice that the statistician (Andrew) has carefully defined the race-religion categories to balance between collapsing subgroups that are distinct and showing too many subgroups so as to cloud the patterns.  That is why there are many more race-religion subgroups that are not shown.  The ones shown are of special interest.  Consider the white protestants, evangelical vs. non-evangelical (third and fourth rows).  If one were to fix the race, geography and income dimensions, and even fix half of the religion dimension, we still find the two subgroups to be on different ends of the spectrum relating to the voucher issue.  This is why the evangelical or not dimension has been included.
  
• The white space is interesting.  Here, the issue faced by the statistician is sparse data when one gets down to multi-dimensional subgroups.  Andrew chose to ignore all the data, which is the wise thing to do.  With so few samples, it is particularly easy to draw bad conclusions.   
• Because of the white space, we get additional information on the spatial distribution of the demographic subgroups.  The black population (at least the voters) are predominantly found in the southeast while Hispanics are in the southwest.  The subgroup of income higher than $150k is essentially all white.  Admittedly, this is a very crude read because we only have two levels (below 2% of state population and above).  Of the colored states, we cannot differentiate between densely populated and not.

  

Such rich graphics deserve careful reading.  Enjoy!
  

Sore-thumb graphics

A particular genre of graphics is designed to induce awe: certain bits are allowed to stick out like a sore thumb.  Via reader Andre L., and an archive of US Army medical photos and illustrations:

This is a small multiples graph designed to display the somewhat seasonal pattern of deaths due to influenza over years.  Basically, we see a U shape in almost every year; however, the height of the peak, and the timing of the peak shows quite a lot of variation.  Further, some years exhibit more of an L-shape than U-shape.

But the attention grabber here is the massive peak that occurred between 1918 and 1919.  It was unusual in many ways... it was the second big peak during 1918, it occurred late in the year and ellided with the next year's peak.  The designer allowed these two components to bleed into the other charts.

From the perspective of scale, readability, cleanliness, this bit sticks out like a sore thumb!  But one has to say it is effective.  

A log scale is often used to deal with data containing such outliers.  But while this makes neater charts, the impact of the orders-of-magnitude difference is lost on the reader, except in her imagination.

Exemplary

From Bernard L., another exemplary effort by the Times.  This one really got me excited.  

The set of line graphs shows how demographics of students in American schools have evolved in the last two decades.  Here, I selected New York City schools, and the tool sensibly decided to compare those with New York State schools (gray line).

There is so much to learn from one simple chart:

• The blue and gray lines are almost parallel everywhere, which tells us that in terms of the change in demographic composition, New York City pretty much resembled New York State during this entire period.
• However, in terms of demographic composition, rather than the change in composition, New York City schools are very different from the rest of the state, in that the proportion of white is lower by a third while that of minorities are much higher, especially black and Hispanic students.
• State-wide (as well as city-wide), black and white students have been declining as a proportion while Hispanics and Asians have increased. 
• The extent of the change is immediately visible, Asians have jumped from 7% to 14% for example.  

  
From a graph design perspective, the execution is very clean.  Data labels are limited to the first and last values.  A small multiples concept is used with the ethnic groups placed side by side.  A great awareness of foreground and background as well.  And imagine how much data has been visualized here, and be impressed.  You can look at any county in the country.

Here's one where the county change does not exactly mirror the state change (Napa in California):

Reference: "Diversity in the classroom", New York Times, March 12 2009.

Mini donuts

As a reader noted, this chart is essentially unreadable.  It contains data for the composition of diets in four countries during two time periods.

What might we want to learn from this data?

Are there major differences in diet between countries?

Within each country, are there changes in diet composition over the thirty years?

If there were changes in diet inside a country over time, did those reflect a worldwide trend or a trend specific to that country?

Unfortunately, the use of donut charts, albeit in small multiples, does not help the cause.  The added dimension of the size of the pies, used to display the total calories per person per day, serves little purpose.  Seriously, who out there is comparing the pie sizes rather than reading off the numbers in the donut holes if she wants to compare total calories?

This data set has much potential, and allows me to show, yet again, why I love "bumps charts".

Here is one take on it.  (Note that the closest data I found was for six different countries - China, Egypt, Mexico, South Africa, Philippines, India - and for different periods.)

The set of small multiples recognizes that the comparison between 1970 and 2000 is paramount to the exercise.  There is a wealth of trends that can be pulled out of these charts.  For example, the Chinese and Egyptians take in much more vegetables than the people of the other countries; in particular, the Chinese increased the consumption of vegetables drastically in those 30 years. (top row, second from left)

Or perhaps, for sugars and sweetners, consumption has increased everywhere except for South Africa.  In addition, the Chinese eat a lot less sugars than the other peoples. (top row, right)

Egg consumption also shows an interesting pattern.  In 1970, the countries had similar levels but by 2000, Mexicans and the Chinese have outpaced the other countries. (bottom row, right)

These charts are very versatile.  The example shown above is not yet ready for publication.  The designer must now decide what are the key messages, and then can use color judiciously to draw the reader's attention to the relevant parts.

Also, some may not like the default scaling of the vertical axes. That can be easily fixed.

Finally, here is another take which focuses on countries rather than food groups.  We note that too many categories of foods make it hard to separate them.

References: "Who's Eating What?", Wired, Oct 2008; "The Double burden of malnutrition", FAO, 2006.

Loss aversion

Loss aversion manifests itself in chart-making, as it does in economics.  In chart-marking, loss aversion can be defined as the tendency to avoid losing data at any cost.  Given a rich data set, designers often make the mistake of cramming as much data into the chart as possible.  This is taking Tufte's concept of maximizing data-ink ratio to the extreme, and it often leads to awkward, muddled charts.

Gelman provided a great example of this recently.  See here. 


Every piece of data is given equal footing, which results in nothing standing out.  The reader gasps for air.


Here is a recent example from the New York Times, in which the designer showed admirable restraint.

  The best evidence is the set of small multiples shown at the bottom.  These give the amount of phosphorus flowing into the lake annually since 1973, as measured from four locations.

The point is that the pollution has been most serious on the northern shores, especially in recent years.  Thus, the Florida plan focusing on the southern region is likely to make limited impact.

The choice of vertical lines is smart, as the typical time-series connected-line chart would jump up and down crazily.  A simple vertical axis marks the amounts, avoiding the temptation to print all the data.  The designer realizes it is the trend, rather than individual values, that is the issue.

Taken together, the three components tell a good story.  This is a well-executed effort.  The Times once again proves itself the leader in developing sophisticated graphics.



Reference: "Florida Deal for Everglades May Help Big Sugar", New York Times, Sep 13 2008.

Close races

Perhaps harkening to the close race between Obama and Clinton, the designer chose to illustrate this with what we have called the "racetrack" graph.  We have previously discussed the problems here and here.

In this rendition, a pie chart was divided into three race tracks with "cities" getting the inside track and "rural/small cities" getting the outside track.  (As the Clinton supporters might say, elitism was in the air.)  There were two great choices: the courage to not print the data and let the chart speak for itself, and the wisdom to white out the votes for "others".

Nevertheless, as we discussed before, the data is coded into the angles rather than the lengths of the strips, which presents a real problem in comparing vote shares.  For example, try figuring out if there were more Obama supporters in rural Tennessee than there were Clinton supporters in cities in Tennessee (bottom right).

Also note where the white "others" space were, and the impossibliity of comparing them.

The arrangement for Wisconsin, meanwhile, posed a challenge for anyone who wanted to estimate how many rural Wisconsin voters went for "others".

In the junkart version, we go with the two-sided bar chart, typically found in population pyramids.  The information presented jumps out at you.

This chart is essentially the same as the racetrack; one just needs to straighten out the strips from the original chart, and pull the Clinton ones clockwise, and Obama ones anti-clockwise.

Reference: some recent issue of New York Times magazine.

Whither complexity?

The ever interesting Gelman blog ("Too clever by half") ponders about this enterprising NYT chart.  Whatever its merits, this is one that requires close study. 

Reception is generally positive.  Andrew himself learnt an important fact, that there are still more white people than other races in America!  In statistics, we distinguish between two types of errors, the significant kind and the ignorable kind.  From this perspective, using admissions count is a gigantic problem; it renders the rest of the chart useless.  So I agree with Andrew.  As ever, picking the right scale is the beginning of making a nice chart.

We can also use this example to discuss the concept of "interactions".  When we go about presenting small multiples, i.e. comparisons of subgroups within a population, it's because we have observed differences between those subgroups; otherwise, it is both simpler and clearer to present the aggregate results.  The present chart presents subgroups defined by race, gender, age and substance abused, that is quite a lot of subgroups. 

Focusing on the first row (Alcohol), we note that the colored mass has shifted to the right, indicating more older people abused alcohol.  This trend appeared for all races.  Now scanning the other rows, we discover that only heroin abuse showed a distinctly different pattern, but only among whites.  For every other row, it seemed that the change from 1996 to 2005 was similar across races.

By breaking out substance abused, the designer added 21 little charts (7 sets of 3).   Only one set  (heroin) added information to what was true in aggregate i.e. that substance abusers got older.  The incremental gain in information does not justify the added complexity.

Nevertheless, the chart had many positive things such as judicious use of axis and gridlines and letting the graphical constructs speak for themselves (without accompanying data labels).

 

Reference: "Why is Mum in Rehab?",  New York Times, Jun 14 2008.

Progress and retrogress

Joran E. pointed to this "icky" chart he found on Clive Crooks' blog at the Atlantic. 


He ordered a "junkchart treatment", so here it comes.

First we wanted to process the triangles, dots and squares to make sense of this data.  We noted that the data came from a single year (2005) so the chart did not trace the development of the education sector over time.  But wait, it used a different route to get at the same idea.  The author compared different generations within each country to see if more and more citizens took university degrees.  So each vertical "arrow" was kind of a historical record of different generations within a country.  Under this criterion, Korea and Japan had come a long way while the US and China stagnated.

The chart is quite impossible to read as designed.  There is little reason to sort by 25-34-year-old proportion when the message concerns improvement over generations.  Besides, what about countries that apparently retrogressed?  (like Russia and Germany)

For this data, I returned to my favored bumps chart.  Here is version one.  There are two ways to read this chart: across countries, we note that most of the European states (blue) had similar profiles showing roughly a constant rate of growth.  The Asian duo of Japan and Korea (brown) had the most marked growth.  Of North America (black), Canada diverged from the US since the 35-44 generation.

Alternatively, we can focus on the change generation-over-generation.  From 55-64 to 45-54, almost all countries in this sample (except Japan) grew at the same rate.  Then between 45-54 and 35-44, the two Asian countries clearly set the pace.  The generation between 35-44 and 25-34 is most interesting: Korea has not slowed, Japan has slowed a little but still grew as fast as Canada.  A trio of European countries (Spain, Ireland, France) outpaced their neighbors.

Below I show version two.  This one combines bumps chart with small multiples.  North America, Europe and Asia/Australia are now in separate charts.  This removes clutter.

 

Small multiples re-imagineered

This chart gave me trouble.  I kept staring at it, staring.  Searching for the legend.  What could the several lines, in different colors, represent?  Take a look yourself.

Well, it turns out all three graphs were duplicates.  A different line was given dark blue to highlight a particular amusement park.

I have not seen this tactic used before.  This is like a small multiples concept except that every chart contains the same data.  Is it better than having just one chart?

Reference: "Will Disney Keep Us Amused?", New York Times, Feb 10 2008.

PS. [4/6/2008]  Here are two alternative charts contributed by our readers.  See comments below.

Derek suggested using sparklines:

Zuil reverted to basics: