Junk Charts

A shocking failure to communicate

So said a reader, Stephen B., of the following graphic (note: pdf) in the London Times concerning Andy Murray's recent tennis triumphs.

How can we disagree?  Shocking?  Yes.  Failure?  Definitely.  Failing to communicate?  No doubt.

Let's first start with the five tennis balls at the bottom.  It fails the self-sufficiency test.  It makes no difference whether the balls (bubbles) are the same size, or different sizes.  Readers will look at the data and ignore the bubbles.

Amazingly, the caption said that "Murray has one of the best returns of serve in the game."  And yet, the graphic showed the five players who were better than Murray, and nobody worse!  For those unfamiliar with tennis statistics, it does not provide any helpful statistics like averages, medians, etc. to help us understand the data.

But that is only the beginning.

Take a look at these two donuts.

(The color scheme from light to dark: first, second, third, fourth round of tournament)

So we're told: the 75% of first-serve points won in the fourth round was 25.6% of the sum of the percentages of first-serve points won from first to fourth rounds (75%+70%+71%+76%).  What does this mean?  Why should we care?

The challenge with these two statistics is that they are correlated and have to be interpreted together.  If a first-serve is won, then there would be no second serve, etc.  Here's one attempt at it, using statistics from the Soderling-Federer match.  It's clear that Federer was better on both serves.

Reference: "Murray's march to the last eight", London Times.

 

The case of the shrinking mall

I had a similar reaction when reading this chart but I will let our reader take center stage.

Top portion

Bottom portion

The back of today's New York Times' Week in Review section devotes half of its space to a lame infographic that wastes space and has a major error (which has been corrected lazily in the version now online at http://www.nytimes.com/interactive/2008/10/14/opinion/20090531_OPCHART.html ) The chart shows the recent decline (or in some cases, rise) of retail sales at 27 common mall chains. My main objection is that the top half of the chart is useless. Yes, it provides a baseline from which shrinkage in area (visual metaphor of income = floor space) in the bottom half can represent the relative declines in sales, but this is redundantly handled better by color. The only part of the chart that I got any information from was the bottom half, and it took me a while before I figured out why the top half was even there. Meanwhile, the error in the printed version is that the +5-10% stores were colored light green while the +0-5% store (Burger King) was colored dark green. It should have been vice-versa. The online version simply swapped the colors in the legend, rather than on the map itself, which works logically but begs the question: why do you have dark red at one end of the spectrum and light green at the other, with dark green in the middle? Thank you for listening-- just blowing off a little steam here. It's a lot of wasted space and I'll bet the New York Times paid a lot for it.

Reference: "Op-Chart: The Fall of the Mall", New York Times. (Ed: I am not sure why the date is given as October 2008.)

Some things never fall

It's New York City.  House prices do not fall.  Enjoy the graphic!

Click here for the interactive version

Reference: "For Housing Crisis, the End Probably Isn't Near", New York Times, April 21 2009.

PS. Memo to grammar teachers: When did they start using contractions in titles? 

Don't mess with the scale

My friend Patrick pointed out the single biggest issue with the chart below -- that the designer chose a scale that precisely undermines the message of the chart.  Undermine may be too mild a word to use here; annihilate may be more apt.

The lines in this chart are anchored at the zero point on the time line (horizontal axis) used to indicate the bottoms of various bear markets in the Dow from 1929 to 2007.  From that anchor, time runs to the left showing the amount of time for the Dow to go from peak to bottom (the decline); time runs to the right showing the amount of time for the Dow to climb back to the prior peak (the recovery).  As the caption said, the point of the chart is "if the decline was fast, the recovery took a considerable time".

Funny thing then that the distances from the zero point are roughly comparable on the left as on the right.

This illusion resulted from some very convoluted and perplexing messing around with the horizontal scale.  First, the left-of-center scale is in months while the right-of-center is in years.  Second, the left-of-center scale has normal spacing while the right-of-center seemingly was suffering from spasms.  Take a closer look:

The first five years (0-5) took up about half the scale while the next five (5-10) took maybe one-eighth.  The first year (0-1) took about as much space as the next two years (1-3).

I am not quite sure what is the logic behind this but since the message of the chart has everything to do with the time duration, it is most unfortunate to introduce such distortions.

There is yet another "innovation" in this chart.  Notice that on the right side, the axis labels are irregular (more spasms)... 0,1,3,4,5,10,15,20, 25...  This is as if the designer is posing one of those IQ questions requiring readers to figure out the next number in the sequence.  The specific time intervals selected may have meaning: note that all the lines are straightened out in between these tick marks.  Given that each line represents a different historical sequence, it is difficult to comprehend the regularity of these intervals across history.  Perhaps this will prove to be the key to unlocking the secret of this chart.  Please comment below if you are able to unravel this mystery.

Besides, the same type of "innovation" was not applied to the left side of the chart.  Here, the designer opted to throw out all the data between the peak and the bottom and straightened out all the intermediate fluctuations.

Below are two different versions of this chart, basically restoring the time scale to the normal, equally spaced, symmetric appearance.  The top one used monthly Dow returns where the volatility obstructed our understanding of the trends, requiring the use of color to differentiate the lines.  In the next version, I used R to generate the loess estimates (a type of smoothing) and the trends became clearer.  (There was a prior discussion of loess on Junk Charts here.)

 

Now, these pictures are very different from the original graph!

I'd be very cautious about reading into these charts anyway.  This question is not one suitable for statistical analysis.  The sample size of six is far too small.  Each recession is different in terms of causes, remedies and context.  The fact that we call them recessions do not make them comparable.  Further, it is also impossible to know at this stage if the 2007 decline has reached bottom.  The chart designer essentially assumed this to be the case but who knows?

PS. Nick Rapp, one of the designers of the chart, responds in the comments.  He has started a blog to feature the work of his graphics team at AP.  His colleague has created an interactive version.  More than anything, this post highlights an aspect of the chart that Nick and his team clearly spent a lot of time doodling over.  The concept of the chart itself is wonderful actually, if I didn't say so already; it is essentially the same chart as the oft-printed chart where the anchor point is the start of each recession, only here the anchor is the bottom of each recession.

Exemplary

From Bernard L., another exemplary effort by the Times.  This one really got me excited.  

The set of line graphs shows how demographics of students in American schools have evolved in the last two decades.  Here, I selected New York City schools, and the tool sensibly decided to compare those with New York State schools (gray line).

There is so much to learn from one simple chart:

• The blue and gray lines are almost parallel everywhere, which tells us that in terms of the change in demographic composition, New York City pretty much resembled New York State during this entire period.
• However, in terms of demographic composition, rather than the change in composition, New York City schools are very different from the rest of the state, in that the proportion of white is lower by a third while that of minorities are much higher, especially black and Hispanic students.
• State-wide (as well as city-wide), black and white students have been declining as a proportion while Hispanics and Asians have increased. 
• The extent of the change is immediately visible, Asians have jumped from 7% to 14% for example.  

  
From a graph design perspective, the execution is very clean.  Data labels are limited to the first and last values.  A small multiples concept is used with the ethnic groups placed side by side.  A great awareness of foreground and background as well.  And imagine how much data has been visualized here, and be impressed.  You can look at any county in the country.

Here's one where the county change does not exactly mirror the state change (Napa in California):

Reference: "Diversity in the classroom", New York Times, March 12 2009.

Popping the bubble, so to speak

Matt H., who authored the previous post, and Charles G. both pointed to a great example of how people like readers here can make a difference.   A bad chart got made over!

The Financial Times published a chart from a JP Morgan report, using ... you guessed it ... bubbles to illustrate the deep plunge in market capitalization of many banks.

Some readers at the blog were none too happy with the choice of bubble charts.   Among other things, the designer made the common mistake of plotting proportional diameters rather than proportional areas.  This is clear from looking at JP Morgan's bubble.

This chart exposes the weakness of bubble charts well.  Look at the top row of bubbles.  Most of them look so similar it is impossible to know, without spending much time studying the circles, which bank was hurt more.

Eventually, a bar chart was produced.  Felix Salmon linked to it.  I am not sure how the banks were ordered in this chart.  It isn't one of the two obvious dimensions, nor is it in alphabetical order.

In fact, neither the bubble nor the bar chart works well for this case.  What we need is the Case-Schiller style asset-bubble life cycle chart.  In order to interpret these changes in market caps, we need to know how big was the bubble, and then how steep was the consequent decline.  Take a look at our discussion of real estate bubble charts here.

Reference: "Bank capitalization chart of the day", Felix Salmon at Portfolio.com and "Bank picture du jour", FT Alphaville, Jan 21 2008.

A Harvard mess 1

There are some innocent-looking charts that throw up more and more problems, the more you look at it.  This example comes from a magazine sent to Harvard alumni.  We have all heard that their endowment fund suffered some horrific losses in the last few months and so the magazine editor thought it useful to describe the potential impact on the different departments.

It's a safe bet our readers would not think to present two related data series as a combination of one bar chart and one column chart. 

As the chart stands, the intended message is completely lost.  It takes a bit of fishing to learn that the Radcliffe Institute has a tiny stake in the endowment fund but supports over 80% of its operating funds from the endowment.

Now looking a little deeper, we find that the scales of the two charts are not standardized so the length of a bar and the length of a column cannot be directly compared.  Nor can the grid-lines as each section accounts for 10% in the bar chart but 20% in the column chart (to make it worse, the larger section represents the smaller percentage!)

Looking further still, we find that "Other" accounts for some 15% of the endowment but apparently consists of entities that do not have operating funds and thus goes missing in the column chart.  In our versions below, we will ignore the "Other" category completely; this is equivalent to assuming that we have allocated to the individual schools a proportional share of "Other"'s share.

Not to mention arranging the schools by alphabetical order.

Much of this mess became apparent when we put the two charts into a uniform setting, like this:

 

A scatter plot provides good information, especially if there is clustering  although we can debate whether it is fit for general publication.

More in our next post.

Reference: "The Endowment: Each school's stake", Harvard Magazine, Jan-Feb 2009.

PS.  The initial post switched the axis labels on the two bar charts. Thanks to Jon for pointing this out.

A joke

For light entertainment, see this chart sent in by Chris P.  Original here.

Chris said: "his graph breaks a few rules, but it has a clear message".

The shocking, out of the box column certainly grabs attention, and it is probably true that football coaches earn too much money.  But the chart really falls down on this one issue:

What's the median salary of these football coaches?

Reference: "Academic salaries", PHD Comics.

The matter of bad choice

Right on the heels of the disastrous bubble chart comes another, courtesy of the NYT Magazine.  Bubble charts are okay for the conceptual ("this is really big, and that is really tiny").  This chart wants readers to compare the sizes of the bubbles, which highlights the worst part of such graphs.

Poor scaling is the huge issue with bubble charts.  They are the prototype of what I call not "self-sufficient" charts.  Without printing all the data, the chart is unscaled, and thus useless (see below middle).  When all the data is printed (as in the original, below left), it is no better than a data table.

In the above right chart, we simulated the situation of a bar or column chart, i.e. we provide a scale.  For this chart, the convenient "tick marks" are at 10, 20, 34, 41.  Unfortunately, this scaled version also fails to amuse.

Note further that the data should have been presented in two sections: the party affiliation analysis and the gender analysis.  Also, it is customary to place "Independents" between "Republicans" and "Democrats" because they are middle-of-the-road.

A profile chart is an attractive way to show this data.  Here, we quickly learn a couple of things obscured in the bubble chart.

On the issue of abortion, Independents are much closer to Democrats than Republicans.  Also, there is barely any difference between the genders, the only difference being the strength of support among those who want to legalize.

Reference: "A matter of Choice", New York Times Magazine, Oct 19 2008.

PS. Based on RichmondTom's suggestion, here are the cumulative profile charts.

 

Bernard L. suggested a "tornado" chart:

Mind the gap

When comparing two time series, one typically wants to discuss the size of the gap as it changes over time.  This Business Week chart, for example, depicted for readers the expanding gap between intra-day high and low prices of the S&P 500 for 2008.


This chart construct is effective at pointing out large changes but lacks precision in conveying smaller differences, or trends.  It is always a good idea to plot the gap directly, as we will show below.

More importantly, a better choice of scale can help a lot.  By focusing exclusively on variability (extreme values), this chart hides the relevant information of the closing prices of the S&P.  A point spread of a 100 points means more when the index is at 800 than at 1200.  In order to capture this, we can divide the point spread by the opening price of that day so we say the gap is one-eighth or one-twelfth of the opening price. 

The junkart version makes both changes.  The top chart fixes the scale, plotting the point spread as a percentage of daily opening prices.  Relative to the original chart, the variability in the front part of 2008 was muted because the index was at higher levels back then. 

The bottom chart plots the gap sizes (lengths of the high-low lines).  It is without doubt that directly plotting the gaps showcases the key message.  The current level of volatility is more than double what occurred at the beginning of the year.

If one wants to illuminate the trend as opposed to daily fluctuations, a further improvement will be using moving averages.

For those interested, shown below is a scatter plot that compares the original point spread and the derived point spread, which shows that the change is not trivial.

 

Reference: "The Market: A Daily Roller Coaster", Business Week, Oct 27 2008.