Junk Charts

Worthy of the Times

Andrew Gelman has put up a great post, discussing how he collaborated with the New York Times editors to transform his chart to a publishable form.  Some time ago, there was a discussion here about the publish-worthiness of my charts (actually, the lack thereof), and I explained that to take my charts to that level would require quite a bit of extra work.  Gelman speaks here from first-hand experience.

Here are the two versions side-by-side.  Readers on the Gelman blog are debating which version is "better" with Gelman himself voting for the revised NYT version:

Let's compile a list of the changes:

1. Removed 2000 data and relied on only 2004 data
2. Reduced number of groups of senators from 7 to 5 (with a special calling out of Senator Lieberman)
3. Re-ordered senators to facilitate classifying into Democrats, Independent, Republicans
4. Added annotation explaining the grouping of senators; in particular, qualified why the 49 Democrats and 32 Republicans were set aside
   
5. Removed scale labels on the top of the chart, retaining only the bottom labels
6. Vastly increased the area devoted to text labels, which now covered half of the chart
7. Added white vertical gridlines
8. Instead of coloring the cross marks, colored the background of the chart; introduced the yellow color for Lieberman (which Gelman originally colored blue)
9. Removed reference to the national average but explained clearly in the legend that the percentages are relative to the national average.
   

Of these, 2,3,4 are definite improvements. 1 is acceptable since 2000 and 2004 data are quite highly correlated (note, however, Iowa, Virginia and Ohio.) 7 looks brilliantly here although we typically don't recommend adding gridlines and such. 5 is also acceptable because of 7 -- with the gridlines, one doesn't need to have two sets of labels. 8 can be debated: I'd like to see a version with large colored squares instead of cross marks in lieu of the background; the current look is not bad. 9 can also be mulled over: would it be so bad as to insert in parentheses 73% after the words "national average" in the legend?  I suspect some feel that this may cause confusion because readers now need to understand relative versus absolute values. 6 is the most debatable because the data-ink ratio has been vastly reduced. 

There are several good features of the original chart that they left unaltered, which should also be duly noted:

• Allowed the positive side of the scale to extend to 10% even though the largest data only reached about 6%. While this creates empty space, it helps readers judge the magnitude of negative numbers relative to positive ones.
• Subdued horizontal gridlines
• Retained the title of the chart (I actually prefer that they use the title to summarize the finding as opposed to the current one which takes a neutral stance.)
• Retained the horizontal scale labels at 5% apart

One final comment on something omitted from both charts: I'd prefer that they plot all the dots for the 49 + 32 senators, on the top and bottom lines respectively, instead of just plotting the group averages.  It would be really interesting to see what the spread is around the 0% point.

Life-enabling charts

In response to my call for positive examples, reader Merle H. sent in an example of how good charts can make our lives simpler and easier.

All of us have seen the following presentation of air travel data.

Not trying to pick on Travelocity - it's the same format whether you use Expedia or any of the airline sites.  For those customers who are looking to decide what dates to travel so as to minimize their air fare, this format is very cumbersome to use.

What about this fare chart at FuncTravel.com?

As you mouse along the line chart, the average fare for each day is visible.  Clicking on a particular day will fix the departure or return dates.

So much easier, isn't it?

A few caveats, though:

• Instead of just providing the historical averages, they should consider including information on variability, such as bars that indicate the middle 50% or 75% of prices.  Also, what about a sliding control for customers to decide which period of past history the averages should use?  More recent data may be more representative.
• This particular feature appeals to the price-sensitive, date-flexible customer segment.  Not everyone will pick itineraries based on those criteria.  There is an easy fix. If some controls are available for customers to indicate other preferences, e.g. exclude all British Airways flights, include only evening flights, etc., and the chart can update itself based on such selections, then the chart becomes a lot more flexible, and useful to many more customers.
• As with many automatically generated charts, the chosen labels on the vertical axis are laughable.  That should be relatively easy to fix, you'd think.
  

A great start.  I happen to notice that Travelocity has a beta feature that shows a similar chart.  A revolution in how travel sites present data to us is long overdue. 

Seth's Rules

(Via Gelman blog)

Prominent marketer Seth Godin came up with some sensible rules for making "graphs that work".  We pretty much agree with most of what he says here, unlike the last time he talked about charting.

One must recognize that he has a very specific type of chart in mind, the purpose of which is to facilitate business decisions.  And not surprisingly, he advocates simple, predictable story-telling.

His first rule: dispense with Excel and Powerpoint.  Agreed but to our dismay, there are not many alternatives out there that sit on corporate computers.  So we need a corollary: assume that Excel will unerringly pick the wrong option, whether it is the gridlines, axis labels, font sizes, colors, etc.  Spend the time to edit every single aspect of the chart!

His second rule: never show a chart for exploration or one that says nothing.  I used to call these charts that murmur but do not opine.  (See here, for example.)  This pretty much condemns the entire class of infographics as graphs that don't work.   This statement will surely drive some mad.  One of the challenges that face infographics is to bridge the gap between exploration and enlightenment, between research and insight.  As I said repeatedly, I value the immense amount of effort taken to impose structure and clarity on massive volumes of data -- but more is needed for these to jump out of the research lab.

In rules 3 and 4, Seth apparently makes a distinction between rules made to be followed and rules made to be broken.  In his view, time going left to right belongs to the former while not using circles belongs to the latter.  He gave a good example of why pictures of white teeth are preferred to pie charts, bravo.  I hope all those marketers are listening.

As readers know, I cannot agree with "don't connect unrelated events".  He's talking about using line charts only for continuous data.  This rule condemns the whole class of profile plots, including interaction charts in which statisticians routinely connect average values across discrete groupings.  The same rule has created the menace of grouped bar charts used almost exclusively to illustrate market research results (dozens to hundreds of pages of these for each study).  I'd file this under rules made to be broken!

What menace?

What menace?

What menace?

What menace?

Alright, I made my point.  If you don't work in market research, the mother lode of cross-tabs and grouped bars, consider yourself lucky.  If you do, will you start making line charts please?

Serving donuts

David Leonhardt's article on the graduation rates of public universities caught my attention for both graphical and statistical reasons.

David gave a partial review of a new book "Crossing The Finish Line", focusing on their conclusion that public universities must improve their 4-year graduation rates in order for education in the U.S. to achieve progress.  This conclusion was arrived at through statistical analysis of detailed longitudinal data (collected since 1999).

This chart is used to illustrate this conclusion.  We will come to the graphical offering later but first I want to fill in some details omitted from David's article by walking through how a statistician would look at this matter, what it means by "controlling for" something.

The question at hand is whether public universities, especially less selective ones, have "caused" students to lag behind in graduation rate.  A first-order analysis would immediately find that the overall graduation rate at less selective public universities to be lower, about 20% lower, than at more selective public universities.  

A doubter appears, and suggests that less selective schools are saddled with lower-ability students, and that would be the "cause" of lower graduation rates, as opposed to anything the schools actually do to students.  Not so fast, the statistician now disaggregates the data and look at the graduation rates within subgroups of students with comparable ability (in this instance, the researchers used GPA and SAT scores as indicators of ability).  This is known as "controlling for the ability level".  The data now shows that at every ability level, the same gap of about 20% exists: about 20% fewer students graduate at the less selective colleges than at the more selective ones.  This eliminates the mix of abilities as a viable "cause" of lower graduation rates.

The researchers now conclude that conditions of the schools (I think they blame the administrators) "caused" the lower graduation rates.  Note, however, that this does not preclude factors other than mix of abilities and school conditions from being the real "cause" of lower graduation rates.  But as far as this analysis goes, it sounds pretty convincing to me.

That is, if I ignore the fact that graduation rates are really artifacts of how much the administrators want to graduate students.  As the book review article pointed out, at the less selective colleges, they may want to reduce graduation rates in order to save money since juniors and seniors are more expensive to support due to smaller class sizes and so on.  On the other hand, the most selective colleges have an incentive to maintain a near-perfect graduation rates since the US News and other organizations typically use this metric in their rankings -- if you were the administrator, what would you do?  (You didn't hear it from here.)

Back to the chart, or shall we say the delivery of 16 donuts?

First, it fails the self-sufficiency principle.  If we remove the graphical bits, nothing much is lost from the chart.  Both are equally impenetrable.

A far better alternative is shown below, using a type of profile chart.

Finally, I must mention that in this particular case, there is no need to draw all four lines.  Since the finding of a 20% gap essentially holds for all subgroups, no information is lost by collapsing the subgroups and reporting the average line instead (with a note explaining that the same effect affected every subgroup).  

By the way, that is the difference between the statistical grapher - who is always looking to simplify the data - and the information grapher - who is aiming for fidelity. 

Reference: "Colleges are lagging in graduation rates", New York Times, Sept 9, 2009; "Book review: (Not) Crossing the Finish Line", Inside Higher Education, Sept 9 2009.

Round up

Here are some interesting reading from other places:

Tag clouds have caught on since we approved them a while ago.  One interesting use was at the Life Vicarious blog.  They use it to compare the inclinations of three New York-based restaurant reviewers.  What they should have done is to remove irrelevant words like  "one", "also", "many", "make"/"made", etc.  In statistics, this is called removing "noise" which helps bring out the "signal".

Andrew Gelman discussed the NYT article that reported the finding of unexpected male bias in the children of Asian American families.  He can be counted on to make useful comments on any accompanying graphics.  He rightly pointed out that this is one example of not starting at zero: the relevant baseline is 100 since the metric is essentially the over-age of males relative to females.  I also agree that a line chart with a longer time series plotting percentages rather than over-age would work better.

The racetrack chart made an appearance at Flowing Data.  This one is even more busy and just as impossible to decipher.

Spinning multi-color 2

Here are two more versions of the greenhouse gas chart.

The first one is a Marimekko which many would consider to be appropriate for this type of data.  It is essentially a stacked bar chart where the width of the bar is scaled to the proportion of the type of gas.  Here's what one would be looking at:

Merimekkos (also called Mosaic charts) share many of the problems of pie charts.  Note the need to use multi-color, the difficulty in comparing the areas of the pieces (even worse than looking at sectors), and the difficulty in comparing across categories since the pieces float in irregular space (take for example the three pink pieces).  My rule is: avoid at all costs. (Well, like the pie chart, when the data is sufficiently simple, with very few pieces and with some outliers, these could be acceptable.)


Secondly, here is a recycled junkart chart, with all white space removed from the interior.  (Thanks to Derek for the suggestion.)

Depending on what the purpose of the chart is, one can decide what is the base for the proportions.  My version preserves equity between the two dimensions.  Anything else will require the designer to make a choice.  If, for example, the base is 100% for each type of gas emitted, then the reader could not derive from the same chart the proportion of each source of emission (across all types of gases).

Sore-thumb graphics

A particular genre of graphics is designed to induce awe: certain bits are allowed to stick out like a sore thumb.  Via reader Andre L., and an archive of US Army medical photos and illustrations:

This is a small multiples graph designed to display the somewhat seasonal pattern of deaths due to influenza over years.  Basically, we see a U shape in almost every year; however, the height of the peak, and the timing of the peak shows quite a lot of variation.  Further, some years exhibit more of an L-shape than U-shape.

But the attention grabber here is the massive peak that occurred between 1918 and 1919.  It was unusual in many ways... it was the second big peak during 1918, it occurred late in the year and ellided with the next year's peak.  The designer allowed these two components to bleed into the other charts.

From the perspective of scale, readability, cleanliness, this bit sticks out like a sore thumb!  But one has to say it is effective.  

A log scale is often used to deal with data containing such outliers.  But while this makes neater charts, the impact of the orders-of-magnitude difference is lost on the reader, except in her imagination.

Bond yield makeover

Message to readers: I have a large backlog of reader suggestions.  Please be patient as I slowly get through them.  The frequency of posts will remain lower for the time being as I am busy finalizing a draft of a book.  More on that in the near future.

Matt H, a reader, sent in the following entry (with minor edits).

I saw a couple of bad charts on money.cnn.com and thought I'd submit them to you. 

They're both part of the same feature on investment bargains caused by the recession. 

Here are the links:  

Chart A: Municipal Bonds

Chart B: Corporate Bonds

It seems to me like both charts would have made their points more eloquently by using a much simpler, more common form, like a bar chart. 

In Chart A, cubes are used to display the difference between treasury bond yields and AAA municipal bond yields at the two-year horizon and the ten-year horizon.  The volume of each cube corresponds to the yield for the given type of bond in the given period (I think), which spreads the one dimension being compared (yields) across three dimensions, making the differences look smaller than they really are.

[...] At the two-year horizon, the two yields being compared are 1.16% for Treasury bonds and 3.01% for AAA municipal bonds.  The yield for AAA municipal bonds in this case is more than 2.5 times larger than the yield for Treasury bonds, but the difference doesn't look nearly that big in the chart provided.  [...] 

Time out.  Let me add that the inadvertent reference to an optical illusion concerning foreground and background!  The "outline only" cube on the left should have approximately the same volume as the "solid red" cube on the right (3.01% versus 3.30%)  and yet the red cube appeared quite a bit larger because our eyes reacted to the solid color more than thin outlines.

Matt continued:

In Chart B, [...] Again, the metric in question is bond yields:  ten-year Treasury bond yields compared to investment-grade corporate bond yields.  The 2008 figure for each is shown alongside the five-year average.  This chart uses the area of a circle to express these yields, spreading the one-dimensional value across two dimensions.  As in Chart A, the result is a chart in which the difference between values does not appear as large as it actually is.  

I will also send a simple bar chart version of each chart -- the bar charts should illustrate the differences in yields more effectively than the charts actually used in this article.

These are his revised charts:

We can do even better to convert the chart on the right to a time-series line chart.  Instead of the five-year average, it is better to display the gap beween treasury and corporate bonds for each of the five years plus 2008.  This should make for a more eye-catching graphic.

Reference: "Investment in the bargain bin", CNN Money.

The shackle of time 1

I ran across this hugely successful chart on Dean Foster's home page (and noted that he and his Wharton colleagues have a nice blog picking apart statistical errors committed in public.)

This is a histogram plotting the historical year-on-year returns of the S&P 500 index, binned into 10%-levels.  It succeeds on two levels: the innovation of printing the years inside little blocks provides extra information without distracting the overall picture; the key message of this plot, that the negative return of 2008 is a negative outlier in the history of returns, is extremely clear.

This, in my mind, is a superior presentation than the usual time-series line chart that we see in every economics publication.  For some purposes, it is better to unshackle ourselves from the linear time dimension, and this is a good example.

One question/comment: within each 10% level, the years are arranged in reverse chronological order fro top to bottom.  This facilitates searching for a particular year.  The obvious alternative is to order by the actual level of return, so that the result is akin to a stem-and-leaf plot.

While I like the graphical aspect of the chart, I feel like it has limited function.  This graph appears useful to anyone who has a one-year investment horizon.  If I want to predict what next year's S&P 500 return is, I might take a random sample from this distribution.  However, as a lazy investor, I never look at a one-year horizon so this creates two problems: if I am looking five years out, I can't take five samples from this distribution because there is serial correlation in this data for sure; even if I could take those five samples, it is difficult to compute the five-year return in my head.

So what I did was to take the data and replicate this histogram for 2-year, 3-year, 5-year, 10-year, etc. returns.  The results are as follows.  I decided to simplify further and use Tukey's boxplot instead of the histogram.  The data are real compounded total returns from S&P 500 from 1910-2008.

 The boxplot on the top right shows that there is about a 25% chance that an investment in the S&P 500 will return negative in real terms in any three-year period (below the green line).  At the other end, there is a 25% chance of getting earning more than 50% on the principal during those three years.

The next set of boxplots compared 5-year returns to 10-year returns and 10-year returns to 20-year returns.  If we have a 10-year horizon, there is still positive chance of reaching the end of the decade and finding the investment under water!  The median 10-year return is approximately doubling the principal (about 8% per annum compounded).  

In a twenty-year period, there is hardly any chance of not making money on the S&P.  There were two positive outliers of over 1000% (about 13% per annum compounded over 20 years).

Reference: Data from Global Financial Data

Seen at Starbucks

Merry Christmas!

If you happened to be in a Starbucks recently, you might have picked up some charts, which was what happened to one of our regular readers and commentators, ZBicyclist, who then tried his hand at chart critique here.  He is worried they may reach millions soon.  So look out!

This graph on the right -- which should rightfully be called a "tree ring graph" -- seems to me a fantastic concept although it is hard to think of data that would deserve this treatment.  Certainly not the retail sales series plotted here! 

This is an object lesson in why bubble charts often fail.

One issue is scale: note the awkward way in which the innermost ring is used to designate the oldest sales data of $375 billion presumably in 1996, and think about how you would decide where to place the 2007 ring.  (It's arbitrary.)

Another problem is labeling: when the growth is slow, the rings are close together, and labels have to be jittered (look at 2001 and 2002).  In this case, a relatively simple solution is to have the entire series of years run diagonally.

Yet another challenge is relative radii versus relative areas.  Inevitably, some readers will respond to the areas while others will respond to radii  ZBicyclist, for example, belongs to the first group while in this case, I find myself siding with the latter.  When the bubbles/rings overlap, it is difficult to assess areas.

Of course, a simple line chart would do the job with minimal fuss.  The following chart issued by the National Retail Federation actually plots the growth rates, rather than the annual sales.

(Please lose the grid-lines, or else add a vertical axis and lose the data labels.  A column or dot chart is also slightly preferred.)

Now go read ZBicyclist's point of view.  In the meantime, let us know if you think of ways to use the tree ring graph.