Junk Charts

The hard work of entertaining

Stefan pointed us to his work for the UN GEO (United Nations Global Environment Outlook) data portal.  This set of information posters highlights a vexing issue that crops up on Junk Charts from time to time, that is, the proper balance between information and entertainment value of data displays.  While this blog concerns itself primarily with the former, it does not mean that we are blind to the flashier side of the enterprise.

Let's take Stefan's recycling spiral chart as an example.  One must admit that visually this presentation is more appealing than either a data table or a set of bar charts.  The reader can obtain the primary piece of information, which is the ranking of different countries in terms of the proportion of collected waste that is recycled. 

And if the reader is curious enough, the chart also provides the data on the per-capita amount of waste collected in each of these countries.  (Like the table and bar chart, this display also has the problem that it is one-dimensional, thus the countries can be sorted by proportion of recycling but then the waste collected data will be out of order.)

For those readers who would like to understand the data better, they would want to know some of the following:

• Is there a relationship between amount of waste collected and amount of waste recycled?
• Are there differences in culture resulting in different recycling rates?
• Is the level of development of a country predictive of its recycling rate?
• Why are some countries recycling more of its waste, and others less?

To address these types of questions, one can start with the following scatter plot.

 

With the exception of South Korea, there is a general pattern of positive correlation: the more waste collected per capita, the larger proportion of such waste recycled.  Any dots that are not in the bottom left or top right quadrant are exceptions to the rule.  These countries are labeled in red or blue, the former indicating that the amount of collection is above average while the rate of recycling is below average. 

Because there is sampling error, dots that are close to the average dot (the center of this scatter plot) are probably just average.  Roughly speaking, dots in the gray circle are close enough to the center that I would not consider them exceptional cases.  That leaves Spain and Iceland in the red corner, and South Korea in the blue corner.  If both data series are considered together, these three countries should merit attention; if only the proportion of recycling is considered, then one would pay attention to Italy, Turkey and Slovak Republic on the lower end and South Korea on the high end.

Scatter plots are very versatile.  The following one explores the issue of development level.  Surprisingly, the level of recycling seems to have little to do with development; the countries are quite widely scattered.

 

Technical note: The data on both axes are expressed in "standardized" units.  So the zeroes represent the average per-capita waste collected, and the average proportion of waste recycled (only of those countries depicted in the original chart).  +1 indicates an amount that is one standard deviation above the average.  Think of "standardized units" as measuring how extreme is a particular country with respect to the average. 

Seth's Rules

(Via Gelman blog)

Prominent marketer Seth Godin came up with some sensible rules for making "graphs that work".  We pretty much agree with most of what he says here, unlike the last time he talked about charting.

One must recognize that he has a very specific type of chart in mind, the purpose of which is to facilitate business decisions.  And not surprisingly, he advocates simple, predictable story-telling.

His first rule: dispense with Excel and Powerpoint.  Agreed but to our dismay, there are not many alternatives out there that sit on corporate computers.  So we need a corollary: assume that Excel will unerringly pick the wrong option, whether it is the gridlines, axis labels, font sizes, colors, etc.  Spend the time to edit every single aspect of the chart!

His second rule: never show a chart for exploration or one that says nothing.  I used to call these charts that murmur but do not opine.  (See here, for example.)  This pretty much condemns the entire class of infographics as graphs that don't work.   This statement will surely drive some mad.  One of the challenges that face infographics is to bridge the gap between exploration and enlightenment, between research and insight.  As I said repeatedly, I value the immense amount of effort taken to impose structure and clarity on massive volumes of data -- but more is needed for these to jump out of the research lab.

In rules 3 and 4, Seth apparently makes a distinction between rules made to be followed and rules made to be broken.  In his view, time going left to right belongs to the former while not using circles belongs to the latter.  He gave a good example of why pictures of white teeth are preferred to pie charts, bravo.  I hope all those marketers are listening.

As readers know, I cannot agree with "don't connect unrelated events".  He's talking about using line charts only for continuous data.  This rule condemns the whole class of profile plots, including interaction charts in which statisticians routinely connect average values across discrete groupings.  The same rule has created the menace of grouped bar charts used almost exclusively to illustrate market research results (dozens to hundreds of pages of these for each study).  I'd file this under rules made to be broken!

What menace?

What menace?

What menace?

What menace?

Alright, I made my point.  If you don't work in market research, the mother lode of cross-tabs and grouped bars, consider yourself lucky.  If you do, will you start making line charts please?

Serving donuts

David Leonhardt's article on the graduation rates of public universities caught my attention for both graphical and statistical reasons.

David gave a partial review of a new book "Crossing The Finish Line", focusing on their conclusion that public universities must improve their 4-year graduation rates in order for education in the U.S. to achieve progress.  This conclusion was arrived at through statistical analysis of detailed longitudinal data (collected since 1999).

This chart is used to illustrate this conclusion.  We will come to the graphical offering later but first I want to fill in some details omitted from David's article by walking through how a statistician would look at this matter, what it means by "controlling for" something.

The question at hand is whether public universities, especially less selective ones, have "caused" students to lag behind in graduation rate.  A first-order analysis would immediately find that the overall graduation rate at less selective public universities to be lower, about 20% lower, than at more selective public universities.  

A doubter appears, and suggests that less selective schools are saddled with lower-ability students, and that would be the "cause" of lower graduation rates, as opposed to anything the schools actually do to students.  Not so fast, the statistician now disaggregates the data and look at the graduation rates within subgroups of students with comparable ability (in this instance, the researchers used GPA and SAT scores as indicators of ability).  This is known as "controlling for the ability level".  The data now shows that at every ability level, the same gap of about 20% exists: about 20% fewer students graduate at the less selective colleges than at the more selective ones.  This eliminates the mix of abilities as a viable "cause" of lower graduation rates.

The researchers now conclude that conditions of the schools (I think they blame the administrators) "caused" the lower graduation rates.  Note, however, that this does not preclude factors other than mix of abilities and school conditions from being the real "cause" of lower graduation rates.  But as far as this analysis goes, it sounds pretty convincing to me.

That is, if I ignore the fact that graduation rates are really artifacts of how much the administrators want to graduate students.  As the book review article pointed out, at the less selective colleges, they may want to reduce graduation rates in order to save money since juniors and seniors are more expensive to support due to smaller class sizes and so on.  On the other hand, the most selective colleges have an incentive to maintain a near-perfect graduation rates since the US News and other organizations typically use this metric in their rankings -- if you were the administrator, what would you do?  (You didn't hear it from here.)

Back to the chart, or shall we say the delivery of 16 donuts?

First, it fails the self-sufficiency principle.  If we remove the graphical bits, nothing much is lost from the chart.  Both are equally impenetrable.

A far better alternative is shown below, using a type of profile chart.

Finally, I must mention that in this particular case, there is no need to draw all four lines.  Since the finding of a 20% gap essentially holds for all subgroups, no information is lost by collapsing the subgroups and reporting the average line instead (with a note explaining that the same effect affected every subgroup).  

By the way, that is the difference between the statistical grapher - who is always looking to simplify the data - and the information grapher - who is aiming for fidelity. 

Reference: "Colleges are lagging in graduation rates", New York Times, Sept 9, 2009; "Book review: (Not) Crossing the Finish Line", Inside Higher Education, Sept 9 2009.

Extra reading

The New York Fed has put up some impressive infographics, depicting the credit conditions around the country.

(I was going to praise the producers for this effort - a rarity among government sites but when I went back to the site today, I find that the graphic is loading only occasionally, and try as I might, I could not get rid of the big brown patch that obscures the West coast.  Hope you have better luck!)

As with similar maps, the unresolved problem is that the pattern is strongly dominated by the relative density of people around the country.  As we click through different maps, they all look similar.

But this map -- as do most infographics -- does well to put structure onto unwieldy, large data sets.  It acts as a table of contents to the data set.

Nick Rapp's team at AP produced this great chart on US life expectancy:

Very sensitive axis labels, putting the line labels into the chart (rather than in a legend box far away), showing only the current data, not every data point, subtle coloring to bring out the decades, etc.

I'm not sure about varying the thickness and tinge of the lines.  Is it necessary?  It hinders a bit as we try to compare the slopes of the lines.  Adding the initial data labels (for 1975) would also help us judge the net change over the period depicted.  If it is the change in life expectancy over time that is the main story, consider indicing all lines to the respective 1975 value.

Reference: "CDC says life expectancy in the US is up, deaths not", Miami Herald, Aug 19 2009. 

PS. This is really an awful title for the article.  It reads as if something bad has happened to the death rate while something good happened to the life expectancy, as if there is a paradox but the article makes clear that the death rate declined, which would make the point redundant.

Degrees of likeness 2

We left off the other day with an interactive graphic with the ability to peer into subgroups.  This feature assumes implicitly that the overall average obscures differences within subgroups.  What statisticians do with this type of data is to compare the subgroups, and identify the factors that make someone different from the average.

For example, there is a clear distinction between the employed and the unemployed in how they spend the day (not surprising).

This happens to be what NYT printed in the paper edition that day.  (Note, though, that the graphic loses quite a bit without the interactivity.)

On the other hand, there appears to be little differentiation between men and women.

Nor is there much difference between blacks and whites.

One factor that matters is age.  Older people are not exactly like the young.  A lot of these factors (for example, age and employment status) are correlated, by the way.

I showed all these in order to talk about the statistical concept of "aggregation".  We noted that the distribution of time use of the employed is different from that of the unemployed.  Thus, we cannot use the "average" distribution to describe both groups, and so we show the data in disaggregated form.  Similarly for time use and age.  

But there is not much gain in disaggregating race and gender: the "average" is representative of the subgroups for these two factors.  This is one distinction I see between information graphics and statistical graphics: the former typically shows all possible subgroups while in the latter, the designer zooms in on the factors that matter.

Degrees of likeness 1

The NYT team just put up a fantastic visualization of the American Time Use Survey data, which purports to measure how the average American spends the time of a day.  (Apparently, thousands of people recalled what they did every minute of an average day.)  The amount of data collected is massive, and this graphic allows readers to explore the data in intuitive ways.

The chart shows for each minute of the day (horizontal axis) the proportion of people doing specific activities.  Not surprisingly, we spend more time sleeping than any other type of activity.  The axis and data labeling as well as gridlines are very restrained.

Normally, I am not a big fan of these proportional area charts because the only relevant dimension to look at is the vertical distance from one curve to the next but the focus on areas put equal weight on the horizontal and vertical distances.   The horizontal distance is meaningless, and thus the area is meaningless.

These designers found a solution to the problem, and good for them!  Because of the mouse-over effect, I could not save the actual appearance -- here, I show what it looks like.

By mousing over different parts of the graph (say, moving vertically), we can compare the actual proportions.  Terrific!

The key interest of this graphic is the following legend.

While the above graphic shows the use of time by all Americans in aggregate, this panel allows us to zoom in on specific groups of Americans.  How alike are Americans?

Reference: "How Different Groups Spend Their Day", New York Times, July 31 2009.

 

Loss aversion

Loss aversion manifests itself in chart-making, as it does in economics.  In chart-marking, loss aversion can be defined as the tendency to avoid losing data at any cost.  Given a rich data set, designers often make the mistake of cramming as much data into the chart as possible.  This is taking Tufte's concept of maximizing data-ink ratio to the extreme, and it often leads to awkward, muddled charts.

Gelman provided a great example of this recently.  See here. 


Every piece of data is given equal footing, which results in nothing standing out.  The reader gasps for air.


Here is a recent example from the New York Times, in which the designer showed admirable restraint.

  The best evidence is the set of small multiples shown at the bottom.  These give the amount of phosphorus flowing into the lake annually since 1973, as measured from four locations.

The point is that the pollution has been most serious on the northern shores, especially in recent years.  Thus, the Florida plan focusing on the southern region is likely to make limited impact.

The choice of vertical lines is smart, as the typical time-series connected-line chart would jump up and down crazily.  A simple vertical axis marks the amounts, avoiding the temptation to print all the data.  The designer realizes it is the trend, rather than individual values, that is the issue.

Taken together, the three components tell a good story.  This is a well-executed effort.  The Times once again proves itself the leader in developing sophisticated graphics.



Reference: "Florida Deal for Everglades May Help Big Sugar", New York Times, Sep 13 2008.

As simple as possible but no simpler

In this political season, we are bombarded with soundbites.  For example, we keep hearing about the Red States versus the Blue States.   By coloring states, we endorse the notion that Red State people are conservatives and Blue State people are liberals.  Using sophisticated analyses of real data, Prof. Gelman tells us why this notion is wrong in his recently published book "Red State, Blue State, Rich State, Poor State".

The key chart is below (courtesy of Gelman). 

There is indeed a red-blue divide but the gap is much much wider among rich voters (right ends of lines) and the middle class (mid-points of lines) than among poor voters (left ends).  Poor voters are almost everywhere liberals on economic issues, and on social issues, they are moderates leaning conservative.  Rich voters, by contrast, are very polarized on both economic and social fronts.

This is one of those charts that express their messages without fuss.  A lot of data was analyzed but only the statistical conclusion is portrayed, much of the hard work hidden.  Contrast this with data-rich infographics.  In my view, this chart fits the description: as simple as possible but no simpler.

It is not that the common notion of a Red-Blue divide is entirely wrong; it suggests that such a view is too simple.  This divide is strongly present among middle-class and rich voters but not so much poor voters.  To be even more nuanced, for middle-class voters, the Red-Blue divide is manifested almost exclusively along the social dimension; middle-class voters are economic moderates everywhere, leaning liberal.

More technically, the above is summarized by saying that the interaction effect (between state residence and wealth) is significant and cannot be ignored.  Prof. Gelman is one of the strongest advocates of always including interaction terms in regression of social and economic data.  And here is an example of why.

The concept of interaction is tough to explain to the business audience, I have found.  In presenting one such chart recently, I found the audience confused by the lines.  Indeed, the lines, their slopes, etc. do not contain any information.  They merely serve as guides to how to read the chart.  In order to see the interactions, our eyes need to trace a path from one dot to another dot, literally tracing the lines on the chart.

Another way to discuss interactions: the common notion of a Red-Blue divide masks the reality that this divide is of varying importance across income groups.  A view that aggregates income groups is too simplistic.

Reference: Red State, Blue State, Rich State, Poor State, by Andrew Gelman (2008).

A graphical playground

RichmondTom pointed us to this New York Times article, which made what I consider a sloppy argument for the inflation rate being over-stated rather than understated.  This article was accompanied by a colorful and playful chart to describe which components make up the index, and which price changes were most drastic.

 

Like most "infographics", this chart has an interactive feature which you need to click here to sample: mousing over the strangely-shaped areas leads to annotations describing the weight of the relevant component, and the change experienced in that component during the past year.  Like Sherlock Holmes holding a magnifying glass, the reader can use the Zoom In, Zoom Out buttons to drill down.

Here is an expanded view on the color scale, with my comments.

We have no issues with the blue or the red & orange regions but what with using light blue and light green for increases between 0 and 4 percent (when all other increases use red)?  Readers not looking at the color palette will be caught off guard, misinterpreting the data.  This is a big problem since those two appear to be the most frequently occurring colors on the chart.

The designer of this chart surely is aiming for entertainment, creating a playground for the data sleuth.  It succeeds if readers take up the invitation to explore the data.  Unfortunately, it is not the most accessible for more sophisticated numerates interested in understanding the data.

It is impossible to ascertain the relative size of pieces that are close in size but often in arbitrary shapes.  We can roughly read out which components are most important to the index: turns out to be "owner's equivalent rent" by a mile.  Owner's equivalent rent, as discussed by Leonhardt, is the most problematic and inaccurate component, and also, I might add, most susceptible to manipulation since it is an estimate, not a measure.  The next biggest pieces -- food and gasoline (excepting rent) -- are taken out because of "volatility".

The most interesting information in this data set is missing: how much does each component contribute to the overall inflation rate?  This requires multiplying the weight of each component with the change of each component.

It would also be helpful to label which components are taken out of the Bernanke calculation that claims only a 4% inflation when my cable company seems to have raised prices every six months, and groceries and meals and so on are rising at 10-20%.  (Of course, Leonhardt thinks this is all loss aversion.)

Reference: "Seeing Inflation Only in the Prices That Go Up", New York Times, May 7 2008.

Graphs as catalogs

Junk Charts typically concerns itself with statistical graphics a la Tufte and Cleveland, treating charts as a means to summarize, elucidate and highlight aspects of data.  We haven't been too kind on so-called infographics, often finding these cluttered and confusing.  Recently, I have a small change of heart.

I now see infographics as innovative in one way, and a complement to traditional graphics.  This is the idea of graphs as catalogs.  What many of these graphs try to do is to present a structured way for users to explore massive amounts of data.  They don't serve the traditional purpose of summarization and that's why they are innovations.

The following chart from NYT tracing Serena Williams' tennis ranking prompted this post.

As a traditional statistical graphic, this chart leaves much to be desired.  The general outline of her career could be described in one sentence without the need for any graphic.  The colorful vertical lines serve little purpose, nor the short line segments on the other side of the axis.

However, as a catalog of data on Serena's career, this graphic is fascinating.  Mousing on the vertical lines changes the information on the top right corner, including the tournament being played and the media event she participated in, as well as photos and her rankings.  Similarly, the left and right arrows on the top left allow readers to browse through the list of events chronologically. (You need to click on the link to use the interactive features.)  Without this chart, it would have been very difficult to learn about Serena's record at a particular tournament or point in time.  It acts like a data table but presents the information in a much more accessible way.

Thus, relying on interactivity, this compact graphic enables any of us to browse to a user-defined depth a reservoir of data.  Bravo!

Reference: "Serena William's Professional Career", New York Times, June 2008.