Junk Charts

Worthy of the Times

Andrew Gelman has put up a great post, discussing how he collaborated with the New York Times editors to transform his chart to a publishable form.  Some time ago, there was a discussion here about the publish-worthiness of my charts (actually, the lack thereof), and I explained that to take my charts to that level would require quite a bit of extra work.  Gelman speaks here from first-hand experience.

Here are the two versions side-by-side.  Readers on the Gelman blog are debating which version is "better" with Gelman himself voting for the revised NYT version:

Let's compile a list of the changes:

1. Removed 2000 data and relied on only 2004 data
2. Reduced number of groups of senators from 7 to 5 (with a special calling out of Senator Lieberman)
3. Re-ordered senators to facilitate classifying into Democrats, Independent, Republicans
4. Added annotation explaining the grouping of senators; in particular, qualified why the 49 Democrats and 32 Republicans were set aside
   
5. Removed scale labels on the top of the chart, retaining only the bottom labels
6. Vastly increased the area devoted to text labels, which now covered half of the chart
7. Added white vertical gridlines
8. Instead of coloring the cross marks, colored the background of the chart; introduced the yellow color for Lieberman (which Gelman originally colored blue)
9. Removed reference to the national average but explained clearly in the legend that the percentages are relative to the national average.
   

Of these, 2,3,4 are definite improvements. 1 is acceptable since 2000 and 2004 data are quite highly correlated (note, however, Iowa, Virginia and Ohio.) 7 looks brilliantly here although we typically don't recommend adding gridlines and such. 5 is also acceptable because of 7 -- with the gridlines, one doesn't need to have two sets of labels. 8 can be debated: I'd like to see a version with large colored squares instead of cross marks in lieu of the background; the current look is not bad. 9 can also be mulled over: would it be so bad as to insert in parentheses 73% after the words "national average" in the legend?  I suspect some feel that this may cause confusion because readers now need to understand relative versus absolute values. 6 is the most debatable because the data-ink ratio has been vastly reduced. 

There are several good features of the original chart that they left unaltered, which should also be duly noted:

• Allowed the positive side of the scale to extend to 10% even though the largest data only reached about 6%. While this creates empty space, it helps readers judge the magnitude of negative numbers relative to positive ones.
• Subdued horizontal gridlines
• Retained the title of the chart (I actually prefer that they use the title to summarize the finding as opposed to the current one which takes a neutral stance.)
• Retained the horizontal scale labels at 5% apart

One final comment on something omitted from both charts: I'd prefer that they plot all the dots for the 49 + 32 senators, on the top and bottom lines respectively, instead of just plotting the group averages.  It would be really interesting to see what the spread is around the 0% point.

Reading comprehension

Note: I am in the middle of a holiday and so posting will be limited.

Andrew posted a pretty chart that caught my attention.  This is the sort of sophisticated chart that rewards careful reading. 

Below is a guide to reading the chart:

• It is a small multiples chart with the components arranged in two dimensions (income levels, and a race-religion hybrid category).  The top row is a summary of voters of all race-religion grouped by income.  Note that there is no corresponding summary column for voters of all incomes grouped by race-religion.
• Source of data: 2000 poll but applied to 2008 demographic patterns.  In other words, there is an underlying assumption that opinions have stayed stable within the demographic groups.
  
• The chart is in fact three dimensional because each map gives us the geographical (state by state) breakdown.
• It is useful to figure out the smallest unit of data: in this case, this is the percentage support of federal school vouchers by voters of a given race-religion-income-and-state category.
• The color scheme is such that red represents highest support and blue lowest support, with pink and purple in the middle
• It's almost always better to start from the aggregate (that is, the average) and then study variations along different dimensions, and this is how the chart is arranged from top to bottom
  
• On the top row, the higher income groups tended to favor vouchers more than lower income groups, with a break point around $75k; even here, the regional differences are significant, with northeast and southwest hotter for vouchers at all income levels
• As we move from row to row, we realize that the aggregate data hides many disparities.  For example, white Catholics (second row) are more likely to support vouchers regardless of income level while white non-evangelical Protestants (fourth row) are much less likely than average to support vouchers at all income levels.
• Notice that the statistician (Andrew) has carefully defined the race-religion categories to balance between collapsing subgroups that are distinct and showing too many subgroups so as to cloud the patterns.  That is why there are many more race-religion subgroups that are not shown.  The ones shown are of special interest.  Consider the white protestants, evangelical vs. non-evangelical (third and fourth rows).  If one were to fix the race, geography and income dimensions, and even fix half of the religion dimension, we still find the two subgroups to be on different ends of the spectrum relating to the voucher issue.  This is why the evangelical or not dimension has been included.
  
• The white space is interesting.  Here, the issue faced by the statistician is sparse data when one gets down to multi-dimensional subgroups.  Andrew chose to ignore all the data, which is the wise thing to do.  With so few samples, it is particularly easy to draw bad conclusions.   
• Because of the white space, we get additional information on the spatial distribution of the demographic subgroups.  The black population (at least the voters) are predominantly found in the southeast while Hispanics are in the southwest.  The subgroup of income higher than $150k is essentially all white.  Admittedly, this is a very crude read because we only have two levels (below 2% of state population and above).  Of the colored states, we cannot differentiate between densely populated and not.

  

Such rich graphics deserve careful reading.  Enjoy!
  

Recovery inside a recovery

While reconstructing the Dow price chart (here), I noticed that there was some dubious statistics going on behind the scenes.  The chart made the point that the 1929 bear market took over 20 years to recover to its peak value.  The mystery wrapped in the enigma is the existence of the time series for a 1937 bear market and a 1939 bear market.  This could not happen unless there were bears within bears and recoveries within recoveries.

The uncomplicated time-series view brings this situation out more clearly:



This is a sobering picture in the face of all the talk about "green shoots" and "bear market rallies".

From a statistical perspective, the 1937 and 1939 bear markets cannot be interpreted without noting that they happened inside of a larger bear market.

A Harvard mess 1

There are some innocent-looking charts that throw up more and more problems, the more you look at it.  This example comes from a magazine sent to Harvard alumni.  We have all heard that their endowment fund suffered some horrific losses in the last few months and so the magazine editor thought it useful to describe the potential impact on the different departments.

It's a safe bet our readers would not think to present two related data series as a combination of one bar chart and one column chart. 

As the chart stands, the intended message is completely lost.  It takes a bit of fishing to learn that the Radcliffe Institute has a tiny stake in the endowment fund but supports over 80% of its operating funds from the endowment.

Now looking a little deeper, we find that the scales of the two charts are not standardized so the length of a bar and the length of a column cannot be directly compared.  Nor can the grid-lines as each section accounts for 10% in the bar chart but 20% in the column chart (to make it worse, the larger section represents the smaller percentage!)

Looking further still, we find that "Other" accounts for some 15% of the endowment but apparently consists of entities that do not have operating funds and thus goes missing in the column chart.  In our versions below, we will ignore the "Other" category completely; this is equivalent to assuming that we have allocated to the individual schools a proportional share of "Other"'s share.

Not to mention arranging the schools by alphabetical order.

Much of this mess became apparent when we put the two charts into a uniform setting, like this:

 

A scatter plot provides good information, especially if there is clustering  although we can debate whether it is fit for general publication.

More in our next post.

Reference: "The Endowment: Each school's stake", Harvard Magazine, Jan-Feb 2009.

PS.  The initial post switched the axis labels on the two bar charts. Thanks to Jon for pointing this out.

Divided nation

Professor Gelman generally believes the red state, blue state paradigm is too simplistic to describe the American electorate.  He has been sharing some of his work on his blog, and has just published a book about this topic.  Recently he produced the following chart, which is gimmick-looking but crystal clear in its message.

Here, economic and social ideology are plotted on a scatter chart, with positive values indicating conservatism and negative values liberalism.  Further, each state is represented twice on the chart, the red point for the Republicans and the blue for Democrats within the state.

This is a cluster analyst's dream data set.  The absolute separation of the Republican cluster and the Democrat cluster is astounding: imagine a diagonal line perfectly classifying all points.

We should not miss a host of details:

• as Andrew pointed out, "the big thing we see from the graph ... is that Democrats are much more liberal than Republicans on the economic dimension: Democrats in the most conservative states are still much more liberal than Republicans in even the most liberal states."  This is clear from the wide gap on the horizontal axis.
• there is a small degree of overlap on the social ideology axis so the nation is closer together on that front.
• but wait a minute, the scale on the social axis is not the same as that on the economic axis.  This means that the extremes are more extreme on the social axis: the difference between MS and VT is roughly 0.8 on the social scale while the largest difference on the economic scale is roughly 0.5.  (here, I am assuming that the scales are comparable to each other)
• there is high correlation between social and economic ideologies: the points are well-aligned along the 45-degree line
• especially on social issues, the Democrats are divided within (the elongated shape of the blue cluster).

Reference: Gelman, "Ranking states by conservatism/liberalism of their voters", June 30 2008.

Knit-picking

In celebrating the recent trend by "elite" colleges to lowering the cost of education, the Times printed this chart, the top part of which is shown here.

The three colors represent different levels of aid.  Blue means "grants replace loans"; red means "free tuition"; yellow means "parents pay nothing".  The colleges are grouped by the minimum qualifying income for the blue category.

The whole effect is of a knit.  We shall call this the "knit chart".

I believe a simple data table will do the job nicely.  If any reader has other ideas, please show us your work!

A few points to note about the original:

• Ordering by the minimum income to qualify for "grants replace loans" is arbitrary, as is alphabetizing colleges within each group
• Qualifying "at any income level" should be shown on the left of "$40,000 or below" rather than to the right of $100,000.  The current order is such that qualifying level increases with income from left to right, except from $100,000 to "any income", where it falls off a cliff.
• Qualifying at any income level is better shown as a separate column on the right disconnected from the income scale.  The current configuration devalues the effort spent in making a proper income scale.
  
• Too many lines of equal length, and too few yellow and red lines to make the knit chart effective
  
• Should the graph cater to parents interested in seeing what aid they qualify for given their income level?  Or should the graph highlight the breadth of aid available at individual colleges?

Reference: "The (Yes) Low Cost of Higher Ed", New York Times, April 20 2008.

PS. The original point about the "any income level" was incorrect as pointed out by Chris below.  I have replaced that with a different issue.

PPS. Matias' version (see comments) is a superb demonstration of the power of data tables, well-applied.   It is clean and simple, and addresses both the questions pointed out in the last bullet point.  The only thing sacrificed was the visual representation of the relative size of the income requirements, which I agree is the least valuable part of the original.  As usual, many thanks to our readers for coming up with great ideas!

Trying too hard

In the course of business and governing, a lot of charts are generated.  An anonymous tipster pointed us to a set created by the "Communities and Local Government" division in the UK government.  Judging from the content, this division has responsibility for economic development in local neighborhoods.

Below are a pair of exhibits.  Truly they are trying too hard!  What we see is a hybrid scatter-bubble chart.  Between the jargon, the acronyms (LAD, LSOA), the boxed text, the multi-color circles, the colored axis labels and lack of title, the reader is plunged into a state of confusion.

The chart can be unraveled.  Each district was evaluated based on two measures of "gaps in worklessness".  The vertical axis compares each district to the national average; positive numbers indicate an above-average district relative to the nation.  The horizontal axis compares the most deprived 10% neighborhood within each district to the local average; positive numbers indicate worst neighborhoods improving. 

Thus, the policy goal would be to move all districts into the upper right quadrant.  The multi-color bubbles were designed to show us the state of the nation.  On the left chart, 41% of the districts (or population?) reside in the improving districts while 19% live in deteriorating areas.

The following strategies can help improve readability:

• use English on the axis
• relegate technical definitions to the legend
• add succinct title to tell the story
• use color on the data rather than on axis or data labels
• use color to draw attention to the upper right quadrant
• remove bubbles
• define acronyms

 

Ordering and grouping

The Times reported that January retail sales generally disappointed, and consumers showed a preference for discount retailers over department stores.

Taking the bar chart on the right, re-ordering by change in same-store sales, and grouping companies by type of retailer, we can present the data to match the text more closely.  The divergent performance between discount retailers and department stores is readily visible.

Reference: "Weak January dashed retailers' gift-card hopes", Feb 8 2008.

 

Light entertainment

Christopher P submitted this chart, which is great for our light entertainment series.

Apparently it came from the Netherlands and showed how privileged their citizens are compared to the rest of the world.  It would appear that they need to reverse the color scheme (and font size?) to highlight the privileged.  Comments welcome.

Source: AdsoftheWorld.com

Structuring a chart

This chart from the NYT was intended to show how the EPA has moved the bar on vehicle mileage ratings: 2008 estimates were lower than 2007 estimates across the board, regardless of manufacturer, model and city/highway.

The chart was built from one basic component, repeated for each model.  I like the discreet gridlines (the white ticks) which enable readers to count off the mileage ratings.

The data is rich: ratings were given along three dimensions (model, year of estimate and city/highway).  Readers can benefit from a stronger guidance in where to look for the most pertinent information.  As the chart stands, it is merely a container for the data.  It fails our self-sufficiency test: all the data were printed on the chart, and the bars add little.

In the junkart version, I use knowledge of the data to structure the chart. First, noting that sedans, hybrids and trucks/SUVs/minvans have different levels of mileage ratings, I clustered the models into three groups.  Secondly, the city and highway ratings were separated into two columns as I consider the between-model comparisons more important than city-highway comparisons.  The chart is a dot plot, with a vertical tick for 2007 estimates and a dot for 2008 estimates.  It's easy to see that all dots sit to the left of vertical ticks.

More subtly, we can also see that the hybrids appeared to have been penalized more.  Or perhaps, the higher the rating, the larger the downward adjustment...

Source: "Mileage Ratings Are Still Estimates, Though Closer to Reality", New York Times, Sept 16 2007.