Junk Charts

Seth on bar charts

Seth followed up his post about graphics with a specific post about pie charts versus bar charts.  He prefers pie charts.  We happen to agree with his unhappiness of grouped bar charts.  Unfortunately he compared an univariate pie chart (depicting point-in-time data) with a multivariate bar chart (iluustrating time-series data).

Here we present a different example, derived from a NYT article on diabetes in America.  The original chart is a series of pie charts, one for each age group, and one for the aggregate data.

The junkart version uses a bar chart.  Readers can get a more precise comparison of the prevalence rates across age groups because it is easier to judge lengths than areas.  This has been scientifically proven by the likes of Cleveland.

Dirty trick, you might say because the original chart actually prints the data in each pie.

So now there is no mistaking the data.  This raises a philosophical question: why bother graphing the data if the reader needs to read the data in order to understand the chart?  We call this the self-sufficiency test.  The graphical elements of a pie chart can't stand on their own.

Reference: "Diabetes - underrated, insidious, and deadly", New York Times, July 18 2008.

Right metrics

Yesterday's post focused on the purely graphical aspects of NYT's very rich graphic on CEO compensation.  Today, we take a look at the data being plotted.  Aleks already jumped the gun, pointing out one deficiency of the stock price metric.

Recall the metrics were percent change of total CEO compensation (2006-2007), and percent change in company stock price (2006-2007).

The graphic attempted to simultaneously address two sets of comparisons: the relationship between compensation and stock price changes within one company; and the relationship of each company against "similarly sized" companies on compensation, and on stock price separately.

This graphic violates Godin's (Golden?) Rule #1, Only One Message, with which we generally agree.  In trying to accommodate both comparisons, it managed to confuse readers.  In particular, as pointed out yesterday, the primary comparison of compensation against stock price is hard to discern as the scale was determined by the second comparison (between companies).

The issue Aleks pointed out is that some CEOs are paid by stock; thus, their compensation would rise and ebb as the stock price rises and ebbs.  The correlation would then indicate the structure of the pay package, rather than the (presumed) pay for performance.

Stock price, in fact, is a poor indicator of company performance, especially short-term price changes such as the one-year changes used here.  Further, we have a problem of mismatched timing: pay (excluding the stock component) moves much much more slowly than stock prices; besides, while stock prices experience positive and negative changes, pay changes are skewed positive!  All these make direct comparison of these two metrics ill-advised.

If shareholder value is still the desired metric, then one should use a longer time-series.  This will crowd out the comparison with similarly sized companies but make the graphic more useful.

One final curiosity: according to this data set, Steve Jobs did charity work for Apple during that year; he received no stock or option grants and a nominal $1 salary.  Is this real?

Bound to extremes

The New York Times continued to push the envelope by printing super-complicated data graphics (while the Economist regrettably seemed to have picked the USA Today route... more on that in a future post).  The following graphic was used to illustrate the relationship between CEO compensation and their company's stock performance.

The two dotplot lookalikes depicted the percent change in CEO pay and the change in companies stock price, in both cases, from 2006 to 2007.  The size of the dots indicates the relative value of the CEO's pay.  The gray dots depict "similarly sized" companies for comparability.

In this post, I will focus on the comparison between change in pay and change in stock price for a given CEO.  In particular, the calibration of the axis/scale is problematic.  The scale is automatically determined by an algorithm; as one switches from one CEO to another,  the graphs take on different ranges, use different axis labels, and the zero-percent points shift.

This means that the two charts have different scales.  In this example, each tick mark advances 6% in the top chart but 12% in the bottom chart.

Since the zero points do not line up, the distance between the zero and the orange dot loses meaning:  the 2.5x longer distance in the top chart actually represented the same percentage change as in the bottom chart (31% versus 28%).

In order to respect the grid-lines (white lines), the tick marks fall onto stray percentages (24%, 36%, 48%, etc.).  That's unfortunate.

What's the culprit?  This chart is "bound to extremes".  In other words, the range of the depicted data is used to determine the plot area.  The bottom chart had zero on the left edge because all the stocks depicted rose between 2006 and 2007.  It is often better to use domain knowledge to determine the plot area.  Extreme values should be omitted if they don't add to the message.  Oftentimes, by leaving extreme values in the picture, we squash the rest of the data.

This is also why programs like Excel do a poor job picking a scale.

As an aside, the use of bubbles is almost always troubling. Bubbles do not have a scale so the only information we get is relative size.  However, we can't estimate areas properly so we get the relative size wrong.  Sometimes, even the chart designer may get stumped.  In the chart of Steve Jobs, you would think his bubble (total compensation $1) would be dwarfed by all the other bubbles, as in the WSJ chart we showed the other day.  Not so.

Thanks to Todd B. for submitting this chart.

Reference: "Executive Pay: the bottom line for the those at the top", New York Times, April 5 2008.

Graphs as catalogs

Junk Charts typically concerns itself with statistical graphics a la Tufte and Cleveland, treating charts as a means to summarize, elucidate and highlight aspects of data.  We haven't been too kind on so-called infographics, often finding these cluttered and confusing.  Recently, I have a small change of heart.

I now see infographics as innovative in one way, and a complement to traditional graphics.  This is the idea of graphs as catalogs.  What many of these graphs try to do is to present a structured way for users to explore massive amounts of data.  They don't serve the traditional purpose of summarization and that's why they are innovations.

The following chart from NYT tracing Serena Williams' tennis ranking prompted this post.

As a traditional statistical graphic, this chart leaves much to be desired.  The general outline of her career could be described in one sentence without the need for any graphic.  The colorful vertical lines serve little purpose, nor the short line segments on the other side of the axis.

However, as a catalog of data on Serena's career, this graphic is fascinating.  Mousing on the vertical lines changes the information on the top right corner, including the tournament being played and the media event she participated in, as well as photos and her rankings.  Similarly, the left and right arrows on the top left allow readers to browse through the list of events chronologically. (You need to click on the link to use the interactive features.)  Without this chart, it would have been very difficult to learn about Serena's record at a particular tournament or point in time.  It acts like a data table but presents the information in a much more accessible way.

Thus, relying on interactivity, this compact graphic enables any of us to browse to a user-defined depth a reservoir of data.  Bravo!

Reference: "Serena William's Professional Career", New York Times, June 2008.

Close races

Perhaps harkening to the close race between Obama and Clinton, the designer chose to illustrate this with what we have called the "racetrack" graph.  We have previously discussed the problems here and here.

In this rendition, a pie chart was divided into three race tracks with "cities" getting the inside track and "rural/small cities" getting the outside track.  (As the Clinton supporters might say, elitism was in the air.)  There were two great choices: the courage to not print the data and let the chart speak for itself, and the wisdom to white out the votes for "others".

Nevertheless, as we discussed before, the data is coded into the angles rather than the lengths of the strips, which presents a real problem in comparing vote shares.  For example, try figuring out if there were more Obama supporters in rural Tennessee than there were Clinton supporters in cities in Tennessee (bottom right).

Also note where the white "others" space were, and the impossibliity of comparing them.

The arrangement for Wisconsin, meanwhile, posed a challenge for anyone who wanted to estimate how many rural Wisconsin voters went for "others".

In the junkart version, we go with the two-sided bar chart, typically found in population pyramids.  The information presented jumps out at you.

This chart is essentially the same as the racetrack; one just needs to straighten out the strips from the original chart, and pull the Clinton ones clockwise, and Obama ones anti-clockwise.

Reference: some recent issue of New York Times magazine.

Whither complexity?

The ever interesting Gelman blog ("Too clever by half") ponders about this enterprising NYT chart.  Whatever its merits, this is one that requires close study. 

Reception is generally positive.  Andrew himself learnt an important fact, that there are still more white people than other races in America!  In statistics, we distinguish between two types of errors, the significant kind and the ignorable kind.  From this perspective, using admissions count is a gigantic problem; it renders the rest of the chart useless.  So I agree with Andrew.  As ever, picking the right scale is the beginning of making a nice chart.

We can also use this example to discuss the concept of "interactions".  When we go about presenting small multiples, i.e. comparisons of subgroups within a population, it's because we have observed differences between those subgroups; otherwise, it is both simpler and clearer to present the aggregate results.  The present chart presents subgroups defined by race, gender, age and substance abused, that is quite a lot of subgroups. 

Focusing on the first row (Alcohol), we note that the colored mass has shifted to the right, indicating more older people abused alcohol.  This trend appeared for all races.  Now scanning the other rows, we discover that only heroin abuse showed a distinctly different pattern, but only among whites.  For every other row, it seemed that the change from 1996 to 2005 was similar across races.

By breaking out substance abused, the designer added 21 little charts (7 sets of 3).   Only one set  (heroin) added information to what was true in aggregate i.e. that substance abusers got older.  The incremental gain in information does not justify the added complexity.

Nevertheless, the chart had many positive things such as judicious use of axis and gridlines and letting the graphical constructs speak for themselves (without accompanying data labels).

 

Reference: "Why is Mum in Rehab?",  New York Times, Jun 14 2008.

The right scale

Oftentimes, picking the right scale for a chart makes all the difference.  The following chart showed up in the New York Times Magazine some time ago.  Readers will immediately recognize this as "infotainment" rather than a serious attempt to convey the data.

The data came from a study by the Center on Education Policy which counted the amount of instruction time spent on various subjects at a sample of elementary schools in the U.S.

A simple bar chart would make a nice graphic, as shown on the right.  Instead of sorting by decreasing minutes, we pulled out "lunch" and "recess" since they belong to a separate category.

Our main focus, though, is on the scale.  The original report - and thus the original graphic - used minutes per week.  We contend minutes per day (or even hours per day) to be more user-friendly.  This is because any number makes sense only in comparison to other numbers.  There is no easy reference to a number such as 500 minutes per week.  However, being told it's 100 minutes per day (or 1 hr 40 min per day) means a lot because everyone knows there are 24 hours in a day.

This is a small example of a larger problem with using averages.  The media loves to give out statistics like six people are dying of diabetes every minute (e.g. here).  This is typically done by dividing the total number of diabetes-related deaths in a year by the number of minutes in a year.   Why divide by total number of minutes in a year?  The fallacy of such a calculation is evident if one applies this logic to natural deaths (since we all have to die some day).  As the world population grows, there will just be more and more people dying every minute!

Choosing the appropriate reference point -- just like picking the right scale -- is the beginning of any good analysis.

Reference: New York Times magazine, April 27 2008; Center on Education Policy.

Knit-picking

In celebrating the recent trend by "elite" colleges to lowering the cost of education, the Times printed this chart, the top part of which is shown here.

The three colors represent different levels of aid.  Blue means "grants replace loans"; red means "free tuition"; yellow means "parents pay nothing".  The colleges are grouped by the minimum qualifying income for the blue category.

The whole effect is of a knit.  We shall call this the "knit chart".

I believe a simple data table will do the job nicely.  If any reader has other ideas, please show us your work!

A few points to note about the original:

• Ordering by the minimum income to qualify for "grants replace loans" is arbitrary, as is alphabetizing colleges within each group
• Qualifying "at any income level" should be shown on the left of "$40,000 or below" rather than to the right of $100,000.  The current order is such that qualifying level increases with income from left to right, except from $100,000 to "any income", where it falls off a cliff.
• Qualifying at any income level is better shown as a separate column on the right disconnected from the income scale.  The current configuration devalues the effort spent in making a proper income scale.
  
• Too many lines of equal length, and too few yellow and red lines to make the knit chart effective
  
• Should the graph cater to parents interested in seeing what aid they qualify for given their income level?  Or should the graph highlight the breadth of aid available at individual colleges?

Reference: "The (Yes) Low Cost of Higher Ed", New York Times, April 20 2008.

PS. The original point about the "any income level" was incorrect as pointed out by Chris below.  I have replaced that with a different issue.

PPS. Matias' version (see comments) is a superb demonstration of the power of data tables, well-applied.   It is clean and simple, and addresses both the questions pointed out in the last bullet point.  The only thing sacrificed was the visual representation of the relative size of the income requirements, which I agree is the least valuable part of the original.  As usual, many thanks to our readers for coming up with great ideas!

Small multiples re-imagineered

This chart gave me trouble.  I kept staring at it, staring.  Searching for the legend.  What could the several lines, in different colors, represent?  Take a look yourself.

Well, it turns out all three graphs were duplicates.  A different line was given dark blue to highlight a particular amusement park.

I have not seen this tactic used before.  This is like a small multiples concept except that every chart contains the same data.  Is it better than having just one chart?

Reference: "Will Disney Keep Us Amused?", New York Times, Feb 10 2008.

PS. [4/6/2008]  Here are two alternative charts contributed by our readers.  See comments below.

Derek suggested using sparklines:

Zuil reverted to basics:

Playful and exploratory

I share reader Bernard L.'s enthusiasm for this very imaginative chart, courtesy of the graphics people at NYT.  The chart captures the ebb and flow of weekly movie receipts over the last two decades.

The details that particularly interest me include:

• The addition of area colors (on top of lines) serves to highlight box office successes; this really helps readers sort out the massive amount of data
• Nicely spaced text (and dots) does not interfere with our reading of the chart
• The hiding of text for less important films, plus taking advantage of interactivity to show their titles if the reader mouses over the respective areas

All of the above indicate a keen sense of foreground versus background.  Besides, the authors had the good sense to speak of inflation-adjusted box office sales; I'm tired of the movie industry proclaiming higher sales each year when ticket prices are rising, and the population is growing.

This is another chart where more data do not easily translate into better communication (see my guest post at Flowing Data).  While I like the playful nature of the interactive chart, it is left to the reader to discover the information buried in the data, such as the assertion in the header that Oscar-winning films typically take time to attain box-office success while many blockbusters do not Oscars make.

In this presentation, it is challenging to compare the total receipts of one film versus another (this requiring comparing oddly shaped, partially obscured areas).  It is also hard to compare across years since the data is spread out over a lot of space.

There may really be two types of graphics: the one like the example here which is a dictionary and designed for exploration; and the other kind where the designer has selected a subset of the data to make a specific point.

Reference: "The ebb and flow of movies", New York Times, Feb 23 2008.