Junk Charts

The shackle of time 2

In the last post, we removed the time dimension in order to clarify certain aspects of the S&P 500 returns.  We found that with an investment horizon of five years, there was historically about a 25% chance of losing money and a 25% chance of more than doubling.

Even though we looked at cumulative returns, it was still the case that the data was serially correlated; in other words, it could be that the eventual return was not independent of the starting year of the five-year period.  To gauge this, we must return to the time dimension that was previously removed.

The chart on the right plots the five-year returns for all five-year periods starting from 1910.  What it shows is that even with longer time frames, timing or luck still plays a key role. 

For example, any such investment in the S&P 500 between late 1950s and 1980 did not double in five years no matter which year the investment was made.  Then again, if the investment was made in the 40s and 50s, no one lost money in a five-year period, similarly in the 1980s.

So the fact that we saw a 25% chance of doubling (or losing money) over history says much less about what might happen in the next 5 years than the simple number suggests.

In response to a reader's comment - the data series was described as "real total return" so these are inflation-adjusted.

Seen at Starbucks

Merry Christmas!

If you happened to be in a Starbucks recently, you might have picked up some charts, which was what happened to one of our regular readers and commentators, ZBicyclist, who then tried his hand at chart critique here.  He is worried they may reach millions soon.  So look out!

This graph on the right -- which should rightfully be called a "tree ring graph" -- seems to me a fantastic concept although it is hard to think of data that would deserve this treatment.  Certainly not the retail sales series plotted here! 

This is an object lesson in why bubble charts often fail.

One issue is scale: note the awkward way in which the innermost ring is used to designate the oldest sales data of $375 billion presumably in 1996, and think about how you would decide where to place the 2007 ring.  (It's arbitrary.)

Another problem is labeling: when the growth is slow, the rings are close together, and labels have to be jittered (look at 2001 and 2002).  In this case, a relatively simple solution is to have the entire series of years run diagonally.

Yet another challenge is relative radii versus relative areas.  Inevitably, some readers will respond to the areas while others will respond to radii  ZBicyclist, for example, belongs to the first group while in this case, I find myself siding with the latter.  When the bubbles/rings overlap, it is difficult to assess areas.

Of course, a simple line chart would do the job with minimal fuss.  The following chart issued by the National Retail Federation actually plots the growth rates, rather than the annual sales.

(Please lose the grid-lines, or else add a vertical axis and lose the data labels.  A column or dot chart is also slightly preferred.)

Now go read ZBicyclist's point of view.  In the meantime, let us know if you think of ways to use the tree ring graph.

Political theater

Jens, a long-time reader, tried to re-make the boring data tables used to report poll data.  Here is an example from USA Election Polls (left) and his enhanced version (right).

Like Jens, I find most of the tabular presentation of poll data underwhelming.  Too much data hiding all the useful information.  For example, the pollster and polling date data provide a context for super-serious poll watchers to interpret the data; however, they do not present themselves in a way that actually help readers.  Read further for versions that bring out this data much better.

Meanwhile, Jens' revision uses color and ordering to bring out the current state of affairs.  The addition of electoral votes allows us to understand the relative weight of each row, countering the weakness of the tabular format, that each row has the same height, implying erroneously that they have the same importance.

There are a number of good web-sites where this type of data is presented in attractive ways.

I have been a fan of Political Arithmetik, which made great use of the pollster and polling date data mentioned above.  Those data have been averaged to show the overall trend while the individual poll results are plotted as dots in the background.  The polling date data is embedded in the horizontal positions of the dots.  Even more impressively, the margins of error are presented.  Remarkably, this race has been a statistical tie for all these months, the 95% lower limit never quite making it above the zero level.

Another great site is fivethirtyeight.com.  Below, they essentially turned Jen's enhanced table into a map.  The legend on the right perhaps represents what they call "East Coast bias"?  All of Nathan's graphs are very attractively produced; I just wish he'd put more labels on them (such as the differentials corresponding to shades of red and blue.)

Sloppy statistics

As hinted in the previous post, there are rare situations in which pie charts are acceptable; typically, these charts must show proportions that add up to 100%.  If column charts (or line charts) are used instead, readers who aren't careful may assume incorrectly that the columns add up to the whole.

Pie charts show distributions.  How should one state the key message of the following pie chart?

I. Type A is the majority.

II. The most frequent type is Type A.

III. Type A is a minority.

IV. Every other type but A form the majority.

I would pick statement II, followed by statement I.  Statement I is the only false statement out of the four if one uses a strict definition of "majority" (more than half).  If one goes by the spirit rather than the word of the law, statement I does pick up the key message albeit imprecisely.  Statement III is a true statement but particularly misleading in the context of this pie chart.  For every type is a minority type if we define "minority" as less than half.  Statement IV is a tortuous way to define a "majority" where there is none.

Neither III nor IV points to a key feature of the data.  It seems ridiculous to even include them.  Lets reveal the underlying data.

Last week, a story coursed through the mainstream media, relating to the above projections published by the Census Bureau.   (Projections were created for 2050 but mention was made of the fact that the largest racial group would account for less than half the population by 2042.)  Here were some of the headlines:

"2042 to see a white minority" (New York Post, 8/14/2008) -- III

"Minorities fixed to become new majority" (Daily Vidette, Illinois State University, 8/20/2008) -- IV

"US set for dramatic change as white America becomes minority by 2042" (Guardian, 8/15/2008) -- III

"...minorities collectively will make up the majority of people in America by 2042..." (Detroit Free Press, 8/21/2008) -- IV

Like I said, statement III is strictly speaking true but by 2042 every race is projected to be a minority.  Statement IV is just odd: of course, if one started adding up enough "minority" types, one will eventually attain majority.

Not all is lost, however.  The following headlines painted a more vivid image:

"Whites to lose majority status in US by 2042" (Wall Street Journal, 8/14/2008)

"White Americans no longer a majority by 2042" (Associated Press, 8/13/2008)

Elsewhere, a Boston Globe column makes an important observation: that Hispanic whites should probably be grouped with whites rather than Hispanics.  Technically, he argued that Hispanic is not a race.  From his point of view, the pie chart looks like this:

The dog ate the margins

In his column on automated polls versus traditional telephone polls, the Numbers Guy at Wall Street Journal gave us a few entertaining quotes.

"The dog could be answering the questions, " Ann Selzer, a traditional pollster, said of automated polling, which occurs through automated voice messages to voter who record responses.  Also, WSJ cited a prominent textbook which labelled them as "Computerized Response Automated Polls -- insulting acronym intended."

Reader Mark A. brought this to our attention because of the following chart.  He wondered what the point of the vertical axis was. 

Aside from that cosmetic problem, the biggest issue is the lack of explanation.  Predictive power, pollster-introduced error, methodological error: what are these?  The article itself gives no clues.  To make sense of the chart, readers need to consult Nathan Silver's (excellent) site, fivethirtyeight.com.  (The gory details here.)

By the way, Nathan's site has a variety of nicely produced charts.  (Like this one, readers will need to dig around to collect background information to interpret some of those charts.)

Another improvement is to provide some sense of the variance in the data, either by showing more than the top five pollsters or by showing the range of errors.  Since the average pollster sits on the right edge, it is as if the right half of the chart was clipped.  In the version below, we found most polls hovering around the average, with two egregiously bad.

If we know which polls are automated and which aren't, then color the dots accordingly.

There are bench players on every chart: these are the titles, axes, labels, text and so on.  They provide background information required to interpret the chart.  They may sit in the margins but their value is not to be underestimated.

Don't let the dog eat the marginal information.

Reference: "Press 1 for Obama, 2 for McCain", Wall Street Journal, Aug 1 2008.

Bound to extremes

The New York Times continued to push the envelope by printing super-complicated data graphics (while the Economist regrettably seemed to have picked the USA Today route... more on that in a future post).  The following graphic was used to illustrate the relationship between CEO compensation and their company's stock performance.

The two dotplot lookalikes depicted the percent change in CEO pay and the change in companies stock price, in both cases, from 2006 to 2007.  The size of the dots indicates the relative value of the CEO's pay.  The gray dots depict "similarly sized" companies for comparability.

In this post, I will focus on the comparison between change in pay and change in stock price for a given CEO.  In particular, the calibration of the axis/scale is problematic.  The scale is automatically determined by an algorithm; as one switches from one CEO to another,  the graphs take on different ranges, use different axis labels, and the zero-percent points shift.

This means that the two charts have different scales.  In this example, each tick mark advances 6% in the top chart but 12% in the bottom chart.

Since the zero points do not line up, the distance between the zero and the orange dot loses meaning:  the 2.5x longer distance in the top chart actually represented the same percentage change as in the bottom chart (31% versus 28%).

In order to respect the grid-lines (white lines), the tick marks fall onto stray percentages (24%, 36%, 48%, etc.).  That's unfortunate.

What's the culprit?  This chart is "bound to extremes".  In other words, the range of the depicted data is used to determine the plot area.  The bottom chart had zero on the left edge because all the stocks depicted rose between 2006 and 2007.  It is often better to use domain knowledge to determine the plot area.  Extreme values should be omitted if they don't add to the message.  Oftentimes, by leaving extreme values in the picture, we squash the rest of the data.

This is also why programs like Excel do a poor job picking a scale.

As an aside, the use of bubbles is almost always troubling. Bubbles do not have a scale so the only information we get is relative size.  However, we can't estimate areas properly so we get the relative size wrong.  Sometimes, even the chart designer may get stumped.  In the chart of Steve Jobs, you would think his bubble (total compensation $1) would be dwarfed by all the other bubbles, as in the WSJ chart we showed the other day.  Not so.

Thanks to Todd B. for submitting this chart.

Reference: "Executive Pay: the bottom line for the those at the top", New York Times, April 5 2008.

Reconstruction

The following charts showed up in Internet Retailer, and ripe for some reconstruction.


There are few situations in which a grouped bar or column chart is the best choice.  In such charts, readers frequently have to examine the tips of the bars and yet the bodies of the bars obstruct comparisons.  Placing data labels instead of an axis is a nice touch; lining the labels up would be even better.   The junkart version below uses a dot plot which allows for comparisons within each payment type, and comparisons between payment types, to reveal themselves.



The second chart is also unnecessarily complex.  The use of double axes announces trouble, so too does the superposition of lines and columns.   The data to ink ratio of the chart is low because the data in the columns adds up to the numbers in the line.   Crucially, it is always important to clearly point out projected values (versus actual values).  Here is a junkart version.

The first revision focuses on dollar volume, showing that despite faster growth, alternative payments are merely catching up to traditional payment growth.   The higher growth rate is applied to a much smaller base!

The second revision focuses on growth rates.  Notice that all values here are projections.


Reference: "New way ahead for online payment methods", Internet Retailer, Nov 2007.

Cram it like Koby

You have to gradually build up your gut by eating larger and larger amounts of food, and then be sure to work it all off so body fat doesn't put a squeeze on the expansion of your stomach in competition  -- Takeru Kobayashi, six-time champion of the Coney Island hot dog eating contest

Kobayashi is a phenom.  He can stuff 60 hot dogs or 100 burgers in ten or twelve minutes and show no consequences.  Ordinary people can't hope to emulate these feats.

Junk Charts sees Kobayashi as a hero; an anti-hero really.  We are ordinary people; we can't hope to cram it like Koby.  A message we keep repeating here is: too much data sinks a chart.

Not long after this chart showed up in the Economist, several readers urged us to take a look.  It's a well-nourished chart indeed, one to challenge Kobayashi, but for all that it contains, the reader has to try very hard to find insights.  What with the multiple colors, iron-fisted gridlines, above-and-below boxes, dotted and solid lines, and a legend with nine pieces split in two spots?  Besides, the U.S. boxes grab all the attention by virtue of them being wider (country being more partisan).

The key to unraveling this chart is to identify the relevant comparisons:

• UK average vs US average
• UK left vs US left
• UK right vs US right
• UK independent vs US independent

And then for the gluttonous:

• UK right vs US left
• UK left vs independent vs right
• US left vs independent vs right

In the junkchart version, we address these comparisons sequentially.

(Apologies for the tiny font.)

We are again using a small multiples approach that places four comparisons next to each other: average, left, independent, right. Consistently, the British is to the left of Americans.  The only places where the two cultures meet are where liberals agree on "ideology" and "military action".

Also note that we use a symmetric horizontal scale centered at 0.  There are too many charts out there where the center is not at the center!

A similar presentation addresses the other three comparisons.  Democrats in the U.S. are miles to the right of Tories in terms of "religion".  In the UK, Labor and Tories are not much different except on "ideology".  In the US, Independents lean closer to Democrats.

Joining the lines (I hear the grumbles) helps bring out the gap between the groups being compared.  Without lines, the chart would look like this.

It is often hard to keep track of which dot is which as they trade order from issue to issue.

PS. Anyone knows what is being measured on the horizontal axis?  The original graph mysteriously stated "respondents' views".

References: 

Eric Talmadge: "Pigout champion Kobayashi limbers up for hot dog gold" June 25, 2004

"Anglo-Saxon Attitudes", Economist, Mar 27 2008.

Chart cleanup

Anna E. submitted this great example from Yahoo! Green.  A well-meaning chart but stuffed with redundancy.

Much appear to be going on and yet the entire chart contains 15 data points, Boston's ranks on each of 15 categories.  The bar lengths convey the same information as the data labels.  The legend provides a catchy name for different levels of ranks (0-10 = "leader"; 10-20 = "advances"; etc.).  The colors merely reiterate the catchy titles.  Similarly, the colored squares repeat the information in the bars.

In the name of green, we cleaned up this chart:

As a standalone graph, the categories should be ordered by Boston's ranks.  Here, we assume that cross-referencing cities is needed so we leave the order unchanged.

Football rankings 1

The Times' sports pages made wise use of graphics in a series of NFL articles recently.  Here is a rank plot (below left) comparing Jaguars quarterback David Garrard to seven other quarterbacks who started the weekend of January 5.

Simple and effective, this chart does not fuss around in showing us where Garrard ranks relative to the others. 

The junkart revision (below right) plays with a different scale: the spacing between the tick marks represent proportional differences in the underlying metric.  This gives us a little more: for example, Garrard's second rank in completion percentage is less remarkable than first thought as he essentially tied with the 3rd and 4th best while the top six were bunched between 60 and 65 percent.

But Garrard's touchdown to interception ratio stands out as the next best quarterback attained only about half his ratio.  (Todd Collins who had not thrown an interception until that time was omitted; he also had only started four games.)

References: "Two Dreams (One Big, One Tiny) Come True", New York Times, Jan 4 2008; ESPN statistics.