Junk Charts

How to mess up a bar chart

Karen Lopez at InfoAdvisors sent in what she calls "the stupidest bar chart of 2011" (link), showing us that one can mess up bar charts too.

This chart originally came from an infographic (link) published by Klout, which is a venture-funded startup that is creating an "influence" rating of online entities.

***

If we use the Junkcharts Trifecta Checkup (link), we find that the design failed on at least two of the three criteria.

What is the interesting question being addressed? It's the relative "influence" of the top websites in 2011. Readers would probably want to learn more, such as why it's the Top 11 rather than Top 10. Also, as they pointed out in the write-up, a tiny company called "SoundCloud" appeared in this list, which calls into question the ranking methodology. In addition, I wonder what people go to McDonald's website to do, and why Walmart instead of Amazon.

The choice of data is the key failure here. While Klout is a rating system, for some reason they chose to display the ranking on this bar chart. Ranks invite readers to presume that the difference between Facebook and Youtube is the same as between Walmart and Netflix, or that the difference between Facebook and Skype is double that between Facebook and Youtube, or that the difference between Facebook and Apple is the same as the difference between YouTube and McDonald's, etc.

In reality, and especially for this sort of data (influence), one is likely to encounter highly skewed distributions.

The graph itself is executed poorly. Karen asked:

Notice how 11th place Facebook has more bar?  But it’s in the worst place in the list.  Or is it?

The bar chart would make sense if rating is being plotted. With rank data, the bar chart is completely redundant.

Two lines dropping: further reading

Light entertainment for the weekend, from Business Week (tip from my friend Andrew).

This chart is a nice complement to my post on "Two Lines Dropping" (here).

More charts at the link.

Lost in complexity

Felix Salmon (link) and others linked to this BBC News graphic about European debt recently.

***

At first sight, the use of arrows inside a ring, enhanced by an interactive filter by country, seems to be an inspired idea.

Then, I started clicking. Here is the German view.

 

According to the paragraph beneath the headline, the arrows show how much money is owed by each country to banks in other nations. So, it appears that German banks have lentborrowed about equal amounts tofrom France and Italy, and also good amounts tofrom the U.S., the U.K. and Japan. And German banks would be affected if these debtors were to default.

Now, take a look at the right column where BBC tells us "The biggest European economy is exposed to Greek, Irish and Portuguese, but mostly, Spanish debt." Say what?

Much more important than appearance, the designer must ensure that the data and conclusions make sense. Here, the chart doesn't support the discussion.

***

See also my previous post about Europe debt. 

 

 

Two lines dropping

Reader Ron D. was not pleased to see this dual-axis chart purporting to show a cause-effect between the decline in union membership and the drop in the proportion of income earned by middle-class households (defined as the middle 60% of households). Click here to read the original article. They credit CAP's David Madland and Karla Waters for this chart.

Using dual axes is a well-tested way of creating correlation where there may be none. Playing with the scales will do that for you. I wrote about this issue here.

However, the correlation in this data cannot be denied, as the scatter plot below shows. Note that the scatter plot is much better at revealing correlational patterns than a chart with multiple time-series lines. (Here's an example of two lines that display a spurious correlation.)

If one were to ask for a linear regression line, one will obtain a very high R-squared indeed (over 0.9). The problem is with the interpretation of this correlation. Any two data series that move with time will be highly correlated with each other, just because each series is highly correlated with time. Despite what you might believe after reading Freakonomics, regression -- especially in social science data -- cannot prove causation.

The writers at Think Progress show no such restraint, from the title "The American middle class was built by unions and will decline without them." to the sentences "these assaults have successfully decreased union membership over time... this has had a detrimental effect on the American middle class."

Note: these statements may in fact be true; I'm just pointing out that the chart does not buttress the assertions.

***

It's often hard to elevate a correlation to a causal effect. We have to try different tests. One such test for this data set is: if a change in union membership causes a change in middle-class incomes, then we'd expect that  the annual changes of one to be correlated with the annual changes of the other (at least in direction, better in magnitude).

So, in a year in which union membership declined a lot, one should expect to see middle-class incomes also drop substantially.

The next scatter plot contrasting these annual differences suggests that causation is probably absent. At this smaller time scale, one just doesn't see any correlation at all. Annual declines in the proportion of union membership has been around 2-4% for most of this period but shifts in middle-class incomes have been ranged widely in terms of direction as well as magnitude.

P.S.  Andrew suggested connecting the lines. Here are the charts with the lines:

What appears to be a very strong correlation on the left chart does not look that well-coordinated on the right chart!  (The lines connect the dots in chronological order.)

Another take on 9-9-9

I wasn't too happy with the previous chart (with a nod to zbicyclist). Here is another take on how 9-9-9 affects different households:

 The challenge is how to convey the distributional effects of the tax plan in addition to its effect on averages. One can use expected values and do away with the distributional effects but doing that removes important information. The analysts show that in the top 20%, half of the households receive tax cuts while for the other 80%, at least 85% will see their taxes increased.

In the previous version, I group the data by quintiles (20% buckets); in this version, I group them by whether taxes went up or down.

 

 

Things they don't teach you at Harvard

Numeracy and graphics apparently are not on the syllabus at Harvard, if one were to judge from this chart from the most recent edition of Harvard Magazine:

I added the orange circle to show a typo. But pretty much everything is wrong with this chart, from the sideways labels to the redundant use of data labels, column lengths and axis labels to plot the same series of data. (See the posts on self-sufficiency.)

Meanwhile, the insistent printing of five digits for the dollar amounts and the academic-year styling of the years are giveaways for innumeracy: the reliance on precision where it is neither useful nor necessary. Nothing at all is lost by rounding the amounts or shortening the years.

Most worrisome is the defiant announcement that the data is "not adjusted for inflation". One might as well replace that with "this analysis can't be trusted". When plotting a long time series, there is no excuse for not adjusting currencies for inflation. A dollar in 2010 just does not have the same spending power as a dollar in 1986. This is not a mere oversight; it's a major blunder, as revealed by the following junkart version:

When the data is not adjusted, it appeared that tuition, room and board jumped by 220% between 1986 and 2010. However, when adjusted for inflation, the cumulative growth was only 75%. Without the adjustment, the reported growth was exaggerated threefold.

***

This is a good example to explain, yet again, why statisticians adjust data. The raw data is the blue line shown above. It shows the cumulative growth of 220%. This metric is misleading. One should ask: how big is this 220% growth, really? It should depend on how much more expensive other goods and services have become over the same period of time, shouldn't it? If the costs of groceries, clothing, fuel, etc. have all gone up by 200% or so, or if wages have also gone up by that amount, then the steep blue curve would not be so scary.

The green line is the raw data adjusted by the CPI index. CPI is the inflation index which, put simply, is the average price index of a basket of goods and services. So, the green line is how much tuition, room and board has grown after "controlling" for the growth of the average basket of goods and services; it's the "excess" growth.

Of course, the excess growth is what the chart should be portraying.

The massive burden of pie charts

The world would be a better place if pie charts were banned. I respect the creativity that went to producing charts like the one below but surely, pie charts don't deserve that level of attention:

(Via Business Insider via Jobvine)

Here is the same information using a bar chart.

 

It's amazing how much extra stuff (colors, text, shapes, lines, labels, etc.) is needed when you use pie charts.

Light entertainment: the chartjunk crisis

New York Time Magazine: Dire Straits: How We'll Survive the Economic Crisis (link)

Motion-sick, or just sick?

Reader Irene R. was asked by a client to emulate this infographic movie, made by UNIQLO, the Japanese clothing store.

Here is one screen shot of the movie:

This is the first screen of a section; from this moment, the globes dissolve into clusters of photographs representing the survey respondents, which then parade across the screen. Irene complains of motion sickness, and I can see why she feels that way.

Here is another screen shot:

Surprisingly, I don't find this effort completely wasteful. This is because I have read a fair share of bore-them-to-tears compilation of survey research results - you know, those presentations with one multi-colored, stacked or grouped bar chart after another, extending for dozens of pages.

There are some interesting ideas in this movie. They have buttons on the lower left that allow users to look at subgroups. You'll quickly find the limitations of such studies by clicking on one or more of those buttons... the sample sizes shrink drastically.

The use of faces animates the survey, reminding viewers that the statistics represent real people. I wonder how they chose which faces to highlight, and in particular, whether the answers thus highlighted represent the average respondent. There is a danger that viewers will remember individual faces and their answers more than they recall the average statistics.

***

If the choice is between a thick presentation gathering dust on the CEO's desk and this vertigo of a movie that perhaps might get viewed, which one would you pick?

 

Reading the landscape

Here are some posts I find worth reading on other graphics blogs:

Nick has done wonderful work on the evolution of the rail industry in the U.S., with a flow chart showing how mergers have produced the four giants of today, as well as a small multiples of maps showing how they split up the country.

A lovely feature of the flow chart is the use of red lines to let readers see at a glance that Union Pacific is the only rail company that has lasted the entire 4 decades, while the other 3 giants came into being within the last 20 years.

On the maps, notice a slight inconsistency between the left and right columns: on the right side, both maps have the same set of anchor cities, which act as "axes" to help readers compare the maps; on the left side, the sets of anchor cities are not identical. It would also be interested to see a version with all four route maps superimposed and differentiated by color. That may bring out the competitive structure better.

 ***

Georgette has a nice post summarizing issues with picking colors when producing charts. Her blog is called Moved by Metrics.

 ***

Meanwhile, Martin finds a shockingly poor pie chart here.

***

There was a time where you'd find the kind of heatmaps featured here by Nathan as wallpaper in my office. It's a great visualization tool for exploring temporal patterns in large data sets. However, I'd never even think of putting these in a presentation.  It's a starting point, not an end-point, of an analysis project. Some things are wonderful for consumption only in private!