Junk Charts

Mid-week light entertainment: The Economist imitates USA Today

The Economist imitates USA Today, but without the whimsy. Joe D. submitted this via Twitter.

(link)

Experiments with multiple dimensions

Reader (and author) Bernard L. sends us to the Economist (link), where they walked through a few charts they sketched to show data relating to the types of projects that get funded on Kickstarter. The three metrics collected were total dollars raised, average dollars per project, and the success rate of different categories of projects.

Here's the published version, which is a set of bar charts, ranked by individual metrics, and linked by colors.

This bar chart does the job. The only challenge is the large number of colors. But otherwise, it's not hard to see that fashion projects have the worst success rate and raised relatively little money overall although the average pledge amount tended to be higher than average.

The following chart used more of a Bumps chart aesthetic. It dropped the average pledge per project metric, which I think is a reasonable design choice. The variance in pledge amount is probably pretty high and thus the average may not be a good metric anyway. The Bumps format though suffers because there are too many categories and the two metrics are rather uncorrelated, resulting in a spider web. Instead of using colors as a link, this format uses explicit lines as links between the metrics.

The following version combines features from both. It requires no colors. It drops the third metric, while adopting the bar chart format. The two charts retain the same order of categories so that one can read across to learn about both metrics.

 

PS. Readers want to see a scatter plot:

The overall pattern is clearer on a scatter plot. When there are so many categories, it's a pain to put the data labels on the chart. It's odd that the amount pledged for games is the highest of the categories and yet it has among the lowest rate of being fully funded. Is this a sign of inefficiency?

Gelman joins in the fun

The great Andrew Gelman did a Junk Charts style post today, and very well indeed.

The offending Economist plot is the donut chart, which is a favorite of that magazine.  I commented on this type of chart before.

Andrew created two alternatives, one is a line chart (profile chart) which is often a better option (despite the data being categorical), the other is more creative, and the better of the two.

 

Some of Gelman's readers complained that he arbitrarily "standardized" the data by indexing against the average of the countries depicted; one can further grumble that a 50% "excess" may sound impressive but it would be equivalent to less than an hour, perhaps not as startling. These types of complaints are fair but do realize that blog posts like these are primarily concerned with how data is best visualized. If one prefers a different indexing method, or a different set of countries, or a different color for the lines, etc., one can easily revise the chart to reflect those preferences.

The easiest way to see why the third chart is better than the first is that the strongest message coming off the first chart is that there are no material differences between these six countries in terms of time usage but in the third chart, the designer (here, it's Gelman) is asserting that there are interesting differences.

Another view of the Indian states

The previous post has elicited protests of "it's not that bad" from some corners. Well, it's bad. Let's look at it from another angle.

***

We start with the Economist chart, and ask what is the message.

The chart is saying that in 2006-8, there are 10 Indian states that have female-to-male-babies ratios below the world average (the so-called "natural ratio"). For those who know their Indian geography, the chart gives the names of these anomalous states. The chart also tells us among these 10 states, some have been gaining and others have been losing ground when compared to the 2001-3 period. There is no obvious pattern as to which states are gaining, and which losing.

That's pretty much everything that one can discern from this chart.

The problem is, the average Economist reader already knows that in India, as in China and many Asian countries, there are more male babies than female ones than in other parts of the world. If he or she doesn't know this fundamental statistic, the chart does not help because it says nothing about the other 24 states that make up the Indian average.

Worse, the chart raises the suspicion of voodoo statistics. It suggests that the other 24 states have a gender ratio that is at least equal to, if not above, the natural ratio. One would then have to believe that either the overall Indian average is higher than the natural ratio, or the negative deviations from the world average (as shown on the chart) are quite a bit larger than the positive deviations (not shown), or that the states with positive deviations (not shown) are generally less populous than the ones shown.

Either of the last two conclusions, if true, would be interesting because it implies that the cultural norms, typically claimed to explain this anomaly, are entrenched only within certain geographies. Then, it is inappropriate to speak of India's sex ratio, given this variability between states.

***

As I pointed out in the prior post, with two data series (two observation dates of the same statistic) at their disposal, the Economist chart focuses on the more recent data. This self-imposed restriction obscures meaningful differences between states over time.

The junkcharts version shows that the current 11 "worst" states could be clustered into two groups: the first group (black lines) has gained ground over the last decade, while the second group (gray) has stagnated, and in some cases, lost ground.

What's more, we learn that every one of the states in the gray group is ranked higher than those in the black group at the start of the decade.

Further, while the distance between the black and gray groups have narrowed over the decade, the gray group, despite the slight decline, is still ranked above the black grup except for Kerala, which has seen dramatic improvement.

Those who know their Indian geography might have further insights as to why the states cluster in this way.

In my view, these findings are much more interesting than the things one can learn from the original chart.

 

 

 

A skewed view of ten Indian states

The Economist published this chart to illustrate the problem of the "missing girls" in Indian society.

The girls-to-boys ratio (ages 0-6) should be about 952 but in India, it is 914. That's an average number for 35 territories, and the most skewed ratio was 830 in Punjab.

Curiously, the Economist chose to focus on only 11 states instead of showing all 35. The first 10 of these had sex ratio below the natural number of 952 while the last one was over the average. Nowhere on the chart or in the article is it explained whether the unmentioned 24 states all had above-average sex ratios: unlikely, unless certain states have much higher youth population than others.

In fact, the reference line of 952 is misplaced. Readers will find that there are two metrics depending on which survey one is looking at, either sex ratio at birth or sex ratio for children aged 0-6. The natural ratio of 952 is for the 0-6 measure but the data by territory are all for the at-birth measure. Instead, the dotted red line needs to be at 904, which is the national average sex ratio at birth for India for the 2006-8 period.

***

The lethal error in this chart is not starting the horizontal axis at zero. 
By cutting off the same amount from each bar, this chart messes up the ratio of lengths, and presents a misleading picture of the relative sex ratio between territories.  We may think Punjab's sex ratio is half that of Gujarat (in the original chart) but as the chart on the right shows, that is far from the truth!

***

The other unfortunate practice, typical of the Economist, is to stick a second set of data on the right of the chart as an afterthought. In fact, that data representing the change in the sex ratio over time is more interesting than what the exact sex ratio was in each territory in 2006-8.

A much better way to present the data, without favoring one series or another, is the Bumps chart, as shown below. We can clearly see that the improvement in sex ratio is concentrated on those states that started out the decade in a worse shape.

 

Perhaps the Economist doesn't take its own advice

Given the recent post questioning the value of the MBA degree, one would think the Economist powers-that-be would not be staffing up MBAs. But then, if not useless MBAs, how would the Economist explain this chart they printed next to the said article?

This chart appears to tell us that all the top MBA programs succeed in reducing their students' earning potential. In each case, the "pre-MBA salary" exceeds the "salary on graduation".

More likely, the red part is the incremental salary, possibly explained by the value of the degree while the gray part is the pre-MBA salary.

However, since the author has few nice words to say about business schools, one can never be 100% sure if he is presenting some counter-intuitive data.

***

In the Trifecta checkup, one would find nothing wrong with the chart type, nor is there anything wrong with asking the return on investment of an MBA degree. 

The third component -- having the right data -- is what renders this effort a failure. It is too simplistic to measure return on investment on the salary upon graduation. Surely, one must also include future career paths, intangible benefits from network relationships, personal development, etc.

 

Loss aversion and faux accuracy

Reader Bernie M. is not a fan of this Economist chart.

The chart was prepared by Aurora Flight Sciences, an aircraft manufacturer, commissioned by a professor who supports the concept of maintaining a fleet to pump sulphuric acid into the stratosphere as a way to induce artificial cooling to counteract human-induced global warming.

The chart appears to compare many different ways of shooting the acid into the skies along two dimensions: cost and altitude.

Bernie wrote:

I find the choice of axes extremely counterintuitive. Altitude one would expect on the y-axis. And mixing up the scatter chart elements with the connected line chart doesn' really help either.

The convention regarding axes is to put the outcome variable on the vertical, and the explanatory variable on the horizontal. Thus, in this case, if the cost of a particular solution is primarily determined by the "altitude" (presumably of where the acid would be released), then the designer has followed convention. It is unfortunate that "altitude" is more intuitively put on the vertical axis, but I suspect that defying convention might cause more confusion.

On the other hand, if altitude and cost are not related to each other but two different metrics to evaluate geoengineering concepts, then Bernie's point is right on - swap the axes!

***

The use of connected lines for two of the solutions but not the rest is a symptom of what I have called "loss aversion". The horror of leaving some of the data on the cutting floor.

The only mention of altitude in the article refers to Aurora's assertion that it is sufficient to use newly designed aircraft flying at 20-25 kilometers. If that is Aurora's preferred solution, there is little reason to show all the other altitude configurations that are suboptimal.

Perhaps the designer wants to make the point that the Boeing 747 solution is inferior to the Aurora solution because Aurora could design aircraft to fly at 10-15 km at a lower cost?  If so, then the chart is very misleading in not providing a comparable cost for Boeing's solution if required to fly at 20-25 km.

When comparing different entities, it is always a bad idea to treat the entities differently. Comparison is only possible on equal footing.

In fact, I think the chart would be a lot clearer if they dropped the altitude dimension on the floor. For each solution, plot the yearly cost at the optimal altitude selected by the respective engineers. Use a bar chart. With a single dimension, it is much easier to accommodate the very long data labels.

(Now, I'd defer to the geoengineers as to whether the altitude dimension is dispensable. I don't have any expertise in this science. Judging from the Aurora red line, I'm assuming that there can be feasible solutions at all altitudes, which leads me to conclude that altitude isn't all that.)

***

So what is the biggest problem with this chart? It is the faux accuracy.

Given the tremendous amount of uncertainty surrounding these projected costs, one would expect very big error bars around the cost estimates. Using single dots with no error bars is hard to stomach.

 

 

Stone-age graphic

This Economist chart on the history of world GDP throws the art of graphics back several hundred years. (Thanks Tyler A. for the link.)

And I can't really re-make it since I can't make heads or tails of it.

• How are the columns sorted? (on second thought, maybe the 70 should read 1870, 13 is 1913, and so on?)
• Why are there differing gaps between columns?
• Italy was not a country, and the US was definitely not in existence in AD1 so what does it mean to have values for those on the chart? If this is created by taking current-day boundaries and projecting back in time, why are today's boundaries treated as sacrosanct?
• If the columns are sorted chronologically, a line chart would be much more readable. At the minimum, it will reduce the number of colors to 1. Note that multiple colors are necessitated by the choice of a stacked column chart.
• A stacked column chart with percentages should always extend to 100%. The current chart is very misleading if we want to know the percentage of world GDP produced by "other countries".
• How are the countries ordered within a column? It's neither alphabetical, nor by the starting or ending distributions.
• Don't challenge readers by having vertically stacked categories and a horizontal legend.
• It would also be much better if there are annotations to help the reader understand the chart, e.g. collapse of the Roman Empire, Renaissance, Great Depression, Big Fire, etc.

PS. [8/18/10] Dustin linked to a line-chart version of this chart, from the World Bank site, via Chartporn.

I think the evidence is right here as to why the Economist execution leaves a ton to be desired. The use of lines allows the reader to easily trace the rise and fall of different economies, which is the point of the data set. The stacked-column chart draws attention to a point-in-time distribution of GDP among different countries, which is of secondary importance.

There are other differences: this plots the share of "growth" as opposed to the share of total GDP. It also plots regions rather than countries (well, except for China and Japan). It does not presuppose that the US was in existence before its founding. It could have (should have) included an "rest of the world" line.

The spacing of the years is still problematic but it's an Excel inconvenience, really. But it's ok to stretch the axis on a line chart, it's a problem to do it with a column chart, as demonstrated above. The gaps between columns should be proportional to the years between the data but this is impossible to do in a column chart.

Playthings in an unreal world

It's been said that the economic models used by many mainstream economists this decade suffered from a fatal flaw: that of many unrealistic assumptions (such as no speculative bubbles) that are often needed to "make the math work"; sometimes, these are not direct assumptions but consequences of other assumptions.  See for example Willem Buiter, Paul Krugman, Scientific American.

Another favorite plaything of economists is the so-called "Big Mac index".  The Economist magazine, which seems to own this toy, proclaims it "the most accurate financial indicator to be based on a fast-food item", and the sub-title of the page is "Exchange-rate theory".  It is the cost in US$ of a Big Mac overseas divided by the cost of a Big Mac in the U.S., and is an indicator of whether the current foreign exchange rate (vs. US$) is over-valued or under-valued.

I saw this chart on the Business Insider site, where a claim is made about the Chinese currency being undervalued by 50%.

Sadly, the Economist has gone the way of USA Today in embellishing its graphics with distracting, loud, uninformative images.  Besides the chartjunk, we should always place the zero line in the middle of the chart for this sort of scale where the data could theoretically lean in either direction.  This allows readers to mentally judge the magnitude of the differentials. As pointed out by zbicylist, this statement isn't appropriate to this particular scale.  The scale chosen has the peculiar range of between -100% and +infinity: in order to help readers appreciate this, I would set the left edge of the scale to -100%, and let the right edge expand to cover the actual data.

Food for thought: could it be that under this economic "theory", the US$ can never be over- or under-valued, that it is always correctly valued?

Dealing with skew

Bernard L pointed us to this income distribution chart printed in the Economist.

The accompanying paragraph points to the range of the bars, that is, the gap between the top decile average and the bottom decile average, as evidence of income disparity, concluding that the US and Britain are among the worst.

Bernard likes the use of vertical sections to represent the average incomes by decile and dislikes the USA-Today style background image.  Agreed.  But why plot the middle deciles at all when the only worthy data involve the endpoints of the bars?

A close examination of the spacing of the middle deciles leads to more befuddlement.  There does not appear to be much difference between the countries.

The answer to this is that decile statistics are not appropriate for data as skewed as incomes.  At the high end, the 10% intervals are too coarse.

One clue to this is that the top 10% in the US only earns $90,000 on average but we have all heard of the billion-dollar hedge fund managers and Wall Street bankers and $30 million a movie celebrities.  The problem is that within the top decile, the income distribution is also tremendously skewed. 

The neat idea of plotting the vertical sections indicates an awareness that the red dots (average income) are insufficient because of the skew.  Alas, there remains a lot of skew above the top decile and the designer inadvertently falls back into the same trap by considering the average income within the top 10%.  Thus, the amount of disparity on the right side of the chart is grossly underestimated.  Roughly speaking, we are looking at 10 samples of the distribution, nine of which at the low end of the range and only one at the top end (long tail).  Here is the idea:

Reference: "Spreading the wealth", Economist, Oct 21 2008.