Junk Charts

Reading

Here are some of my favorite links from other places:

A spatial journey illustrating a very long scale, created by the Genetic Science Learning Center (here)

Long scales are very difficult to deal with in charts; I have never been satisfied with log scales since it addresses the designer's challenge of trying to fit everything onto one page, bu does not deal with the reader's need to compare the elements accurately

Not sure how this helps but perhaps some of you will figure it out

Tommi left a comment about this conceptual chart by xkcd, which has been making the rounds.  Fits into our Light Entertainment category.

Says there is no optimal chart type.  A type that works very well for one data set may get hopelessly cluttered for another, similar data set.

From fellow bloggers (especially Jorge), a whole series of views of the U.S. unemployment figures by state over time.  Alternatives that are much more interesting to look at than the typically line chart. Jorge even found something in Excel that looks good.

Following one's nose 2

This is the second post on the immigration paradox study, first discussed on the Gelman blog.  My prior post on the graphing aspect is here; this post focuses on the statistical aspects. I am working backwards on Andrew's discussion points.

Which difference is most interesting?

5. Agree with Andrew; they should publish similar analyses on other minority groups as soon as possible.  One thing that strikes me when looking at the interaction plot is that the U.S. born non-Latino whites have a much higher incidence of mental illness.  The difference between different subgroups of Latinos paled in comparison to the difference between non-Latinos and the Latinos.  This latter difference is particularly acute among the U.S. born than the immigrants. The importance of the Latino analysis hinges upon whether the "paradox" is also found among other minority groups.

(Chris P also pointed this out in his comment on the previous post.)

Disaggregation, Practical Significance, and the Meaning of Not Significant

2. Andrew is also right in expressing moderate skepticism about this sort of disaggregation exercise.  He connects this to the subtle statistical point that "the difference between significant and not significant is not significant."  A related but less obtruse issue is that as one disaggregates any data, the chance of seeing variations that stray from the average gets higher and higher.  This is because the sample size is decreasing, and so the statistical estimates are less reliable.

(To give a flavor of the scale, there were a total of 2500 Latinos in the sample, with 500 Puerto Rican Latinos. The analysis drilled down to the level of different types of mental disorders, subgroups of Latinos, and also adjusted for demographics.  The details of the demographic adjustment are not available but in any case, one should be concerned about whether there were sufficient numbers of say, male immigrant Puerto Rican Latinos age 18-25 with income < $10,000 living in a rental apartment, for such an elaborate exercise.)

Expanding on this point further, one observes that the measured gap between U.S. born and immigrant Puerto Rican Latinos was about 5%.  But this 5% is probably of considerable practical significance since the base rate of incidence is about 30% (I say probably since I am not an expert in mental illness).  The current statistical analysis judged this to be insignificant -- if the sample size were larger, this difference could conceivably be statistically significant, and also practically significant.

But, doesn't the significance test deal with the small sample size problem?  Yes, if the authors merely described the Puerto Rico result as inconclusive.  Here, as is done very commonly, insignificance is equated to "no difference": they said

No differences were found in lifetime prevalence rates between migrant and U.S.-born Puerto Rican subjects.

In reality, a difference of 5% was found in the sample that was analyzed.  The statistical procedure found that this difference could have been a result of chance -- notice "could", not "must".  If the measured difference was 0.5% on 30%, then I might be willing to accept a finding of "no difference"; when it was 5% on 30%, I would like to see a larger sample analyzed.

The Meaning of Paradox

1. Andrew was perplexed by why the phenomenon is known as a "paradox". I had the same issue until I read the paper. The authors were a bit sloppy in the abstract. In the paper itself, they explained that the conventional wisdom has it that immigrants should be more likely to have mental illness because of the stress from the immigration process, and yet the statistics showed the exact opposite. That is the paradox.

Publication Bias

I was a little shocked to see the data tables that gave all the estimates of the various effects at the various subgroup levels: shocked because the authors were allowed (or asked) to include only the p-values that were below some unspecified level (which I surmised is 10% although a 5% significance level is used to judge significance as per convention). This is publication bias within publication bias. P-values that are not significant still provide valuable information and should not be omitted. They did provide confidence intervals but for each subgroup separately, rather than for the difference -- and as they noted, such intervals by themselves are inconclusive when they overlap moderately.

Getting the Nobel wrong

Why do we pay so much attention to seemingly inconsequential details of a chart's scale, colors, labels, etc.?

Here's why. (Reader Omegatron pointed us to the FAIL blog that captured this beauty.)

Notice the messed-up horizontal scale, in particular, the failure to start the axis at 0.  The result: the tiniest difference presented as a wide gulf.

The graph, published by the Washington Post, has since been fixed.  See here.  Nevertheless, the comments left by readers lent witness to the confusion.  I copied the first bunch that mentioned the graphical display - there were plenty of for-and-against-the-prize comments, many assuming that the poll result was as lop-sided as the chart seemed to indicate.

By starting at a base of 45% (as of this reading), your graphic grossly misrepresents the results of your poll. The "no" bar is four times as big as the "yes" bar, giving the visual impression that the vote must have gone 80-20 against Obama's Nobel. On the contrary, as of now the vote is 53-47 against. Whoever produced this graphic should re-read Edward Tufte's "The Visual Display of Quantitative Information."

Posted by: threefab | October 9, 2009 8:33 AM

Someone made comment that the graph is misleading, but it really isn't if you know how to read a graph.

This type of graph highlights the difference, not the complete number of votes and is appropriate when viewing percentages.

Posted by: gconrads | October 9, 2009 8:39 AM

why is graph not drawn to scale? The vote is 51 - 49 no and the graph looks like an overwhelming number of "voters' said no

Posted by: spitts1 | October 9, 2009 8:41 AM

The graph is purposely designed to make it appear that there are a huge number of no votes and demonstrates obvious bias.

Posted by: fingersfly | October 9, 2009 8:41 AM

I'm looking at the graphic here and cannot figure out what you guys are trying to show?
The tiny slice of blue is supposed to be 50% and the huge slice of red is supposed to be 50% and the scale at the bottom is......
WHAT?

Posted by: Tomcat3 | October 9, 2009 8:53 AM

It is my understanding that the President did not "win", but was "awarded" the prize...much like a bonus given to you from your employer...deserving or not. In addition, I could not help but notice that after casting my vote, the graphic scale indicated 49% yes and 51% no; however, the graph lines seemed much more disproportionate for only a 2% spread with emphasis on “no”…

Posted by: jonbwnfd | October 9, 2009 9:40 AM

Why does the graph make the vote look so lopsided?

Posted by: subwayguy | October 9, 2009 9:48 AM

The graph is not lopsided .. the numbers are. Read the posts and you will see the posts are against the idea not for.

Posted by: T-Tom | October 9, 2009 9:53 AM 

Who designed the bar graph? The difference between the "yes" votes and the "no" votes is two percentage points, yet the "no" bar is three times as long as the "yes" bar.

In fact, "yes" is 49% and "no" is 51%: the graph is supposed to illustrate reality, not. . . well, just what DOES it illustrate?

Posted by: cmcintyr | October 9, 2009 10:02 AM

Inconsequential detail?  Big fail?  You decide.

A tale of four charts

Speaking of rules for making charts, I think the most important is "if at first you don't succeed, try, try and try again."  It's absolutely essential to produce multiple looks before settling on the one that helps tell the story.

While researching U.S. consumer credit this week, I came across these four views of presumably the same data set.

The chart on the top left (via Business Insider) shows a downward sloping line, with a steep decline on the right edge of the chart.  The authors clearly wanted to show us that consumer credit in the U.S. was collapsing.  The bottom was falling out.  To aid in this effort, they chose to:

• start the plot at the peak of the time series (2000);
• set the minimum value of the vertical scale to coincide with the lowest value attained thus far (2009) -- we have reached rock bottom, ladies and gentlemen; and
• pay no particular attention to the 0% line, which is placed in the lower half of the chart, rather than the middle, thus obscuring the fact that credit grew at faster rates during the height of the boom than it has been declining during the recession.

(Thanks to Excel's default setting, we are treated to a centipede effect on the horizontal axis.  Also thick lines and line shadows.)

Readers who previously complained about my willingness to draw a line through things that shouldn't be connected will be none too amused by the use of a line chart to plot year-on-year changes.  Thus, the zig-zagging of the line represented the change of change, which has been dubbed the "second derivative" during a period when optimists used technical wizardry to show us "green shoots".  To read this chart properly, one should focus on the actual annual decline (the dots) rather than the line.

If one takes a longer-term view, as in the chart on the top right (via Rolfe Winkler), the recent drop in consumer credit can be put in perspective.  Since the 1960s, consumer credit has been positively exploding with few years of decline.  Note, again, that the falling line between 2000 and 2008 represented a slowing rate of growth, not a decline.  The question to ask is: after so many years of almost continuous growth, is the current correction such a big cause for alarm?

This situation of exploding growth followed by a slight correction is better visualized in the third chart (lower left, via chartingtheeconomy.com)  The author also answered a question asked by Rolfe, which is to compare the total consumer credit to the population, as he plotted the per-capita consumer credit.  An eyeball estimate tells us that consumer credit jumped by more than 800 percent since the 1970s, and the current retrenchment is a blip if we take a long view.  The explosion in consumer credit has no doubt enhanced wellbeing in the U.S. for decades; even though credit might well have been over-extended in the recent past, it is far from something evil.

The final chart (bottom right, via The Big Picture) should carry a health warning.  It really should not be shown by itself, or without comment.  Both charts on the right, in fact, came from the same presentation, by David Rosenberg.  This chart, not surprisingly, is the most dramatic -- the chart designer is going for the jugular.  The decline on the right side is much more exaggerated than in any of the other three.  The trick here is:

• to plot the dollar value of credit change, rather than percentage value, and thus ensuring that the further back the time, the more insignificant the change;
• to connect dots which bring out the steep decline when in fact, the steep drop reflected the second derivative; and
  
• to put the 0% line below the middle of the chart, which causes the bottom "half" of the chart to be smaller than the top "half", which plays with our perception so that we may not realize that there were multiple years of growth above the absolute level of decline recently experienced.
  

If this chart were to be believed, our focus should not be on the cliff-diving at the far right -- instead, the chart has the hallmark of a system getting completely out of control, and oscillating to oblivion.  One hopes that is not the message intended by its creator.

For anyone creating charts, it would be a great idea to have attempted all of these versions, and more.

Lining them up

Consider this simple and effective display accompanying the NYT article titled "Recession Drives Women Back to the Work Force".  This is somewhat surprising because jobs are harder to come by during a recession. 

The journalist also told us that the interpretation of this data is controversial among economists.  The trend seemed to have been a decline in participation till 2004 and then a rise since then for 25-to-34-year-old women.  If there were an up-trend, it appeared before the start of this recession.  And participation rate dropped after the 2001 recession.  So much is still under study.

The use of small multiples is a good idea, and the color coordination between the two charts is well thought out.  A minor miss is the vertical scale: for any small multiples chart, one should always use the same scale.  In this case, using the same scale will allow readers to compare the levels of participation between men and women.

The following chart plots a related data set, focusing on the difference between men and women, among those married.  As before, the participation rates are of very different levels.  The male rate has steadily declined in recent decades while the female rate rose from the 30s in the 60s to about 70% in the 90s. 

Reference: "Recession Drives Women Back to the Work Force", New York Times, September 19, 2009; Bureau of Labor Statistics.

A world full of bubbles

As promised, we stick to bubbles.  Like the street artist blowing soap bubbles at passers-by, this map -- published in the Guardian (UK) -- is a gift of bubbles.

And our reader Frederic M. is not amused.  "A tremendous failure", he said.

In terms of conveying the data, a simple bar chart would do a better job in exposing the biggest polluters, as well as the relative magnitude between the biggies and the small fish.

The chart reveals more problems if one clicks on, say, Europe, and sees the following:

For starters, compare the bubbles labeled 858, 468, 586, 418 with those labeled 23, 20, 18.  And look at the little ones in the periphery labeled 133, 174, 128.  Baffling, isn't it?

What they did was to print the ranks for every country, except the top four in Europe for which the ranks are placed next to the country name (in small font), and the actual amounts are placed in the middle of the bubbles.  The ranks, of course, are pretty useless, and they obliterate the scale of the differences between countries.

Besides, the bigger the polluter, the smaller the rank but the larger the bubble.  This built-in disconnect can also be disorienting.

Every bubble chart typically contains lots of data labels, and the reason is that the bubble form lacks self-sufficiency.  Without the data labels, the reader has trouble comparing the areas.

Reference: "The Carbon Atlas", Guardian, Dec 9 2008.

Pie Cubed

Omegatron also did away with a set of cascading pie charts on Wikipedia, a particularly ineffective use of this chart type.  Whenever there are more than two or three categories, the necessary use of many colors can really make one's head spin.

Here, the cascade is being used like a log scale, to artificially elevate the small pieces, which unfortunately are also the least significant pieces of the energy pie.  There is no reason for nuclear, bio-mass, hydro and "others" to add to 100% except that the author decides to group them together.  The 41% nuclear or the 41% solar heating in the second and third charts, respectively, have no meaning in the larger context.

In deference to the original author, Omegatron's new version preserves the arbitrary three-level cascade.  He converts to stacked bar charts, which brings out the differences better.


He also sensibly exposes the original proportions rather than the arbitrary relative proportions.  For example, nuclear energy accounts for 6% of the total, not 41% of the arbitrary "others" bucket which in turn contains 14% of the total.

I'd prefer an even cleaner presentation with unstacked bar charts.  This can be done in either one chart with all eleven categories, or in two charts, as shown below.  The two-chart version assumes that the reader have two key questions: alternative energy sources as a proportion of the total, and the mix of different sources within the alternative category.

With the ordinary bar chart, many fewer colors are needed, and there is no need to print out each data point, nor a need to use guides to point to labels and data.  The trouble of the latter is its tendency to draw attention to the least important aspects of the data.

With this further example, I continue to find the Wikipedia rule to discourage text annotations on graphs bewildering.   Such a rule apparently does not apply to data labels, as can be seen here.  Of course, a graph without any labeling of categories is robbed of meaning but if labels can be saved, so should annotations!

Community outreach

Omegatron re-cycled some Wiki charts and we are happy to report that they are great improvements over the originals.  I welcome other readers to alert us when you have done your bit of community outreach, by ridding the world of chartjunk.  The email address is the name of the blog at gmail for any submissions.

Not surprisingly, the originals were pie charts.

The recycled art: (can be seen here and here)

Why are pie charts such poor tools for communications?  Think about where we can place the message, and you'll find that this chart type is far too rigid.  In a pie chart, the key resource is the relative size of the sectors, followed by the number of sectors, and sometimes the size of the total pie.  Other than those, there are little else useful in a pie chart.

The histograms used by Omegatron are much more flexible.  There can be information encoded in the height of the bars, the width of the bars, the total area, the relative distribution of bar areas, the existence and location of peaks and troughs, etc. etc.

For comparing data collected in slightly different formats, the pie charts are hopeless.  Notice that the lowest category on the left (pink) corresponds to 8 weeks or less, which would include two and a half sectors on the right plus potentially a missing sector for 4 weeks or less.  The histograms below handle this easily.

Omegatron asked for some feedback.  I think the new ones are significantly better.  A few minor points:

• Instead of coloring the background to the chart, I'd color the bars themselves into green/yellow/orange according to the trimester
• I'd put the trimester labels under the horizontal axis, close to the "week" labels 
• The charts obviously need to identify the country and year of the data (which I added).  Omegatron pointed me to an inexplicable Wiki convention of not putting text inside charts (see here).  I must disagree with this convention.  Annotations on charts are some of the most useful things.
• If these two charts are to be placed side by side for comparison, then we need to sort out the vertical scale.  It cannot be the absolute number of abortions but some kind of relative scale in proportion to the population size, or some similar metric. 
• In addition, if comparison is the point, I'd suggest an overlapping histogram with bars having no fill.  

Great work!  And I love to see more of it! 

Bubble after bubble

I finally checked the Junk Charts mailbox again, and I found an uprising against bubble charts and pie charts.   It appears that despite their shortcomings amply demonstrated here and elsewhere, editors everywhere continue to believe that the public has a lovefest with these creatures.

I will start off the parade with this one from the Wall Street Journal, purportedly showing that the Bank of England has continued to inject cash into the economy, and at ever increasing rates.  The headline said Bank of England to expand bond-buy plan.


This chart has a variety of problems, in addition to the use of overlapping bubbles.  As has been documented, it is almost impossible to gauge the relative sizes of circular areas, especially when they are overlapping.

If we remove all but one of the data labels, the chart is non-functional.  This is what we mean by not self-sufficient: the interpretation of this chart requires, indeed demands, that all the underlying data be printed on the same chart.  The only way readers can understand what is going on is by reading the data itself!

The horizontal axis (indicating time) is also non sensical.  The separation from month to month is variable.  Besides, and this is the key flaw of the chart, the projected number is a three-month total cumulative growth being treated like a monthly figure.

 

Since the Bank is projected to inject 175 50 billion extra pounds in the next three months, that would work out to be roughly 60 16 billion per month.  That would turn the story upside down: one would conclude that the Bank is gradually slowing the rate of injection.  The following bar chart points this out with little fuss:

 

When bars are used, there is no need to print every single data point.  The relative lengths of the bars can be estimated easily.  The months are equally spaced.

One final point: the exchange rate cited is not very helpful.  What would have been more useful for readers would be the scale of the cash injection with respect to each nation's GDP.

Reference: "Bank of England Expands Bond-Buy Plan", Wall Street Journal, Aug 7 2009.

PS. Per Andrew's comment, here is a line chart, where the growth/decline in the injection is encoded in the slope of the line segments:

Space and time

When it comes to space or time in graphics, old habits die hard.   When we have spatial data, the default is to put it on a map.  When we have a time series, the default is to plot time along the horizontal axis.  Sometimes, these defaults work; other times, breaking up the map or straight-time-line works better.

Thanks to a reader, I noticed that Google put up a "Flu Trends" website to help us track the flu season.  They use two main charts to plot the data, as shown below.




On the right side is the time series, showing the severity of flu cases from month to month.  There are many great things about this chart and one serious flaw.  I love the fact that they did not plot time on the horizontal axis; they realize the seasonality and they create overlapping lines.  They make good use of foreground and background; it's easy for us to compare year to year differences.

The serious flaw: no vertical scale.  This was a problem with Google Trends from day one (see my post here).  They still haven't fixed it.  Because of this, we don't know if the peak shown was 5 cases or 5000 cases.  While for Google key word searches, one can excuse them for trying to protect commercial secrets.  I would imagine that this public health data is, well, public.  Since the apparent purpose of this chart is to allow citizens to declare a flu epidemic (say, when they see the current trend depart from the historical norm), not having the scale is a huge problem.

I also disagree with shifting the months around for the Northern Hemisphere so that the peaks of the graphs are aligned towards the middle.  It is better for the peaks to appear on the left and let the order of the months conform to our expectation.  (The "peak" would be split on the sides and the chart would look like a valley, which presumably is why they did it this way.)

The charts on the left side plot the spatial data, not surprisingly on maps.  Sadly, the standard exhibited on the time-series charts is nowhere found on these maps.

First, the legend is seriously deficient.

Second, the gradation of the colors is not fine enough, or put differently, the aggregation to the state/province level is much too coarse for any interesting pattern to be seen.

This poorly aggregated map becomes a farce when applied to the U.S.  There is not much left to be said, is there?