Junk Charts

Reading

Here are some things I have been reading while I'm traveling (the posting schedule will be erratic):

Does the vaccine matter?  Shannon Brownlee and Jeanne Lenzer investigates for The Atlantic.  About 100 million Americans get the flu shot each year; what benefit does it confer?  This is an excellent article.

Some provocative quotes:

Flu comes and goes with the seasons, and often it does not kill people directly, but rather contributes to death by making the body more susceptible to secondary infections like pneumonia or bronchitis. For this reason, researchers studying the impact of flu vaccination typically look at deaths from all causes during flu season, and compare the vaccinated and unvaccinated populations.

The estimate of 50 percent mortality reduction is based on “cohort studies,” which compare death rates in large groups, or cohorts, of people who choose to be vaccinated, against death rates in groups who don’t. But people who choose to be vaccinated may differ in many important respects from people who go unvaccinated—and those differences can influence the chance of death during flu season. [Ed: people who can afford the flu shot vs. those who can't; people who are more health-conscious vs. those who aren't, etc.]

“For a vaccine to reduce mortality by 50 percent and up to 90 percent in some studies means it has to prevent deaths not just from influenza, but also from falls, fires, heart disease, strokes, and car accidents. That’s not a vaccine, that’s a miracle.”

In the flu-vaccine world, Jefferson’s call for placebo-controlled studies is considered so radical that even some of his fellow skeptics oppose it. ... “It is considered unethical to do trials in populations that are recommended to have vaccine,” a stance that is shared by everybody from the CDC’s Nancy Cox to Anthony Fauci at the NIH. They feel strongly that vaccine has been shown to be effective and that a sham vaccine would put test subjects at unnecessary risk of getting a serious case of the flu.

Another pie chart, Fox News (via FlowingData, thanks to reader Katherine M for the pointer; also via Wonkette and reader omegatron).

Yet another pie chart, Business Insider.  Not on the same scale as the one above but still why?

Clean Water Act Violations, New York Times. Can we trust tap water?  As usual, a set of small bars would work better than concentric circles.

How does your state compare to California? (via Pew and Mother Jones) This is a nice illustration that often it is better to plot data derived from the raw data, as opposed to the raw data itself.  Since the designer decided to hide the information, let's figure out what were the cut-off points for the color categories.  If the size of each category is not the same, the designer needs to explain the scale.  Also, the two shades of light blue are hard to tell apart.  But all in all, a good effort here.

A tale of four charts

Speaking of rules for making charts, I think the most important is "if at first you don't succeed, try, try and try again."  It's absolutely essential to produce multiple looks before settling on the one that helps tell the story.

While researching U.S. consumer credit this week, I came across these four views of presumably the same data set.

The chart on the top left (via Business Insider) shows a downward sloping line, with a steep decline on the right edge of the chart.  The authors clearly wanted to show us that consumer credit in the U.S. was collapsing.  The bottom was falling out.  To aid in this effort, they chose to:

• start the plot at the peak of the time series (2000);
• set the minimum value of the vertical scale to coincide with the lowest value attained thus far (2009) -- we have reached rock bottom, ladies and gentlemen; and
• pay no particular attention to the 0% line, which is placed in the lower half of the chart, rather than the middle, thus obscuring the fact that credit grew at faster rates during the height of the boom than it has been declining during the recession.

(Thanks to Excel's default setting, we are treated to a centipede effect on the horizontal axis.  Also thick lines and line shadows.)

Readers who previously complained about my willingness to draw a line through things that shouldn't be connected will be none too amused by the use of a line chart to plot year-on-year changes.  Thus, the zig-zagging of the line represented the change of change, which has been dubbed the "second derivative" during a period when optimists used technical wizardry to show us "green shoots".  To read this chart properly, one should focus on the actual annual decline (the dots) rather than the line.

If one takes a longer-term view, as in the chart on the top right (via Rolfe Winkler), the recent drop in consumer credit can be put in perspective.  Since the 1960s, consumer credit has been positively exploding with few years of decline.  Note, again, that the falling line between 2000 and 2008 represented a slowing rate of growth, not a decline.  The question to ask is: after so many years of almost continuous growth, is the current correction such a big cause for alarm?

This situation of exploding growth followed by a slight correction is better visualized in the third chart (lower left, via chartingtheeconomy.com)  The author also answered a question asked by Rolfe, which is to compare the total consumer credit to the population, as he plotted the per-capita consumer credit.  An eyeball estimate tells us that consumer credit jumped by more than 800 percent since the 1970s, and the current retrenchment is a blip if we take a long view.  The explosion in consumer credit has no doubt enhanced wellbeing in the U.S. for decades; even though credit might well have been over-extended in the recent past, it is far from something evil.

The final chart (bottom right, via The Big Picture) should carry a health warning.  It really should not be shown by itself, or without comment.  Both charts on the right, in fact, came from the same presentation, by David Rosenberg.  This chart, not surprisingly, is the most dramatic -- the chart designer is going for the jugular.  The decline on the right side is much more exaggerated than in any of the other three.  The trick here is:

• to plot the dollar value of credit change, rather than percentage value, and thus ensuring that the further back the time, the more insignificant the change;
• to connect dots which bring out the steep decline when in fact, the steep drop reflected the second derivative; and
  
• to put the 0% line below the middle of the chart, which causes the bottom "half" of the chart to be smaller than the top "half", which plays with our perception so that we may not realize that there were multiple years of growth above the absolute level of decline recently experienced.
  

If this chart were to be believed, our focus should not be on the cliff-diving at the far right -- instead, the chart has the hallmark of a system getting completely out of control, and oscillating to oblivion.  One hopes that is not the message intended by its creator.

For anyone creating charts, it would be a great idea to have attempted all of these versions, and more.

Extra reading

The New York Fed has put up some impressive infographics, depicting the credit conditions around the country.

(I was going to praise the producers for this effort - a rarity among government sites but when I went back to the site today, I find that the graphic is loading only occasionally, and try as I might, I could not get rid of the big brown patch that obscures the West coast.  Hope you have better luck!)

As with similar maps, the unresolved problem is that the pattern is strongly dominated by the relative density of people around the country.  As we click through different maps, they all look similar.

But this map -- as do most infographics -- does well to put structure onto unwieldy, large data sets.  It acts as a table of contents to the data set.

Nick Rapp's team at AP produced this great chart on US life expectancy:

Very sensitive axis labels, putting the line labels into the chart (rather than in a legend box far away), showing only the current data, not every data point, subtle coloring to bring out the decades, etc.

I'm not sure about varying the thickness and tinge of the lines.  Is it necessary?  It hinders a bit as we try to compare the slopes of the lines.  Adding the initial data labels (for 1975) would also help us judge the net change over the period depicted.  If it is the change in life expectancy over time that is the main story, consider indicing all lines to the respective 1975 value.

Reference: "CDC says life expectancy in the US is up, deaths not", Miami Herald, Aug 19 2009. 

PS. This is really an awful title for the article.  It reads as if something bad has happened to the death rate while something good happened to the life expectancy, as if there is a paradox but the article makes clear that the death rate declined, which would make the point redundant.

A world full of bubbles

As promised, we stick to bubbles.  Like the street artist blowing soap bubbles at passers-by, this map -- published in the Guardian (UK) -- is a gift of bubbles.

And our reader Frederic M. is not amused.  "A tremendous failure", he said.

In terms of conveying the data, a simple bar chart would do a better job in exposing the biggest polluters, as well as the relative magnitude between the biggies and the small fish.

The chart reveals more problems if one clicks on, say, Europe, and sees the following:

For starters, compare the bubbles labeled 858, 468, 586, 418 with those labeled 23, 20, 18.  And look at the little ones in the periphery labeled 133, 174, 128.  Baffling, isn't it?

What they did was to print the ranks for every country, except the top four in Europe for which the ranks are placed next to the country name (in small font), and the actual amounts are placed in the middle of the bubbles.  The ranks, of course, are pretty useless, and they obliterate the scale of the differences between countries.

Besides, the bigger the polluter, the smaller the rank but the larger the bubble.  This built-in disconnect can also be disorienting.

Every bubble chart typically contains lots of data labels, and the reason is that the bubble form lacks self-sufficiency.  Without the data labels, the reader has trouble comparing the areas.

Reference: "The Carbon Atlas", Guardian, Dec 9 2008.

Space and time

When it comes to space or time in graphics, old habits die hard.   When we have spatial data, the default is to put it on a map.  When we have a time series, the default is to plot time along the horizontal axis.  Sometimes, these defaults work; other times, breaking up the map or straight-time-line works better.

Thanks to a reader, I noticed that Google put up a "Flu Trends" website to help us track the flu season.  They use two main charts to plot the data, as shown below.




On the right side is the time series, showing the severity of flu cases from month to month.  There are many great things about this chart and one serious flaw.  I love the fact that they did not plot time on the horizontal axis; they realize the seasonality and they create overlapping lines.  They make good use of foreground and background; it's easy for us to compare year to year differences.

The serious flaw: no vertical scale.  This was a problem with Google Trends from day one (see my post here).  They still haven't fixed it.  Because of this, we don't know if the peak shown was 5 cases or 5000 cases.  While for Google key word searches, one can excuse them for trying to protect commercial secrets.  I would imagine that this public health data is, well, public.  Since the apparent purpose of this chart is to allow citizens to declare a flu epidemic (say, when they see the current trend depart from the historical norm), not having the scale is a huge problem.

I also disagree with shifting the months around for the Northern Hemisphere so that the peaks of the graphs are aligned towards the middle.  It is better for the peaks to appear on the left and let the order of the months conform to our expectation.  (The "peak" would be split on the sides and the chart would look like a valley, which presumably is why they did it this way.)

The charts on the left side plot the spatial data, not surprisingly on maps.  Sadly, the standard exhibited on the time-series charts is nowhere found on these maps.

First, the legend is seriously deficient.

Second, the gradation of the colors is not fine enough, or put differently, the aggregation to the state/province level is much too coarse for any interesting pattern to be seen.

This poorly aggregated map becomes a farce when applied to the U.S.  There is not much left to be said, is there? 

Reading comprehension

Note: I am in the middle of a holiday and so posting will be limited.

Andrew posted a pretty chart that caught my attention.  This is the sort of sophisticated chart that rewards careful reading. 

Below is a guide to reading the chart:

• It is a small multiples chart with the components arranged in two dimensions (income levels, and a race-religion hybrid category).  The top row is a summary of voters of all race-religion grouped by income.  Note that there is no corresponding summary column for voters of all incomes grouped by race-religion.
• Source of data: 2000 poll but applied to 2008 demographic patterns.  In other words, there is an underlying assumption that opinions have stayed stable within the demographic groups.
  
• The chart is in fact three dimensional because each map gives us the geographical (state by state) breakdown.
• It is useful to figure out the smallest unit of data: in this case, this is the percentage support of federal school vouchers by voters of a given race-religion-income-and-state category.
• The color scheme is such that red represents highest support and blue lowest support, with pink and purple in the middle
• It's almost always better to start from the aggregate (that is, the average) and then study variations along different dimensions, and this is how the chart is arranged from top to bottom
  
• On the top row, the higher income groups tended to favor vouchers more than lower income groups, with a break point around $75k; even here, the regional differences are significant, with northeast and southwest hotter for vouchers at all income levels
• As we move from row to row, we realize that the aggregate data hides many disparities.  For example, white Catholics (second row) are more likely to support vouchers regardless of income level while white non-evangelical Protestants (fourth row) are much less likely than average to support vouchers at all income levels.
• Notice that the statistician (Andrew) has carefully defined the race-religion categories to balance between collapsing subgroups that are distinct and showing too many subgroups so as to cloud the patterns.  That is why there are many more race-religion subgroups that are not shown.  The ones shown are of special interest.  Consider the white protestants, evangelical vs. non-evangelical (third and fourth rows).  If one were to fix the race, geography and income dimensions, and even fix half of the religion dimension, we still find the two subgroups to be on different ends of the spectrum relating to the voucher issue.  This is why the evangelical or not dimension has been included.
  
• The white space is interesting.  Here, the issue faced by the statistician is sparse data when one gets down to multi-dimensional subgroups.  Andrew chose to ignore all the data, which is the wise thing to do.  With so few samples, it is particularly easy to draw bad conclusions.   
• Because of the white space, we get additional information on the spatial distribution of the demographic subgroups.  The black population (at least the voters) are predominantly found in the southeast while Hispanics are in the southwest.  The subgroup of income higher than $150k is essentially all white.  Admittedly, this is a very crude read because we only have two levels (below 2% of state population and above).  Of the colored states, we cannot differentiate between densely populated and not.

  

Such rich graphics deserve careful reading.  Enjoy!
  

The case of the shrinking mall

I had a similar reaction when reading this chart but I will let our reader take center stage.

Top portion

Bottom portion

The back of today's New York Times' Week in Review section devotes half of its space to a lame infographic that wastes space and has a major error (which has been corrected lazily in the version now online at http://www.nytimes.com/interactive/2008/10/14/opinion/20090531_OPCHART.html ) The chart shows the recent decline (or in some cases, rise) of retail sales at 27 common mall chains. My main objection is that the top half of the chart is useless. Yes, it provides a baseline from which shrinkage in area (visual metaphor of income = floor space) in the bottom half can represent the relative declines in sales, but this is redundantly handled better by color. The only part of the chart that I got any information from was the bottom half, and it took me a while before I figured out why the top half was even there. Meanwhile, the error in the printed version is that the +5-10% stores were colored light green while the +0-5% store (Burger King) was colored dark green. It should have been vice-versa. The online version simply swapped the colors in the legend, rather than on the map itself, which works logically but begs the question: why do you have dark red at one end of the spectrum and light green at the other, with dark green in the middle? Thank you for listening-- just blowing off a little steam here. It's a lot of wasted space and I'll bet the New York Times paid a lot for it.

Reference: "Op-Chart: The Fall of the Mall", New York Times. (Ed: I am not sure why the date is given as October 2008.)

Colorful maps

Bernard L. loved the recent NYT take on immigration in America.

The very pretty maps are found here.

An amazing amount of data is being visualized here.  Mousing on the map will pick up the specific data for each county.  There is a bar up top for discovering the evolution over time.  It would be great if there is an animation button so the map can be played out without clicking.  An animated gif will also do (similar to the disease map we featured some time ago).

The colors on the first map represent the origin of the top ethnic group in each county.  Within each group, the tint of the color further displays the percentage of the population that group accounts for.  The subgroups appear to be 0-2%, 2-5%, over 5%.  The last subgroup is very wide.

Not so keen on the second map with all those bubbles.  They show the number of people from each country by county.  The bubble size is proportional to population.  Every version of this map looks the same because the population is concentrated in the cities and the interior is sparsely populated, no matter what ethnic group.

Regardless, this is another laudable effort by the crew at the Times.

Reference: "Immigration Explorer", New York Times, March 10 2009.

The trouble with maps

Todd B. pointed us here.  These are maps that supposedly show the distribution of respondents for each answer choice in a survey exploring accents in different parts of the country.  The full set of maps for every question can be found here.  

Amusingly, the researchers also provided a map of "all respondents".  (I won't ask how the proportions of respondents were reduced to binary output to produce the above maps.)

Here is Todd to lead off the discussion:

Just because you put data on a map doesn't make it effective. Check out these mapped responses that either tell us that there is no difference in dialects or fails to illustrate differences effectively.

Reference: "Dialect Survey", University of Wisconsin-Madison.

How to read a graph

Via Gelman, here is a nifty book-buying map from Amazon, displaying the split between "red books" and "blue books" bought by Amazon users in each state in the months leading up to the 2004 and 2008 presidential elections.



Gelman noted the similarity between the Amazon map and the red-blue split of rich voters.

This post is about how to read a graph.  Here are some things that come to mind looking at the map:

• Sampling bias: how does Amazon's customer base compare with the U.S. population, or rich voters?  It would be prudent to check this before making generalizations.  Gelman's point may be that Amazon customers behave like rich voters.

• Sampling period: is the period long enough to capture the average inclination of the book buyers?  As is well known, book sales follow a long-tail distribution (Chris Anderson wrote an entire book based on this observation.)  Best-sellers have a disproportionate influence on average values.  If the time period is too short, the data may only represent the best-sellers.  Consider the following two maps in successive periods in 2004:

Much of the red in the first map was due to John O'Neill's "Unfit for Command", published in August 2004, and much of the blue in the second map was due to John Dean's "Worse Than Watergate", published in April 2004.  If one of these two-month periods was used to draw conclusions, we would make big mistakes!

• Classification: The long-tailed nature of book sales has wide-reaching implications on interpreting the data.  The most essential feature is that single books (bestsellers) have a disproportionate impact on average sales.  Since the key metric here is proportion of red (or blue) books, it follows that whether a best-seller is classified as red or blue makes a huge difference. 
  

Thus, one of the first things to look at is Amazon's helpful explanation of how they classified books as "red" or "blue".  We learn that they also have "purple" books which are those they could not decide if it's red or blue.  Each red or blue book is given equal weight but it appears that purple books are not tallied.  Glancing at the list of purple books, I see some hugely important books, e.g. Ron Paul's "The Revolution: A Manifesto" (Amazon rank #56  among all books), Tom Friedman's "Hot, Flat and Crowded" (#15).

If the purple books include best-sellers, then the decision to call it purple rather than red or blue causes an influential book to be excluded from the calculation.  We often forget that the decision to exclude is not a neutral decision; it is an active decision that says the excluded data contains no useful information.
 
This is not to say that excluding those books is the wrong decision.  We must make these decisions with considerable care, and realize that excluding best-sellers when book sales have a long-tailed distribution must not be taken lightly.

• Causality: Lets say we are sufficiently satisfied that we can make a statement about book buying habits and voting behavior.  Then we need to think about the direction of causality.  Is the map saying that red book buyers are likely to vote red?  Or that red voters are likely to buy red books?  No prolonged staring at this data set will resolve this issue as other data would be needed to address it.

The more data is used to create a graph, the harder our task is to interpret it.  But the pay-off for spending the time is all the sweeter.  Happy graph-reading!

One final note: there is no doubt that this interactive map feature is a brilliant marketing move by Amazon.  This is a great and fun way for readers to find interesting books.

Reference: "Amazon, U.S.A.", Gelman blog, Oct 5 2008.