Junk Charts

Trying too hard

In the course of business and governing, a lot of charts are generated.  An anonymous tipster pointed us to a set created by the "Communities and Local Government" division in the UK government.  Judging from the content, this division has responsibility for economic development in local neighborhoods.

Below are a pair of exhibits.  Truly they are trying too hard!  What we see is a hybrid scatter-bubble chart.  Between the jargon, the acronyms (LAD, LSOA), the boxed text, the multi-color circles, the colored axis labels and lack of title, the reader is plunged into a state of confusion.

The chart can be unraveled.  Each district was evaluated based on two measures of "gaps in worklessness".  The vertical axis compares each district to the national average; positive numbers indicate an above-average district relative to the nation.  The horizontal axis compares the most deprived 10% neighborhood within each district to the local average; positive numbers indicate worst neighborhoods improving. 

Thus, the policy goal would be to move all districts into the upper right quadrant.  The multi-color bubbles were designed to show us the state of the nation.  On the left chart, 41% of the districts (or population?) reside in the improving districts while 19% live in deteriorating areas.

The following strategies can help improve readability:

• use English on the axis
• relegate technical definitions to the legend
• add succinct title to tell the story
• use color on the data rather than on axis or data labels
• use color to draw attention to the upper right quadrant
• remove bubbles
• define acronyms

 

Ordering and grouping

The Times reported that January retail sales generally disappointed, and consumers showed a preference for discount retailers over department stores.

Taking the bar chart on the right, re-ordering by change in same-store sales, and grouping companies by type of retailer, we can present the data to match the text more closely.  The divergent performance between discount retailers and department stores is readily visible.

Reference: "Weak January dashed retailers' gift-card hopes", Feb 8 2008.

 

Redundancy

Nick B., who occasionally writes about statistical graphics, found some classic chart junk from a Canadian report on the Afghan army.  Here's one example, together with the junkchart version.

Redundancy is an enemy of good graphics, and incongruous redundancy is worse.  Here, troop level is variously described as "total force size", "strength" and "army growth"; the chart on the right uses only the army concept.  The data labels ("47000 Strength"), the axis labels ("50000 Total Force Size"), and the gridlines all germinate from the five grand data points underlying the entire chart!

Another distorting feature is that use of different-sized time intervals, which we space out appropriately on the right chart.

Ultimately, the key message should be growth in the army size, not the absolute number of troops.  The slopes of the line segments encode this information.  Alternatively, a data table can be rather powerful for simple data like this:

By what is called the "end state", there would be 70% more troops than those as of December 2007.

 

Oscar diseconomy

Business Week dissected the beneficiaries of the Oscar show as shown on the right.  Although this doesn't work well as a data graphic, if thought as a variant on the data table, it is more engaging for readers.

Lets have some fun with the Oscar statue.  First, putting a bar chart next to the statue confirms that the height of the segments (rather than the area) is in proportion to the dollar values (below left).

Tufte, Chambers and others have shown that our eyes react to the areas, not heights.  So next, I estimated the areas but stretched them out into segments of equal width.  Squeezing the entire column back down to the height of the statue, the following chart (below right) puts perceived proportions next to the true proportions, displaying visually the extent of distortion. 

Reference: "News you need to know", Business Week, Jan 28 2008.

Hits and misses 2

In the previous post, we discussed how charts need to address the key question posed by the data.  In this case, the journalist was trying to show that police shots often go errant, and are largely unpredictable even when the distance of the target is given.

In the comments, there is interest in seeing the hit rate v. distance chart.  Because the data came to us in buckets, we do not have enough to continue the analysis.  If one were to guess, the real curve would start out with 100% accuracy at distance 0, fall sharply to a plateau in the 20-40% range at modest distances, and then drop again at large distances, decaying to zero.

Andrew Gelman has conducted this analysis for a similar problem, that of predicting accuracy of golf putts based on distance from the hole.  Here are two key charts from his paper (joint with Deborah Nolan):

The left chart is our hit rate chart above, except the golf data set is larger, allowing a curve fitting.  The right chart is the fitted curve which is a "model" for the true relationship between accuracy and distance from the hole.  The model fitted the data well.

Gelman and Nolan didn't just find any best fitting line through the data.  They started out with a trigonometric model (shown on the right), with the angle of the putt as a random variable.  With this setup, they wrote down the formula for computing the probability that the putt will fall in, that is, the proportion of success.  The angle is assumed to follow a normal distribution with the standard deviation being an unknown parameter.  The standard deviation is estimated from the available data.

Of course, the human body is a bit harder to model than the hole in the ground but this procedure could very well apply.

For more details, check out the paper (PDF).  This example is also found in their book on teaching statistics.

Source: Gelman and Nolan, "A Probability Model for Golf Putting".

Hits and misses

In this NYT article, we are told that "the most likely result when a policeman discharges a gun is that he or she will miss the target completely."  That's a shocker for those of us conditioned by Hollywood movies to think anyone who picks up a gun for the first time hits the villain right on the temple.  The following graphic attempts to tell the story.

The one hit here is how the distances are visually presented.  The elliptical lines remind us of the neglected variable of direction; it also means the scale is correct only along one direction.

The dot matrix construct highlights the absolute numbers of shots, hits and misses but barely addresses the key issue of hit rates (accuracy). Specifically, this data set was presumably collected to explore the relationship between hit rates and distances from the target.  The use of different widths clouds our judgement of proportions.  To wit, it is not obvious that the 10-wide block and the 40-wide block shown left depict roughly equal hit rates (23%, 29%).

The junkart version adopts a different approach.  This is the Lorenz curve, often used to show income inequality (see also here and here).  Here, the shots were ordered from closest to furthest from target, then summed up by distance segments.  For example, shots from 0 to 6 feet accounted for 60% of all shots but 72% of all hits.

If distance does not affect hit rates, we'd expect 60% of all shots to result in 60% of all hits.  This data point would show up on the 45-degree diagonal on the chart, labelled "totally unpredictable".  Any data appearing above the diagonal indicates that closer shots are more accurate, accounting for more than their fair share of hits.

Comparing the fitted blue line and the diagonal, one sees that distance is a weak predictor of hit rate.  The police commissioner explains this in the article; many other variables also affect accuracy, including "the adrenaline flow, the movement of the target, the movement of the shooter, the officer, the lighting conditions, the weather..."

Note that the shots with "unknown" distances were removed from the analysis.  Also, the categories of 21-45 and 45-above were combined: the rates were similar and with only three hits, it does not make sense to treat these as separate categories.

Of course, this version would not work well in the mass media.  For that, one can just plot hit rates against the distance categories.

Source: "A Hail of Bullets, a Heap of Uncertainty", New York Times, Dec 9 2007; New York Firearms Discharge Report 2006.

Digging deeper

Two items from other places caught my eye this week as they directly relate to some things we discussed on this blog.

First, I second Andrew's suggestion of a recent NYT article for teaching the concept of margin of error, or how to read political poll coverage intelligently.  Towards the end of this piece is a small gem:

Some pundits began by saying the horse race numbers were close but then tried to marshal evidence that they were not. On ABC's own Web site, Chris Cillizza, wrote: "Among women in the Post poll, Obama actually leads Clinton 32 percent to 31 percent among women. Voters 45 years of age or older are similarly divided, choosing Clinton by a 27 percent to 26 percent margin over Obama. Ditto for those who earn $50,000 or less a year; 29 percent for Clinton, 29 percent for Obama."

Mr. Cillizza failed to mention that if the margin of sampling error is plus or minus five percentage points for all of the likely Democratic caucus goers, then it is even higher for subgroups like women.

In a recent post, I call this the "oft-used device of subgroup support of a hypothesis".  This example illustrates the fallacy more clearly.  It's the "let dig deeper since we haven't found the gold yet" phenomenon.  Such analysis suffers from two serious statistical problems.  The article deals with the sample size problem: the margin of error at the subgroup level is by definition larger; what this means is the bar for statistical significance has been raised; and rare is the case where such analysis could lead to any further insights.  (Of course, I am assuming the original poll was not designed to be analyzed at the subgroup level.)

The other issue -- more difficult to explain and omitted in the article -- is the multiple hypothesis problem.  It is well known that if we dig around long enough, we may get so dizzy that anything that glitters will look like gold.  In other words, false positives.  Like the sample size problem, the remedy is to raise the bar for statistical significance even higher.  In practice, this frequently wipes out the rationale for such analysis.

I will address the other interesting item in a new post.

The punch line

Mike K submitted this great entry months ago.  It's a map depicting stock market correction across the globe during the summer.  You have to click on the link to the WSJ website in order to see the interactive element.



Here are Mike's comments and mine:

Why it's bad:

First, to see the detail you have to click on the countries one by one. Hard to do a comparison of two countries. This makes it close to FlashJunk.

The color scheme is supposed to help but:

Second, the colors are too close together to allow easy comparisons of, say, Canada and Australia.

In addition, the binning of the colors is uneven and oddly chosen.  In the middle of the scale, each color shift represents 1% but at the edge, it is 5%, or more.

Third, area of these countries, or their geographic location, isn't really that relevant. Market cap might be. Then tiny-but-richly-capitalistic Netherlands wouldn't have to be shown in the middle of the Atlantic, as if the dikes had all burst and Amsterdam had floated out to sea.

Indeed, it begs the question: what were the gold dots suppose to signify? (Hint: it's not location.)

Fourth, why the selectivity? There's stock markets in Turkey, and in Russia, and in Ireland and in Thailand. (Oh, wait, they show the one in Thailand -- except they put it in Myanmar instead.)

Finally, the chart lacks a punch line. 

In the junkart version, I want to test the hypothesis of a global contagion so I plot the data in order of closing times of individual stock markets.  (I just guessed the closing times based on the map.)  Not much here though.

Source: "Global Correction", Wall Street Journal, August 2007.

Large tables

Richard J. asked how we might make sense of this table.  Large tables present lots of challenges.  The trick is to enhance the table with colors and shapes; and as usual, remove any data that doesn't help make your argument.

This table compares countries across different measures of privacy.  Each measure is rated on a scale of 1 to 5, with some blanks.  These ratings are averaged to obtain an overall rating, listed on the right.

In the junkart version, the ratings are presented as slots inside a box.   The overall rating is placed right below the name of the country since this is the most important measure, and how the countries were ordered.  The rows and columns are reversed so as to explain how the overall rating can be decomposed into individual metrics for each country.  I have only shown the top five countries but obviously the chart can be extended to cover all the data. 

If desired, the top 5 countries in each measure can be given a different color: this would increase the data-ink ratio on the chart.  One weakness of this type of chart is that the rows and columns do not have equal status: comparing across rows is more difficult than comparing up and down columns.

Richard also wonders about their treatment of the blanks.  It appears that they omit blanks so each country's rank is the average of non-blank measures.  Omitting blanks may seem innocuous but in fact, this is equivalent to assigning the blank measures ratings equal to the country's average non-blank rank.  Richard wonders if this is the best way to treat these blanks.

 

Source: "Leading surveillance societies", Privacy International.

(Thanks to Richard for sending me the data.)

Red-lining by marriage

Tom W., a reader, noticed this map featured on a BBC News page about the UK family.

One can roughly make out the shape of Great Britain so this is some kind of cartogram.
The title announces that this cartogram concerns the "distribution of population". 

In a typical map like this, the redder reds would indicate higher densities of people.  Yet, the article tells us that the population is divided evenly into 85 squares, each containing "roughly half a million people over 18 years old".

Instead, we seem to have 500K widowed people next to 500K re-married people (most of whom prefer the coasts, by the way), etc.  Apparently, the Brits practise a form of red-lining based on marital status!

The S/M/W/D/R labels are also redundant and very distracting; and the white gridlines interfere with our ability to read the grey boundaries.

Source: "The UK family", BBC News.