Junk Charts

Light entertainment: others' idea of fun

This chart (link)

I think it's a line chart, not an area chart.

Highway Safety Agency goes rogue

A reader sends me to Adam Obeng, who did the dirty work deconstructing a set of charts by the U.S. National Highway Traffic Safety Administration on his blog. Here's an example of these charts:

Aside from the sneaker chart, they concocted a pop stick, a pencil, a tower of Hanoi, etc. These objects are ones I think should be evaluated as art. Adam gamely tells us that the proportions are totally off, and they are both internally and externally inconsitent.

***

I'll add two small points to Adam's post.

First, these charts pass my self-sufficiency test, that is to say, they did not print the entire data set (just one number here) on the page. Alas, given the distortion identified by Adam, not printing the data means everyone is free to create their own data. Herein lies the problem: there is an argument for allowing a small degree of distortion in exchange for "beauty" but these charts without any data have gone too far.

Second, see Adam's last point (the footnote). The original data is something quite convoluted: “3 out of 4 kids are not as secure in the car as they should be because their car seats are not being used correctly.” (How would they know this, I wonder.) This is a statistic about kids while the picture shows a statistic about their parents (or drivers).

 

 

Fifteen points, add confusion

Bloomberg Markets (November) has this chart showing national debt as a percentage of GDP for selected countries.

If you count the points, there are only 15 on the chart, three annual numbers for each of five countries.

The theme appears to be redundancy. Countries are identified by both their names and their flags, one placed vertically and one placed horizontally. The 2011 debt-to-GDP ratios are displayed twice, once as the rightmost points on those lines, and once as data labels on the far right. The vertical axis plus gridlines too provide information that is quite unnecessary when there are labels on the right.

The five dots soak up much of our attention but there is no data in them. In particular, the Greek and U.S. dots fall in between two years where the straight lines are just convenience.

If all the lines were given the same color, then it would be straightforward to highlight the U.S. data by giving it a different color. The switch to an area chart for U.S. data is both ugly and distorting. The distortion is due to the vertical axis not starting at zero -- this is acceptable for line charts but certainly not for area charts.

***

I find that the data themselves present a challenge for interpretation. Based on my reading of economic news, Greece, Spain and Italy are current poster children for countries with debt problems while Japan has had a debt problem but is more associated with a stagnation problem more recently.

The issue is the comparison of each of these countries to the U.S. brings almost zero insight. Italy, for example, probably has some kind of legislative control that puts a lid on debt at 120 percent of GDP. Japan, while not currently in the conversation about countries in danger of default, has much worse debt-to-GDP ratio than Greece, which is in serious trouble. By this measure, Spain is nowhere near as indebted as the U.S. and yet it is in much trouble.

In the following version, I plotted the relative change in the metric with 2009 set to 100.

Poking at the data behind a chart

Reader Jamie D. wasn't very amused by the following chart, from the Freakonomics blog (link):

Jamie summarized his view as follows:

First of all, a quick look of the graph makes you think you're comparing states with helmet laws vs. those w/out helmet laws.  But, upon closer reading, it's actually just a comparison between states that have repealed their helmet laws between 1994 and 2007 and ALL OTHER STATES.  Reading further down it appears that even in the heyday of helmet laws, only 26 states had them.  Thus, the graph is really a comparison between 7 states that repealed the helmet laws in that time period and the other 43 states, 24 of which have never had helmet laws at all.

In the Trifecta checkup, this problem surfaces as a disconnect between the question being investigated, and the data used to address the question. (For an explanation of the Trifecta checkup, see this post.)

Further, Jamie asked:

More importantly from a graphical perception point of view: the horizontal axis identifies itself as "years relative to repeal."  While that time horizon makes sense with respect to the repeal states (in light green), it is not clear at all what "year relative to repeal" means in the 43 states that did not have a helmet law repeal during the time at issue (the dark green).  This might be further explained in the book (which I don't have), but even if does, the chart is misleading and not helpful in explaining data (which is its raison d'être.)

Aligning the data to a particular event (like the repeal of a particular law) is typically a very smart thing to do... and it belongs to one of many statistical adjustments that make perfect sense, like the seasonal adjustments of economic data (link). But here, as Jamie pointed out, in the "control" group in which states did not repeal the helmet laws, it isn't clear what should be the "anchor" year (time = 0).

***

At a more abstract level, the designer is working with a dataset with four dimensions: the state, the year, the status of the helmet law within a state, and the organ donation rate. The data can be arranged as a 50-row, 4-column table.

The first issue has to do with the values in the third column (status of the helmet law). It would be a mistake to positively identify the states that have repealed the law as "repeal states", and then by default label all the rest as "non-repeal states". Instead, there should be three levels: repeal states, non-repeal states, and no-helmet-law states. I'd then plot three lines instead of two.

The second issue arises when the designer tries to transform the second column, from actual years (2000, 2001, etc.) to relative years (anchor year = 0, and other years go +1, +2 and -1, -2, etc.). At some point, she would need to make an explicit decision of how to create "relative years" for the non-repeal and no-helmet-law states.

***

One other problem with this chart is not starting the vertical axis from zero when they are drawing attention to the area under the lines, and not the levels of the lines themselves. If they use a line chart instead, the start-at-zero rule is not as important.

I'll skip the critique of the overall plan of this Freakonomics analysis as I already wrote much about that (with Andrew Gelman). See our article here.

The "data" corner of the Trifecta

In the JunkCharts Trifecta checkup, we reserve a corner for "data". The data used in a chart must be in harmony with the question being addressed, as well as the chart type being selected. When people think about data, they often think cleaning the data, processing the data but what comes before that is collecting the data -- specifically, collecting data that directly address the question at hand.

Our previous post on the smartphone app crashes focused on why the data was not trustworthy. The same problem plagues this "spider chart", submitted by Marcus R. (link to chart here)

Despite the title, it is impossible to tell how QlikView is "first" among these brands. In fact, with several shades of blue, I find it hard to even figure out which part refers to QlikView.

The (radial) axis is also a great mystery because it has labels (0, 0.5, 1, 1.5). I have never seen surveys with such a scale.

The symmetry of this chart is its downfall. These "business intelligence" software are ranked along 10 dimensions. There may not be a single decision-maker who would assign equal weight to each of these criteria. It's hard to imagine that "project length" is equally important as "product quality", for example.

Take one step backwards. This data came from responders to a survey (link). There is very little information about the composition of the responders. Are they asked to rate all 10 products along 10 dimensions? Do they only rate the products they are familiar with? Or only the products they actively use? If the latter, how are responses for different products calibrated so that a 1 rating from QlikView users equals a 1 rating from MicroStrategy users? Given that each of these products have broad but not completely overlapping coverage, and users typically deploy only a part of the solution, how does the analysis address for the selection bias?

***

The "spider chart" is, unfortunately, most often associated with Florence Nightingale, who created the following chart:

This chart isn't my cup of tea either.

***

Also note that the spider chart has so much over-plotting that it is impossible to retrieve the underlying data.

 

 

Showing off the world in charts

Stefan S. who works for the UN data project and is a regular contributor to this blog, points us to a new report they have issued that contain a host of charts. The report is an update on what has happened to our Earth since 1992 (The Earth Summit). Link to the PDF file here.

***

This life expectancy chart (shown on left) uses a Bumps-type chart, and is very nicely done, clean and informative. 

***

This age distribution chart shown on the right is unusual. It's a case of the data defeating the chart type. The magnitude of the 5-year changes is just not large enough as a percentage of the total to register. On a different data set, I can see this chart type being more effective.

***

Now, this criss-cross chart (bottom left) reminds me of Friedman's foolish attempt some time ago. It has various issues, like dual axes, excessive labels and inattentive titles (not indicating that the base population was only of developing countries).

  Instead, I attempted an area chart, using population size as the primary metric. Perhaps a more direct way to illustrate this point is to plot the growth rate of the slum population versus that of the total population.

***

This map is excellent, showing the spatial distribution of the countries with above-average and below-average GDP per capita. It would be even better if smaller geographic units can be used so that the distribution within each country can also be seen.

***

I'd like to salute all the people around the world who work at statistical agencies and who collect and make sense of all of this data, without which any of these charts would not have been possible.

False promises of equality and structure

(Here's something especially for those like me who are stuck in their homes in the Northeast USA this weekend.)

A few readers weren't impressed by Nielsen's presentation of the smartphone marketplace:

This chart type is very popular, both among business consultants and statisticians. Consultants call them "marimekko charts" while statisticians call them "mosaic charts". It's got multiple names as it has been reinvented multiple times. I have nightmares from having to produce this sort of charts in Powerpoint by hand (deconstructing and reconstructing column charts), and I have written before about my dislike of them (see here, and here).

***

Supporters point to two advantages of this type of chart:

• Equality: it puts the two dimensions of the market place -- operating system/software, and producer/brand -- on equal footing. As an added bonus, the areas of the rectangles are meaningful: they correspond to the relative market shares.
• Structure: the chart often reveals interesting aspects of the structure of the data. For instance, here it shows that certain smartphones have "closed" systems where the OS and producer forms a one-to-one relationship while some producers like HTC makes phones with different operating systems.

A little thought exposes these as false promises.

The two dimensions are, in fact, not equal. Look at the one contiguous column for Apple versus two separate sections for HTC. In order to know the market share of HTC, the reader needs to do additions... in his/her head. While this is not so hard when HTC appears only twice, your reader would not be amused if HTC appears seven times on the same mosaic. It is a limitation of this chart type that one cannot get the column sections to be of one piece without destroying the one-piece structure of the row sections.

In addition, I don't think it is easy to compare the areas of fat rectangles versus narrow rectangles, or squares versus long strips, etc. On consulting style charts, you almost always find the entire data set printed, which is to say, this chart is rendered not self-sufficient. On statistical charts, you typically find axis labels; this is not much better because of the difficulty in estimating relative areas.

The extent to which one can learn the structure of the data is restricted by our ability to estimate and sum areas.

***

In the junkart version, I use a flow chart. Special attention is paid to expressing as clearly as possible the structure of the marketplace, thus the separate sections for the "open" versus "closed" systems, as well as the many-to-many relationships among the "open"-system players.

The thickness of the flows is proportional to the market shares. I added a few data points to anchor the scale. The two dimensions of the data are treated symmetrically.

There is also no need to startle readers with a kaleidoscope of colors so typical of marimekkos.

 

 

 

 

 

Did days get longer in the last 30 years? Fast Company thinks so.

Craig N. sent us to this infographic from Fast Company about MTV's 30th anniversary, nominating it as the worst infographic ever.

Apply the self-sufficiency test to this chart. Wish away the printed data. Now, does the chart convey any message?  Where is the data embedded? Is it in the white dot, the black dot, the gold ring, the gold disc, the black ring, the eye-white? All of the above?

Now, do the same test on this chart (I removed the sales data, replacing it with years):

How would one compare the white to the orange? If one measures the lengths of the sides, the ratio of white to orange is about 1.32. If one compares areas of the squares, then the ratio is 1.73. Note that this requires the reader to see through the orange area to size up the area of the large white square. Alternatively, we can compute the ratio of the white area as observed to the orange square, and that ratio is 0.73.

The real ratio between 1980 and 2010 sales is given as 3.9/2.7 = 1.44. Given rounding errors, it seems like the designer may have used a ratio of lengths of the sides.

The problem is the same whether sides or areas are used. Can the reader figure out that the 1980 sales is about 40% higher than the 2010 sales?

I suspect that most of us react primarily to the visible areas, which means that we'd have gotten the direction of the change wrong, let alone the magnitude.

***

 

Craig really dislikes this one. It's a variant of the racetrack chart. As any athlete knows, inner tracks are shorter than outer tracks. Could it be that days have gotten longer in the last 30 years? Apparently, the editors at Fast Company think so.

The best way to handle two dimensions may be to not use two dimensions

Guess what the designer at Nielsen wanted to tell you with this chart:

Reader Steven S. couldn't figure it out, and chances are neither can you.

What about...

• The smartphone (OS) market is dominated by three top players (Android, Apple and Blackberry) each having roughly 30% share, while others split the remaining 10%.
• The age-group mix for each competitor is similar (or are they?)

Maybe those are the messages; if so, there is no need to present a bivariate plot (the so-called "mosaic" plot, or in consulting circles, the Marimekko). Having two charts carrying one message each would accomplish the job cleanly.

***

Trying to do too much in one chart is a disease; witness the side effects.  

The two columns, counting from the right, contain rectangles that appear to be of different sizes, and yet the data labels claim each piece represents 1%, and in some cases "< 1%".  The simultaneous manipulation of both the height and the width plays mind tricks.

Also, while one would ordinarily applaud the dropping of decimals from a chart like this, doing so actually creates the ugly problem that the five pieces of 1% (on the left column shown here) have the same width but clearly varying heights!

What about this section of the plot shown on the left? Does the smaller green box look like it's less than 1/3 the size of the longer green box? This chart is clearly not self-sufficient, and as such one might prefer a simple data table.

The downfall of the mosaic plot is that it gives the illusion of having two dimensions but only an illusion: in fact, the chart is dominated by one dimension, as all proportions are relative to the grand total.

For instance, the chart says that 6% of all smartphone users are between the ages of 18 and 24 AND uses an Android phone. It also tells us that 2% of all smartphone users are between 35 and 44 AND uses a Palm phone. Those are not two numbers anyone would desire to compare. There are hardly any practical questions that require comparing them.

Sometimes, the best way to handle two dimensions is not to use two dimensions.

***

 The original article notes that "Of the three most popular smartphone operating systems, Android seems to attract more young consumers." In the chart shown below,  we assume that the business question is the relative popularity of phone operating systems across age groups. 

The right metric for comparison is the market share of each OS within an age group.

For example, tracing the black line labeled "Android", this chart tells us that Android has 37% of the 18-24 market while it has about 20% of the 65 and up market. 

Android has an overall market share of about 30%, and that average obscures a youth bias that is linear with age.

On the other hand, the iPhone (green line) has also an average market share of about 30% but its profile is pretty flat in all age groups except 65 and up where it has considerable strength.

Further, the gap between Android and iPhone at the older age group actually opens up at 55 years and up. In the 55-64 age group, the iPhone holds a market share that is similar to its overall average while the Android performs quite a bit worse than its average. We note that Palm OS has some strength in the older age groups as well while the Blackberry also significantly underperforms in 65 and over.

Why aren't all these insights visible in the mosaic chart? It all because the chosen denominator of the entire market (as opposed to each age group) makes a lot of segments very small, and then the differences between small segments become invisible when placed beside much larger segments.

Now, the reconstituted chart gives no information about the relative sizes of the age groups. The market size for the older groups is quite a bit smaller than the younger groups. This information should be provided in a separate chart, or as a little histogram tucked under the age-group axis.

 

 

The chart that reveals a mysterious death

I agree with Business Insider that the following chart is attractively drawn. It nicely illustrates the rise and fall of various music media over time.

Area charts are more visually appealing than line charts, largely because line charts frequently leave large patches of white space. But one should be aware of some shortcomings of area charts.

Notice that the outer envelope of the area chart represents the growth in music sales across all media, not to be mistaken for the growth of any particular media. However, the primary message of this chart relates to the change in mix among different media, not the growth of the total market. Because of the stacking of different areas on top of each other, it is not an easy task to read the growth of any individual piece, such as CDs.

***

Unlike Business Insider who found some answers on this chart, I find that this chart raises a mysterious -- and important -- question: what happened around 2001 to damage CD sales? Since according to this chart, digital sales didn't really show up till 2004-ish, there is a gap of two years or so when CD sales dropped drastically, seemingly of its own volition.