Junk Charts

Spring flowers and striking hours

Reader Joe DiNoto sent me to the following National Post (Canada) chart via Twitter, complaining about the circles. (The full chart is found here.)

This chart is supposed to show that the students in Quebec are wrong to go on strike against a roughly 10% increase in tuition fees because the cost of education in Quebec is dwarfed by those in other provinces. This particular message is visible by virtue of the small amount of space occupied by the Quebec "flower" relative to other provinces.

However, to convey that message would require only a chart of the average tuition of the seven provinces. The dataset here contains a lot more information than just the average: it has the tuition by major. But, does the general pattern of relative tuitions apply to individual majors? This chart type (a disguised bubble chart) does the reader few favors. (At least, the designer managed to keep each "petal" at the same angles; otherwise it would make our lives even harder.)

***

In order to bring out the tuition by major comparison, the following set of dot plots helps:

The purple dots are Quebec tuitions. The gray dots are the remaining provinces. We find that Quebec is at the bottom of the cost scale for every major. We also learn that the variance of tuition for dentistry, medicine, and law is very high. Surprisingly, the business degree is rather cheap - maybe the demand for it up north is lower?

High-effort graphics

Jon Quinton made a chart for Cancer Research UK, which is quite an eyeful.

The full infographic is here.

Below is a close-up of the key of this chart:

Where would this chart fall in my "return on effort matrix"? It is an extremely high-effort chart; I got tired trying to figure out what all those dimensions mean.

Is it a high-reward or a low-reward chart? It depends on why you're reading the chart. For most readers, I suspect it's low-reward.

***

In my view, the best charts are high-reward, low-effort. I'd emphasize that by effort, I mean effort by the reader. In general, the effort by the chart designer is inversely proportional to that by the reader.

In some special cases, high-effort charts may have high reward justifying the destruction of some brain cells.

Low-effort, low-reward charts are harmless.

More on the return-on-effort matrix here.

***

One simple improvement to a chart like this one is to produce separate charts for men and women. Outside academia, it seems to me almost all use cases for this chart would involve only one gender.

 

Reading

Rachel features some poor bubble charts on Stats Chat, which is the blog of the Auckland Statistics department. (here, and here)

***

Martin discusses color scales on maps. (link)

***

The JMP blog takes on the challenge of visualizing European debt data (link). We discussed this some time ago (link). The difficulty of this data set is that for each pair of countries, there are two values, and each country can simultaneously be lending to and borrowing from the other.

 

 

 

 

Little orange circles spell trouble

Economists Banerjee and Duflo are celebrated for their work on global poverty but they won't be winning awards for the graphics on the website that supports their book "Poor Economics" (link). Thanks to a reader  for the pointer.

Here is one view ("radial") of the data:

Here is the so-called linear view of the same data:

And here is what the data really look like:

***

The linear view contains a host of misleading features. The length of the row of bubbles does not indicate how many total hours the individual spends doing the selected chores, not even directionally. Instead, the length is a proxy for the number of bubbles per person, which is a measure of the variety of chores - the more chores, the longer the chain of bubbles.

And then, you ask where the color legend is. This information is hidden in the mouse-over effect, or in the drop-down menu by clicking "compare daily activities". It's a bit of wasted energy to have to press on various bubbles in order to understand which color is mapped to which chore, isn't it?

The chart also contains a brainteaser. What is the logic behind the order of the individuals?

And finally, what to make of the little orange circles dancing around the chart? (They also decorate the radial view.) Go to the page and try clicking.

***

Our version tries to do "less is more". One of the tricky features of this dataset is the profusion of little categories capturing daily activities that occupy 0.5 to 1 hour of someone's time. Instead of printing every activity, I chose to bundle all activities requiring less than 1 hour into a single "Others" category.

The "total" category is  a reference level. One can also choose to print white boxes around every single bar and eliminate that row.

 

 

 

A counterfeit data graphic

Just as there are counterfeit handbags that look like the real thing, there are fake data graphics that look like the real thing. Reader San C. shows us an example of this (found on the All Things D blog here):

At first sight, this appears to be a bubble chart. Further, the legend is telling us that the colors are meaningful. So, the bubbles correspond to different types of data, grouped by color, and the size of the bubbles represents the relative level of concern expressed by respondents.

That would be true if we were looking at the "real thing". But this is a counterfeit. How do we know it's fake?

First, the size of the bubbles was not sized to scale. Just look at the Social Security Number versus credit card number (shown on the right). A 1% difference shouldn't be visible on this sort of chart but the credit card bubble is clearly smaller.

Second, the legend gives the impression that the tint of the color carries information. However, it really doesn't. I lined up all of the green bubbles in order of decreasing data, and couldn't find any pattern to the tints.

There also isn't a clear pattern in the location of specific bubbles. Were they randomly scattered onto the chart?

In summary, this is the ultimate non-self-sufficient chart. If we remove the actual printed data from this chart, we're left with nothing.

***

This data can be put onto a grouped bar chart dot plot.

***

Aside from the graphical aspects, we should pay attention to some statistical issues.

The article does not stress enough the potential bias of this survey. The survey is an online survey of Internet users. Their average opinion about Internet-related issues should not be used to represent the opinion of the average American without careful consideration. There is a good possibility that people who have concerns about Internet privacy are less likely to be found on the Internet.

I also wonder if survey takers understand clearly the poll question. What does it mean by a company "accessing my personal information"? Does it mean I give the company the information (such as my credit card number) because I need to complete a transaction? Does it mean the company purchases such information from an information exchange? And if so, with or without informing me?

In particular, I don't understand the 28% who say they are not concerned about companies accessing their social security number.

 

Motion-sick, or just sick?

Reader Irene R. was asked by a client to emulate this infographic movie, made by UNIQLO, the Japanese clothing store.

Here is one screen shot of the movie:

This is the first screen of a section; from this moment, the globes dissolve into clusters of photographs representing the survey respondents, which then parade across the screen. Irene complains of motion sickness, and I can see why she feels that way.

Here is another screen shot:

Surprisingly, I don't find this effort completely wasteful. This is because I have read a fair share of bore-them-to-tears compilation of survey research results - you know, those presentations with one multi-colored, stacked or grouped bar chart after another, extending for dozens of pages.

There are some interesting ideas in this movie. They have buttons on the lower left that allow users to look at subgroups. You'll quickly find the limitations of such studies by clicking on one or more of those buttons... the sample sizes shrink drastically.

The use of faces animates the survey, reminding viewers that the statistics represent real people. I wonder how they chose which faces to highlight, and in particular, whether the answers thus highlighted represent the average respondent. There is a danger that viewers will remember individual faces and their answers more than they recall the average statistics.

***

If the choice is between a thick presentation gathering dust on the CEO's desk and this vertigo of a movie that perhaps might get viewed, which one would you pick?

 

Why should charts exist?

That sounds like a silly question. Isn't the answer self-evident? Am I suggesting that we banish the discipline of charting?

Maybe I won't go so far. But it's difficult not to have such a destructive thought when one stares at charts like this:

Now, compare the above with this version shown on the right ...  and it's clear all the squares and bubbles and colors gave us nothing. Readers have to read the fine print in order to take in the unequal distribution of income. This chart violates the notion of self-sufficiency we often speak about.

Peering back at the original chart, we find that the entire square grid edifice only serves to explain that 0.01% is one-tenth of 0.1%, which is one-tenth of 1%, etc. On the other hand, the part that has a chance of conveying the main message -- the relative size of the biggest bubble versus the smallest bubble -- is shoved off the screen. The gigantic yellow bubble being mostly off the chart, readers are essentially asked to read the data labels.

***

The same article (via Yahoo!) contains other charts that are well executed. 

This one, for instance, shows the increasing inequality very well. (The legend is on the left panel which I did not include here: the top red line is the top 1%, the other five lines are the quintiles or 20% buckets). At least four-fifths of the country is worse-off now than in 1980 in terms of their share of after-tax income.

Charts should not be used as map lessons

Like Australia-based reader Ken B., I don't understand why many chart designers insist on using charts to deliver lessons to the public on map geography. Here is a recent example from Down Under, on earthquakes: (click on this link for the interactive version)

Was there a quake that shook the middle of the Pacific? Did a new geological formation give New Zealand a Pinocchio nose? No and no. The ugly presentation of the 2010 and 2011 Christchurch earthquakes -- as two ends of a dumbbell -- makes clear the straitjacket that maps are when it comes to delivering quantitative information.

Besides, the bubbles represent the relative magnitude of the quakes when one would hope that their sizes represent the geographical extent of the damage; at least, that would be information that has a spatial dimension.

The location of the quake is the only data with a spatial dimension surfaced on this plot. The only purpose of the map background is to tell us where Christchurch, Sichuan, etc. are on a map. In order to deliver this map lesson, the designer has to hide all of the more interesting data, like the relative magnitudes, the time-lines, the extent of the damage, the mortality rates, etc. In my mind, that is a very poor tradeoff.

From text documents to our eyes

Today we look at an example of a powerful visualization of some unstructured data. The data team at Guardian (UK) organized the Wikileaks data concerning reported incidence of IEDs in Afghanistan.

A scatter plot on a map provides an overview of the intensity of attacks from a spatial perspective. (A part of this map is shown on the right.) The background data -- the relief map of Afghanistan, and the major thoroughfares -- add to our understanding of why attacks were concentrated in certain parts of the country. It is always a great idea to add (con)textual data to help readers grasp the information shown on the chart.

Readers may want to understand the temporal pattern of attacks as well. The designer chose a small-multiples format to show this data, disaggregated by year of occurrence. This graphical construct is very versatile, and illustrates this data well... even though there has been little change over time, apart from a general increase in the number of reported attacks across the country.

  It is a good idea to track the total number of attacks over time -- but not with those bubbles! The bubble chart almost always fails the self-sufficiency test; our eyes are not  equipped to read relative areas of circles, and so any information we obtain about the aggregate number of attacks comes from reading the data directly. Switching to a bar chart, or removing the bubbles, leaving just the data, is recommended.

The major problem with a dataset like this is reporting bias: only attacks that were reported by U.S. personnel were included. The following chart helps close the gap a little by also showing the number of defused attacks, reported in the U.S. database. I'd have preferred a stacked column chart here since the total of defused (gray) and detonated (red) IEDs is an interesting statistic.

A stock trading volume type chart would also be nice, something like this:

 

 

Reading behind the chart

I could have filed this one under Light Entertainment but it's too good a chart to lay to waste:

 
(The chart is from Internet Retailer.)

Let's focus on the (mis)match between the question being addressed and the data collected to address it. The intention of the analyst is fully divulged in the title of the accompanying article: "Don't sell Twitter short: those 140-character messages reach an affluent and engaged audience". The chart supports this claim by showing that Twitter users disproportionately represent the types of consumers that marketers most covet, i.e. those with advanced education, and those earning higher incomes.

***

Up top, the concept "user" is very pliable. Is it someone who has a Twitter feed or is it someone who reads Twitter feeds or is it someone who subscribes to Twitter feeds? Is it someone who is registered or everyone who visits? Is it someone who has visited the site in the last x months? or posted a tweet in the last x months? If a writer, does it include someone with no subscribers? no page views? What about people who simultaneously publishes multiple feeds (like John Cook who writes one of my recommended blogs)?

Now, we don't expect the analyst to describe fully how a "user" is defined but while interpreting this chart, we should ask appropriate questions of the data.

Next, the analyst establishes a reference level called "general population". Is this the right metric? This depends on what the chart is used for. If you are choosing between spending money on Twitter, and say spending the money on national TV advertising, then perhaps this comparison is valid. If you are selecting between Twitter and say Google, then absolutely not. For most readers, I think a more relevant point of comparison would be the general Web user, rather than the general population. This is an important distinction because the general Web user also earns higher incomes and has higher educational attainment than the general population, thus the current set of data exaggerates the "value" of Twitter exposure.

Finally, if you are a marketer looking to spend with Twitter, you are also worried about "reach". Say, 50% of website ABC's users are rich people compared to 10% of your reference population. That sounds like an amazing opportunity. Well, only if website ABC has sufficient number of users! If ABC is a niche website serving only 1% of your reference population, then despite the benefit of targeting, its scale is too small for your need.

Don't take a chart on its surface. Read behind the chart!