Junk Charts

The Earth Institute needs a graphics advisor

Reader Dave S. was disturbed by the graphics in the inaugural World Happiness Report, published by Jeffrey Sachs's Earth Institute (link). It's a 200-page document with lots of graphs, many of which require rework.

Here's a pie chart showing (purportedly) what "happy" people in Bhutan are happy about:

I'm really curious how these domains add up to 100% exactly. Since the data came from some kind of survey, you typically would allow each respondent to pick more than one domains in which he or she is happy. If that is the case, then it would not make sense to add up responses, nor would the total (100%) signify anything.

If, on the other hand, respondents are forced to pick only one domain, it is very suspicious that all 9 domains would essentially receive the same number of votes. Nor would it make sense to ask survey-takers to select only one domain if all 9 domains contribute to someone's happiness.

Pie charts are perhaps the most abused chart type. There are just endless examples of poorly executed pie charts (just browse my last few posts). The prevalence of abuse may be reason enough to ban them.

***

Paired with Figure 4 shown above is Figure 5 shown below, which deepens the mystery:

Compare the captions. What's the difference between "In which domains do happy people enjoy sufficiency?" and "Indicators in which happy people enjoy sufficiency"? The categories are related but not identical (Education vs. Schooling, Health vs. Self reported health status, etc.) However, in Figure 5, the distribution is uniform as in Figure 4. Is the data contradictory? Or the captions misleading?

This column chart would be better presented as a horizontal bar chart so that readers don't have to break their necks trying to read the category names.

The designer should also perform the routine task to get rid of the 120% tick mark on the proportion axis that comes from Excel.

 

 

 

Still shaken from the quake

On the news that the tsunami warning has been called off in Asia, I reminded myself of the human toll of the 2004 calamity by looking up Wikipedia (link).

I didn't receive a warm greeting as I was confronted with the following pie chart:

 

Maybe someone can help me understand what is being plotted here.

Maybe the designer was still shaken from the devastation of the quake when this was drawn?

Light entertainment: Spinning wheel at the fun fair

@TheChadd submitted the following chart via Twitter.

I don't know if "fun fairs" mean the same thing to me as to you but that's where I got introduced to spinning wheel games. You stand 10 feet away from a multi-colored pie chart,  you are supposed to throw darts (or other objects) at the circle, you win gigantic teddy bears if you hit the narrow wedge and maybe a sweet if you hit the big wedge.

To add to the fun, the pie chart is made to spin around slowly.

***

Well, we are at the fun fair and here is the spinning pie chart:

To see the real thing, click here.

Notice that this game has an extra level of difficulty; it spins both clockwise and counterclockwise.

Have a great weekend.

The meaning of most

Megan McArdle started the war on infographics (link). And reader Danielle A. contributes this example, from KissMetrics.

This is one part of a big infographics poster. Needless to say, a bar chart renders this data much better:

The categories are sensibly sorted, and useless tinges of color removed.

***

But I want to draw attention to their conclusion:

Most participants in the survey would wait 6-10 seconds before they abandon pages.

Now we know writers of opinion pieces in the major newspapers have long lost control over the titles of their pieces. Is it true that graphic designers have ceded control over their conclusion statements?

It would appear so. The category being labeled as "most participants in the survey" accounted for 30% of the respondents. When is 30% considered "most"?

Also, surveys are typically tools for generalization so we expect conclusions about the general population of mobile users. Here, whoever wrote this conclusion timidly restricted the remark to "participants of the survey". This is probably an oversight because in other panels, they talk about x% of consumers or y% of mobile internet users. If the survey was probably designed and executed, they should be confident about the whole population, not just the sample.

Finally, nowhere on this poster can you discover which survey this data came from. We have no idea what the sample size is, nor the margin of error.

A data mess outduels the pie-chart disaster for our attention

Reader Daniel L. sends us to a truly horrifying pie chart. This:

Link to the original here.

The background: a smartphone monitoring company Crittercism compiled data on the frequency of app crashes by version of mobile operating systems (Android or Apple iOS). The data is converted into proportions adding to 100%.

If we spend our time trying to figure out the logic behind the ordering and placing of the data (e.g. why iOS is split on both sides? why pieces are not sorted by size?), we will miss the graver problem with this chart - the underlying data.

***

Here is a long list of potential issues:

• Crittercism sells app monitoring tools for app developers. Presumably this is how it is able to count app crashes. But who are their customers? Are they a representative set of the universe of apps? Do we even know the proportion of Android/iOS apps being monitored?
• There is reason to believe that the customer set is not representative. One would guess that more crash-prone apps are more likely to have a need for monitoring. Also, is Apple a customer? Given that Apple has many highly popular apps on iOS, omission of these will make the data useless.
• The data wasn't adjusted for the popularity of apps. It's very misleading to count app crashes without understanding how many times the app has been opened. This is the same fallacy as making conclusions about flight safety based on the list of fatal plane accidents; the millions of flights that complete without incident provide lots of information! (See Chapter 5 of my book for a discussion of this.)
• The data has severe survivorship bias. The blog poster even mentions this problem but adopts the attitude that such disclosure somehow suffices to render useless data acceptable. More recent releases are more prone to crashes just because they are newer. If a particular OS release is particularly prone to app crashes, then we expect a higher proportion of users to have upgraded to newer releases. Thus, older releases will always look less crash-prone, partly because more bugs have been fixed, and partly because of decisions by users to switch out. iOS is the older operating system, and so there are more versions of it being used.
• How is a "crash" defined?  I don't know anything about Android crashes. But my experience with PC operating systems is that each one has different crash characteristics. I suspect that an Android crash may not be the same as an iOS crash.
• How many apps and how many users were included in these statistics? Specifying the sample size is fundamental to any such presentation.
• Given the many problems related to timing as described above, one has to be careful when generalizing with data that only span two weeks in December.
• There are other smartphone OS being used out there. If those are omitted, then we can't have a proportion that adds up to 100% unless those other operating systems never have app crashes.

***

How to fix this mess? One should start with the right metric, which is the crash rate, that is, the number of crashes divided by the number of app starts. Then, make sure the set of apps being tracked is representative of the universe of apps out there (in terms of popularity).

Some sort of time matching is needed. Perhaps trace the change in crash rate over time for each version of each OS. Superimpose these curves, with the time axis measuring time since first release. Most likely, this is the kind of problem that requires building a statistical model because multiple factors are at play.

Finally, I'd argue that the question being posed is better answered using good old-fashioned customer surveys collecting subjective opinion ("how many crashes occurred this past week?" or "rate crash performance"). Yes, this is a shocker: a properly-designed small-scale survey will beat a massive-scale observational data set with known and unknown biases. You may agree with me if you agree that we should care about the perception of crash severity by users, not the "true" number of crashes. (That's covered in Chapter 1 of my book.)

 

 

 

The massive burden of pie charts

The world would be a better place if pie charts were banned. I respect the creativity that went to producing charts like the one below but surely, pie charts don't deserve that level of attention:

(Via Business Insider via Jobvine)

Here is the same information using a bar chart.

 

It's amazing how much extra stuff (colors, text, shapes, lines, labels, etc.) is needed when you use pie charts.

Motion-sick, or just sick?

Reader Irene R. was asked by a client to emulate this infographic movie, made by UNIQLO, the Japanese clothing store.

Here is one screen shot of the movie:

This is the first screen of a section; from this moment, the globes dissolve into clusters of photographs representing the survey respondents, which then parade across the screen. Irene complains of motion sickness, and I can see why she feels that way.

Here is another screen shot:

Surprisingly, I don't find this effort completely wasteful. This is because I have read a fair share of bore-them-to-tears compilation of survey research results - you know, those presentations with one multi-colored, stacked or grouped bar chart after another, extending for dozens of pages.

There are some interesting ideas in this movie. They have buttons on the lower left that allow users to look at subgroups. You'll quickly find the limitations of such studies by clicking on one or more of those buttons... the sample sizes shrink drastically.

The use of faces animates the survey, reminding viewers that the statistics represent real people. I wonder how they chose which faces to highlight, and in particular, whether the answers thus highlighted represent the average respondent. There is a danger that viewers will remember individual faces and their answers more than they recall the average statistics.

***

If the choice is between a thick presentation gathering dust on the CEO's desk and this vertigo of a movie that perhaps might get viewed, which one would you pick?

 

Reading the landscape

Here are some posts I find worth reading on other graphics blogs:

Nick has done wonderful work on the evolution of the rail industry in the U.S., with a flow chart showing how mergers have produced the four giants of today, as well as a small multiples of maps showing how they split up the country.

A lovely feature of the flow chart is the use of red lines to let readers see at a glance that Union Pacific is the only rail company that has lasted the entire 4 decades, while the other 3 giants came into being within the last 20 years.

On the maps, notice a slight inconsistency between the left and right columns: on the right side, both maps have the same set of anchor cities, which act as "axes" to help readers compare the maps; on the left side, the sets of anchor cities are not identical. It would also be interested to see a version with all four route maps superimposed and differentiated by color. That may bring out the competitive structure better.

 ***

Georgette has a nice post summarizing issues with picking colors when producing charts. Her blog is called Moved by Metrics.

 ***

Meanwhile, Martin finds a shockingly poor pie chart here.

***

There was a time where you'd find the kind of heatmaps featured here by Nathan as wallpaper in my office. It's a great visualization tool for exploring temporal patterns in large data sets. However, I'd never even think of putting these in a presentation.  It's a starting point, not an end-point, of an analysis project. Some things are wonderful for consumption only in private!

 

 

 

 

 

Return this plate, I want my pie chart

Reader Brad E. reminds me about the USDA's attempt to "improve" the visual presentation of dietary standards. As reported here, the food pyramid failed its mission and is retired. Here comes MyPlate!

 
According to this report, the government wants to impart these key points:

• MyPlate offers a visual reminder to make healthy food choices when you choose your next meal.
• It can help prioritize food choices and remind us to make fruits and vegetables half of our plates each meal.
• On the other side of the plate – and beside it – we see the other important food groups for a healthy meal: whole grains, lean proteins, and low fat dairy.

We have been warned not to think of this as a pie chart.

What do we call a circular chart in which the area of the circle is partitioned into separate regions?

***

How does one say return MyPlate to the kitchen fast enough? The biggest problem here is that the key points are out of sync with the chart details. The MyPlate diet, as depicted, has less than the recommended amounts of fruits and vegetables! Since those two important food groups only equal grains and proteins, the presence of dairy means that fruits and vegetables form less than half of this diet.

The core message is that one should split one's diet in half, with fruits and veggies on the one side and grains, proteins and dairy on the other. If this is so, the following chart gets this point across with minimal probability of confusion:

If, on the other hand, the above chart is deemed too simple, and the message really does require proportions of each of the five food categories, then the sad truth is that a pie chart would have conveyed the message better.

MyPlate serves up strange portions that cannot be properly sized. How big is the "dairy" circle compared to any of the quadrants? How does one judge the irregularly-sized quadrants (grains and vegetables)? If there is any use for the pie chart, it is to display simple concepts with limited dimensions.

Another reminder: most pie charts are unreadable

I'm outsourcing today's post to reader Joel D., who describes himself as a geographer, professional researcher and son of a graphics designer. He wrote a wonderful entry about the following pie chart:

 You know this "big graphic" is in big trouble when the caption tells you only 4 clubs made a profit and yet you'd be hard pressed to locate those 4 clubs from the chart itself!

The other telling sign is using blocks to represent profit/loss. This never works whether the blocks are pie sectors or stacked bars. Line charts are much better.

Joel has much more commentary, and an improved chart here.  Enjoy!