Ryan McCarthy linked to a post by Ruchir Sharma running on Ezra Klein's blog analyzing global billionaires.
It has an accompanying chart, which fails our self-sufficiency test. That test involves erasing raw data from a chart, and figuring out how much information the graphical elements themselves convey.
The primary metric used by Sharma is the billionares' total net worth as a percentage of the country's GDP. This metric is embedded in double concentric circles. Unfortunately, without mental gymnastics, readers can't tell what the proportion is. This means we must look at the raw data which is supplied as a column on the right of the graphic. If readers are taking the information from the column of raw data, then why draw a chart?
The actual data is revealed on the left . Don't tell anyone you read it here but pie charts would work well with this dataset. You might complain that there is a conceptual problem - that if we sum up the net worth of everyone in a country, it would not equal GDP. I think the sum doesn't work - economists can chime in about this. Sharma seems to imply that the total would sum to 1. Anyone's net worth is accumulated over a number of years in which the GDP is fluctuating while the total GDP is given for a specific end of quarter of some year so does it make sense to divide one by the other?
Also, the fact that some people may have negative net worth creates problems with the pie-chart format and it's not much better in a concentric-circle format either.
*** A maddening decision puts the United States, which is the biggest circle, at the bottom of the chart. Notice that the countries are sorted from larger billionaires' share to smaller. The U.S. belongs to the top 5 nations with the worst inequality by this metric and yet a cheeky little bookmark sends us to the bottom of the list together with the more-equal nations.
Not only is the location of U.S. privileged, the location of the text, the number of decimal places given in the net worth amount, and the presence of the GDP value all set the U.S. apart from the other countries plotted.
The most interesting piece of information is waiting to be reconstructed. In Malaysia, nine citizens own as much as 18.3% of the country's GDP. In Mexico, 11 people own 10.9% of the country's GDP.
To make the number even more telling, we have to incorporate the population size. For Malaysia it is 28 million. This means that the top 0.000032% of the population owns 18.3%. In the case of perfect equality, this proportion would own 0.000032%. We can say the inequality index is 570,000. In Mexico, the index is 1.1 million. So in fact, the concentration of wealth at the time is worse in Mexico than in Malaysia. For reference, the U.S. comes in at 78,000.
Of course, the use of billionaires as a filtering device to determine who to count or not is completely arbitrary. In measuring income inequality, one should look at what proportion of the population control 50% of the wealth, for example.
There is no explanation for the choice of countries. The U.S. is the only developed nation in the entire chart.
Reader Dave S. was disturbed by the graphics in the inaugural World Happiness Report, published by Jeffrey Sachs's Earth Institute (link). It's a 200-page document with lots of graphs, many of which require rework.
Here's a pie chart showing (purportedly) what "happy" people in Bhutan are happy about:
I'm really curious how these domains add up to 100% exactly. Since the data came from some kind of survey, you typically would allow each respondent to pick more than one domains in which he or she is happy. If that is the case, then it would not make sense to add up responses, nor would the total (100%) signify anything.
If, on the other hand, respondents are forced to pick only one domain, it is very suspicious that all 9 domains would essentially receive the same number of votes. Nor would it make sense to ask survey-takers to select only one domain if all 9 domains contribute to someone's happiness.
Pie charts are perhaps the most abused chart type. There are just endless examples of poorly executed pie charts (just browse my last few posts). The prevalence of abuse may be reason enough to ban them.
Paired with Figure 4 shown above is Figure 5 shown below, which deepens the mystery:
Compare the captions. What's the difference between "In which domains do happy people enjoy sufficiency?" and "Indicators in which happy people enjoy sufficiency"? The categories are related but not identical (Education vs. Schooling, Health vs. Self reported health status, etc.) However, in Figure 5, the distribution is uniform as in Figure 4. Is the data contradictory? Or the captions misleading?
This column chart would be better presented as a horizontal bar chart so that readers don't have to break their necks trying to read the category names.
The designer should also perform the routine task to get rid of the 120% tick mark on the proportion axis that comes from Excel.
@TheChadd submitted the following chart via Twitter.
I don't know if "fun fairs" mean the same thing to me as to you but that's where I got introduced to spinning wheel games. You stand 10 feet away from a multi-colored pie chart, you are supposed to throw darts (or other objects) at the circle, you win gigantic teddy bears if you hit the narrow wedge and maybe a sweet if you hit the big wedge.
To add to the fun, the pie chart is made to spin around slowly.
Well, we are at the fun fair and here is the spinning pie chart:
Megan McArdle started the war on infographics (link). And reader Danielle A. contributes this example, from KissMetrics.
This is one part of a big infographics poster. Needless to say, a bar chart renders this data much better:
The categories are sensibly sorted, and useless tinges of color removed.
But I want to draw attention to their conclusion:
Most participants in the survey would wait 6-10 seconds before they abandon pages.
Now we know writers of opinion pieces in the major newspapers have long lost control over the titles of their pieces. Is it true that graphic designers have ceded control over their conclusion statements?
It would appear so. The category being labeled as "most participants in the survey" accounted for 30% of the respondents. When is 30% considered "most"?
Also, surveys are typically tools for generalization so we expect conclusions about the general population of mobile users. Here, whoever wrote this conclusion timidly restricted the remark to "participants of the survey". This is probably an oversight because in other panels, they talk about x% of consumers or y% of mobile internet users. If the survey was probably designed and executed, they should be confident about the whole population, not just the sample.
Finally, nowhere on this poster can you discover which survey this data came from. We have no idea what the sample size is, nor the margin of error.
The background: a smartphone monitoring company Crittercism compiled data on the frequency of app crashes by version of mobile operating systems (Android or Apple iOS). The data is converted into proportions adding to 100%.
If we spend our time trying to figure out the logic behind the ordering and placing of the data (e.g. why iOS is split on both sides? why pieces are not sorted by size?), we will miss the graver problem with this chart - the underlying data.
Here is a long list of potential issues:
Crittercism sells app monitoring tools for app developers. Presumably this is how it is able to count app crashes. But who are their customers? Are they a representative set of the universe of apps? Do we even know the proportion of Android/iOS apps being monitored?
There is reason to believe that the customer set is not representative. One would guess that more crash-prone apps are more likely to have a need for monitoring. Also, is Apple a customer? Given that Apple has many highly popular apps on iOS, omission of these will make the data useless.
The data wasn't adjusted for the popularity of apps. It's very misleading to count app crashes without understanding how many times the app has been opened. This is the same fallacy as making conclusions about flight safety based on the list of fatal plane accidents; the millions of flights that complete without incident provide lots of information! (See Chapter 5 of my book for a discussion of this.)
The data has severe survivorship bias. The blog poster even mentions this problem but adopts the attitude that such disclosure somehow suffices to render useless data acceptable. More recent releases are more prone to crashes just because they are newer. If a particular OS release is particularly prone to app crashes, then we expect a higher proportion of users to have upgraded to newer releases. Thus, older releases will always look less crash-prone, partly because more bugs have been fixed, and partly because of decisions by users to switch out. iOS is the older operating system, and so there are more versions of it being used.
How is a "crash" defined? I don't know anything about Android crashes. But my experience with PC operating systems is that each one has different crash characteristics. I suspect that an Android crash may not be the same as an iOS crash.
How many apps and how many users were included in these statistics? Specifying the sample size is fundamental to any such presentation.
Given the many problems related to timing as described above, one has to be careful when generalizing with data that only span two weeks in December.
There are other smartphone OS being used out there. If those are omitted, then we can't have a proportion that adds up to 100% unless those other operating systems never have app crashes.
How to fix this mess? One should start with the right metric, which is the crash rate, that is, the number of crashes divided by the number of app starts. Then, make sure the set of apps being tracked is representative of the universe of apps out there (in terms of popularity).
Some sort of time matching is needed. Perhaps trace the change in crash rate over time for each version of each OS. Superimpose these curves, with the time axis measuring time since first release. Most likely, this is the kind of problem that requires building a statistical model because multiple factors are at play.
Finally, I'd argue that the question being posed is better answered using good old-fashioned customer surveys collecting subjective opinion ("how many crashes occurred this past week?" or "rate crash performance"). Yes, this is a shocker: a properly-designed small-scale survey will beat a massive-scale observational data set with known and unknown biases. You may agree with me if you agree that we should care about the perception of crash severity by users, not the "true" number of crashes. (That's covered in Chapter 1 of my book.)
Reader Irene R. was asked by a client to emulate this infographic movie, made by UNIQLO, the Japanese clothing store.
Here is one screen shot of the movie:
This is the first screen of a section; from this moment, the globes dissolve into clusters of photographs representing the survey respondents, which then parade across the screen. Irene complains of motion sickness, and I can see why she feels that way.
Here is another screen shot:
Surprisingly, I don't find this effort completely wasteful. This is because I have read a fair share of bore-them-to-tears compilation of survey research results - you know, those presentations with one multi-colored, stacked or grouped bar chart after another, extending for dozens of pages.
There are some interesting ideas in this movie. They have buttons on the lower left that allow users to look at subgroups. You'll quickly find the limitations of such studies by clicking on one or more of those buttons... the sample sizes shrink drastically.
The use of faces animates the survey, reminding viewers that the statistics represent real people. I wonder how they chose which faces to highlight, and in particular, whether the answers thus highlighted represent the average respondent. There is a danger that viewers will remember individual faces and their answers more than they recall the average statistics.
If the choice is between a thick presentation gathering dust on the CEO's desk and this vertigo of a movie that perhaps might get viewed, which one would you pick?
Here are some posts I find worth reading on other graphics blogs:
Nick has done wonderful work on the evolution of the rail industry in the U.S., with a flow chart showing how mergers have produced the four giants of today, as well as a small multiples of maps showing how they split up the country.
A lovely feature of the flow chart is the use of red lines to let readers see at a glance that Union Pacific is the only rail company that has lasted the entire 4 decades, while the other 3 giants came into being within the last 20 years.
On the maps, notice a slight inconsistency between the left and right columns: on the right side, both maps have the same set of anchor cities, which act as "axes" to help readers compare the maps; on the left side, the sets of anchor cities are not identical. It would also be interested to see a version with all four route maps superimposed and differentiated by color. That may bring out the competitive structure better.
Georgette has a nice post summarizing issues with picking colors when producing charts. Her blog is called Moved by Metrics.
Meanwhile, Martin finds a shockingly poor pie chart here.
There was a time where you'd find the kind of heatmaps featured here by Nathan as wallpaper in my office. It's a great visualization tool for exploring temporal patterns in large data sets. However, I'd never even think of putting these in a presentation. It's a starting point, not an end-point, of an analysis project. Some things are wonderful for consumption only in private!