Junk Charts

Rotating circle, loose ends, on-line dashboards and charts

There is a tendency when producing dashboards to go for the cutesy-cutesy. Reader Daniel L. came across an attempt by Facebook to document its data center metrics (link). They chose this circular, spiraling design:

Notice that the lines of equal distance on a circular plot are the concentric circles. Thus, when they connect different points in a continuous way, as if it were a standard line chart, the line segments between data points are distorted. The diagram below shows the problem:

One potential advantage (although not worthwhile) of wrapping the data into a circle is that the 24 hours become a continuous line. Except that it isn't the case here! Weirdly, the purple and blue lines show a huge discontinuity at the ray that points vertically upwards from the origin. This leads to an even more fascinating find.

The circle actually rotates! It's like a rotating restaurant. The time shown vertically pointing upwards keeps changing as I write this post. This makes the discontinuity even more baffling. You'd think the previous data point just shifts anti-clockwise but apparently not. If any of you can figure this out, please leave a comment.

***

As Daniel pointed out, the traditional line charts shown in the bottom half of the page would have done the job with less fuss. Not as eye-catching, but not as baffling either.

***

One innovation of on-line charts is the replacement of axis labels with mouse-over effects. Mousing over the chart here produces the underlying data values. This is elegance.

One horrible trend with on-line charts is the horrendous choice of scale. Look at the top two charts, especially the orange line chart about power usage. It makes no sense to choose a scale that completely annihilates the underlying fluctuations.

I have found the same problems with many Google charts. It looks as if nothing is happening except when you look more closely, you learn that a tiny distance represents a big percentage shift in the underlying data.

 

Bad charts can happen to good people

I shouldn't be surprised by this. No sooner did I sing the praise of Significance magazine (link) than a reader sent me to some charts that are not deserving of their standard.

Here is one such chart (link):

Quite a few problems crop up here. The most hurtful is that the context of the chart is left to the text. If you read the paragraph above, you'll learn that the data represents only a select group of institutions known as the Russell Group; and in particular, Cambridge University was omitted because "it did not provide data in 2005". That omission is a curious decision as the designer weighs one missing year against one missing institution (and a mighty important one at that). This issue is easily fixed by a few choice words.

You will also learn from the text that the author's primary message is that among the elite institutions, little if any improvement has been observed in the enrollment of (disadvantaged) students from "low participation areas". This chart draws our attention to the tangle of up and down segments, giving us the impression that the data is too complicated to extract a clear message.

The decision to use 21 colors for 21 schools is baffling as surely no one can make out which line is which school. A good tip-off that you have the wrong chart type is the fact that you need more than say three or four colors.

The order of institutions listed in the legend is approximately reverse of their appearance in the chart. If software can be "intelligent", I'd hope that it could automatically sort the order of legend entries.

If the whitespace were removed (I'm talking about the space between 0% and 2.25% and between 8% and 10%), the lines could be more spread out, and perhaps labels can be placed next to the vertical axes to simplify the presentation. I'd also delete "Univ." with abandon.

The author concludes that nothing has changed among the Russell Group. Here is the untangled version of the same chart. The schools are ordered by their "inclusiveness" from left to right.

This is a case where the "average" obscures a lot of differences between institutions and even within institutions from year to year (witness LSE).

In addition, I see a negative reputation effect, with the proportion of students from low-participation areas decreasing with increasing reputation. I'm basing this on name recognition. Perhaps UK readers can confirm if this is correct. If correct, it's a big miss in terms of interesting features in this dataset.

 

 

The state of charting software

Andrew Wheeler took the time to write code (in SPSS) to create the "Scariest Chart ever" (link). I previously wrote about my own attempt to remake the famous chart in grayscale. I complained that this is a chart that is easier to make in the much-maligned Excel paradigm, than in a statistical package: "I find it surprising how much work it would be to use standard tools like R to do this."

Andrew disagreed, saying "anyone saavy with a statistical package would call bs". He goes on to do the "Junk Charts challenge," which has two parts: remake the original Calculated Risk chart, and then, make the Junk Charts version of the chart.

I highly recommend reading the post. You'll learn a bit of SPSS and R (ggplot2) syntax, and the philosophy behind these languages. You can compare and contrast different ways to creating the charts. You can compare the output of various programs to generate the charts.

I'll leave you to decide whether the programs he created are easier than Excel.

***

Unfortunately, Andrew skipped over one of the key challenges that I envision for anyone trying to tackle this problem. The data set he started with, which he found from the Minneapolis Fed, is post-processed data. (It's a credit to him that he found a more direct source of data.) The Fed data is essentially the spreadsheet that sits behind the Calculated Risk chart. One can just highlight the data, and create a plot directly in Excel without any further work.

What I started with was the employment level data from BLS. What such data lacks is the definition of a recession, that is, the starting year and ending year of each recession. The data also comes in calendar months and years, and transforming that to "months from start of recession" is not straightforward. If we don't want to "hard code" the details, i.e. allowing the definition of a recession to be flexible, and make this a more general application, the challenge is more severe.

***

Another detail that Andrew skimmed over is the uneven length of the data series. One of the nice things about the Calculated Risk chart is that each line terminates upon reaching the horizontal axis. Even though more data is available for out years, that part of the time series is deemed extraneous to the story. This creates an awkward dataset where some series have say 25 values and others have only 10 values. While most software packages will handle this, more code needs to be written either during the data processing phase or during the plotting.

By contrast, in Excel, you just leave the cells blank where you want the lines to terminate.

***

In the last section, Andrew did a check on how well the straight lines approximate the real data. You can see that the approximation is extremely well. (The two panels where there seems to be a difference are due to a disagreement between the data as to when the recession started. If you look at 1974 instead of 1973, and also follow Calculated Risk's convention of having a really short recession in 1980, separate from that of 1981, then the straight lines match superbly.)

 

***

I'm the last person to say Excel is the best graphing package out there. That's not the point of my original post. If you're a regular reader, you will notice I make my graphs using various software, including R. I came across a case where I think current software packages are inferior, and would like the community to take notice.

A reader likes the four-point perception range chart

Note to readers: Sorry for the infrequent updates. We'll be back on schedule in February with exciting news.

***

Robert Kosara wrote a rebuttal to my previous post on the chart that shows the human's visual and audio ranges of perception in a box. Here is his full post. The chart under discussion is shown on the right. It appeared at the Abstruse Goose site.

Like me, he has obviously spent time thinking about four points on a chart. I have to say I'm not convinced by his points.

***

Kosara writes:

The point of this chart is not to communicate a lot of data or to inform, but merely to entertain and perhaps to make people pause and think for a moment.

I buy the first part of the sentence only. For me, the chart is misleading unless, as I said last time, we are told how much important stuff we are missing in the dark regions.

Think of this analogy: for some people, to realize that there are planets, galaxies and universes beyond our own is a wow moment. I need to know more. If you are told that no life exists outside planet earth, that the rest is barren and nothingness, are you still as fascinated?

***

Kosara likes the log-log axes, saying first:

That provides an interesting comparison, that I don’t think a lot of people have seen before.

Then:

The difference in light frequencies contains the sound frequencies many billions of times.

Then:

Our perception largely works in a logarithmic way.

My point was that light and sound are measured on completely different scales. To plot them in a bivariate chart would require some kind of standardization. I'd imagine if we can figure out the minimum perceptible difference for each dimension, we'd have made some headway.

That third comment really intrigues me. I have never liked log charts. I always find that audiences can't read them. They have to imagine that each layer is 10 times the size of the one below even though visually they appear exactly the same. In my experience, it leads to underestimating the large values, and massively exaggerating the importance of tiny differences on the small end of the axis. I'd be intrigued to see some scientific studies that show that logarithmic perception is natural.

 

 

Visualization as an analysis tool

Visualizing data has many uses. We often explore how charts can be used to convey data insights and tell stories. We talk less on this blog about how slicing and dicing data helps us form impressions about the structure of the data sets we're analyzing.

I have been digging around some payroll employment data recently. (You can find the data at the Bureau of Labor Statistics website.) I thought the following two charts are quite instructive.

The first one surfaces one type of recurring patterns: there is a seasonal pattern running from January to December that repeats every year. I use a small-multiples setup, with each chartlet indiced by year.

The second chart shows a different kind of regularity: there is a cyclical pattern running from 2002 to 2012, no matter which month we're looking at. Again, we have a small-multiples setup, this time with each chartlet indiced by a month of year.

This second chart is a simple form of "seasonal adjustment". The data used in this plot are unadjusted. The chart shows that there is a larger cyclical pattern during the period of 2002-2012 that affects every month of the year.

I already hear grumbling about using a line chart when there is no continuity from one dot to the next. In this chart, in fact, time runs left to right, top to bottom, then starts again at the first chartlet, and so on. This is a profile chart. As the name suggests, we should be focused on the shape of the line. It doesn't have to have physical meaning; we are only looking for regularity.

***

Statisticians love to find this kind of regular patterns because they are easy to describe. Of course, most data are much messier.

There's life without animation

Just as we don't always need a map to do justice to geographic data, we don't always usually need animation to convey time-varying data.

Some examples of good visualizations of time-series information without moving parts include the "horse-race" charts on the Presidential election (link), and the NY Times plot of Olympic race times (link).

Reader Jeff Cole did some nice Web charts (link) that show NFL matches as horse races. The look of these charts is exceptionally clean and easy to understand. They tell us which matches were blow-outs, and which ones were close, and which ones were tales of two halves.

 

This Green Bay-New England match looked like a thriller, with five lead changes, and a final score gap of only 4 points. Green Bay scored pretty much regularly through the four periods while New England had two relatively long droughts in scoring.

I'm not sure if the Margin of Victory section is worthwhile. It seems redundant to me. The labeling can be better, showing that New England leads are above the zero reference line while Green Bay leads are below the line. I'd consider making this a cumulative amount of time in the lead up to a specific point of the match. That would give an extra piece of information that is difficult to grasp from the Game Scoring chart.

This was an exciting match for a different reason. Dallas was always ahead and eventually won by three points. But it was a shootout in which each team scored regularly. Dallas had a scoring drought for most of the seond half which allowed Washington to get even but then scored a field goal before the final whistle to come out on top. I didn't watch the game but this chart tells me all of that.

On the Game Scoring chart, I'd add vertical tickmarks to help read the intermediate scores. Also maybe put dots to highlight the crossover points, which are where lead changes occur.

NFL is a complex game that is difficult to fully capture in a simple chart like this. It would be nice to add some extra event indicators, such as interceptions, fumbles, and other turning points. That's easier said than done, especially when trying to automate the graph production.

Four numbers say little, even on a busy chart

Reader Robert J. calls this a "really bad" chart (link). The data-ink ratio, he notes, is horrible.

The message of the chart can be stated in one or two sentences. And it's not clear what the other items are buying us. I usually love text annotations but three for this simple chart are too many.

***
The biggest issue I have are the axes. What is left unsaid is whether the inability to perceive outside the zone of human ability is inconvenient or not. What's missing is a histogram of the stimuli. I'd guess that the distribution is uneven, and there is a concentration inside the humanly perceptible zone. It would be helpful to include what type of stimuli exists at different frequency bands to illustrate what we are missing.

A different comparison that helps interpretation is other mammals. What is the range of perception of dogs, pigs, cows, etc.?

We should also think carefully before putting these two independent quantities onto the same chart. The rectangular region is merely a construct of the chart designer. In fact, the size of the rectangle is arbitrary as the scales on either axis can be made however large we want.

How to fail three tests in one chart

The November issue of Bloomberg Markets published the following pair of pyramid charts:

This chart fails a number of tests:

Tufte's data-ink ratio test

There are a total of six data points in the entire graphic. A mathematician would say only four data points, since the "no opinion" category is just the remainder. The designer lavishes this tiny data set with a variety of effects: colors, triangles, fonts of different tints, fonts of different sizes, solid and striped backgrounds, and legends, making something that is simple much more complex than necessary. The extra stuff impedes rather than improves understanding. In fact, there were so many parts that the designer even forgot to add little squares on the right panel beside the category labels.

Junk Charts's Self-sufficiency test

The data are encoded in the heights of the pyramids, not the areas. The shapes of the areas are inconsistent, which also makes it impossible to decipher. The way it is set up, one must compare the green, striped triangle with two trapezoids. This is when a designer realizes that he/she must print the data labels onto the chart as well. That's when self-sufficiency is violated. Cover up the data labels, and the graphical elements themselves no longer convey the data to the readers. More posts about self-sufficiency here.

Junk Charts's Trifecta checkup

The juxtaposition of two candidates' positions on two entirely different issues does not yield much insights. One is an economic issue, one is military in nature. Is this a commentary of the general credibility of the candidates? or their credibility on specific issues? or the investors' attitude toward the issues? Once the pertinent question is clarified, then the journalist needs to find the right data to address the question. More posts about the Trifecta checkup here.

Minimum Reporting Requirements for polls

Any pollster who doesn't report the sample size and/or the margin of error is not to be taken seriously. In addition, we should want to know how the sample was selected. What does it mean by "global investors"? Did the journalist randomly sample some investors? Did investors happen to fill out a survey that is served up somehow?

***

The following bar charts, while not innovative, speak louder.

Purists stay clear; many rules to be broken

This chart, from Internet Retailer (March 2012, p. 26), is okay, at least they didn't use pie charts. But it could have been much more effective.

To make it better, we have to break all the rules:

• Use lines instead of columns. The following reproduces the right side of the chart above which deals with shopping behavior by income groups. One of the issues with this chart is that the gray partition between the gender section and the income section is too incognito.

• Use rounded percentages instead of two decimal places. If we need to see two decimal places in order to tell two categories apart, then the difference is surely insignificant.

• Round up group boundaries. Notice that I committed the sin of imprecision by not specifying whether $60,000 belongs to $30-60K or $60-100K. Is that level of precision necessary? Is it worth the ugliness of printing a number like $59,999? If there is information in whether $60,000 belongs to $30-60K or $60-100K, what does that say about your analysis?

• Dare to throw away information. Look at this final version of the above charts:

I have collapsed the five income groups into two. That's because the bottom three income groups are more or less the same in terms of online/offline shopping behavior, and the top two groups are more or less the same. Bear in mind that any analysis of this type has a margin of error so differences of a few percentage points are not worth representing. It is possible that even the 10 percent or so differences between the two remaining income groups are not meaningful--we won't know unless we know the margin of error of their methodology.

(Note that you have to consult the Census to get the relative proportions of each income group in order to average the data shown in the original chart.)

In any case, the key message of the article becomes a lot clearer than in the original chart.

• Use black and white instead of color. Having simplified the presentation--but retaining all the important information--we no longer need colors.

Start breaking some rules today, and you'll make better charts.

When simple is too simple

Thanks to reader Don M, I came across this fascinating chart published in the New York Times Review recently (link). The main article, about gender segregation in job categories, is found here.

This is one of those charts that require a reader's guide.

The chart shows the proportion of women in each job category in year 1980 and in year 2010 (and nothing in between). The jobs are divided into three large chunks: the top chunk (shaded) consists of jobs in which women account for more than 70 percent of the total; the middle chunk (white background) are those jobs with 30 to 70 percent women; the bottom chunk (also shaded) are jobs with more than 70 percent men.

The designer then uses the red, green and gray colors (apologies to the color-blind folks) to group the jobs into three clusters. This is usually a great idea except that it is poorly executed here. Don is very annoyed with this because these colors lead the readers to the wrong conclusion, and I agree.

***

The color scheme is unnecessarily convoluted. Here is an alternative I prefer:

• if the change is 5 percent or less, color as gray no matter where the line is. (It is insane to color the line for housekeepers "red" for going from 87 to 89 percent in 30 years). 
• if the change is over 5 percent in the female direction, color it red to indicate the occupation is becoming more female. (There would be many red lines, such as for managers in education, HR staff, social workers, architects, etc.)
• if the change is over 5 percent in the male direction, color it blue to indicate the occupation is becoming more male (There would be only one blue line, and that is for welfare service aides.)

This would mean the lines for dentists and architects would be labelled progress. So too with most of the jobs that were predominantly male in 1980. In fact, there really isn't any occupation that went backwards--all those red lines in the bottom shaded chunk indicate shifts of only 1 to 4 percent, over 30 years!

This conclusion usurps the premise of the column in which the author claims that the conventional wisdom is wrong.

***

The other precaution in reading this chart is to realize that each occupation is put on equal footing in this chart even though some job categories employ a lot more people than others. Also confounded with this data is the differential growth/decline in job categories over the 30-year period. Further, the proportion of women entering the labor force must be accounted for.

This is a case in which less is less. The structure of the problem is complex, and it requires a more sophisticated approach.