Junk Charts

Motion-sick, or just sick?

Reader Irene R. was asked by a client to emulate this infographic movie, made by UNIQLO, the Japanese clothing store.

Here is one screen shot of the movie:

This is the first screen of a section; from this moment, the globes dissolve into clusters of photographs representing the survey respondents, which then parade across the screen. Irene complains of motion sickness, and I can see why she feels that way.

Here is another screen shot:

Surprisingly, I don't find this effort completely wasteful. This is because I have read a fair share of bore-them-to-tears compilation of survey research results - you know, those presentations with one multi-colored, stacked or grouped bar chart after another, extending for dozens of pages.

There are some interesting ideas in this movie. They have buttons on the lower left that allow users to look at subgroups. You'll quickly find the limitations of such studies by clicking on one or more of those buttons... the sample sizes shrink drastically.

The use of faces animates the survey, reminding viewers that the statistics represent real people. I wonder how they chose which faces to highlight, and in particular, whether the answers thus highlighted represent the average respondent. There is a danger that viewers will remember individual faces and their answers more than they recall the average statistics.

***

If the choice is between a thick presentation gathering dust on the CEO's desk and this vertigo of a movie that perhaps might get viewed, which one would you pick?

 

Largest ever difference between 328 and 327 spotted in New York City

This, according to Dan Goldstein, who found the following chart:

For bar charts, one must start at zero... no exceptions. Chopping off arbitrary amounts of the base can create different perceptions of difference.

Here is another example.

Reading behind the chart

I could have filed this one under Light Entertainment but it's too good a chart to lay to waste:

 
(The chart is from Internet Retailer.)

Let's focus on the (mis)match between the question being addressed and the data collected to address it. The intention of the analyst is fully divulged in the title of the accompanying article: "Don't sell Twitter short: those 140-character messages reach an affluent and engaged audience". The chart supports this claim by showing that Twitter users disproportionately represent the types of consumers that marketers most covet, i.e. those with advanced education, and those earning higher incomes.

***

Up top, the concept "user" is very pliable. Is it someone who has a Twitter feed or is it someone who reads Twitter feeds or is it someone who subscribes to Twitter feeds? Is it someone who is registered or everyone who visits? Is it someone who has visited the site in the last x months? or posted a tweet in the last x months? If a writer, does it include someone with no subscribers? no page views? What about people who simultaneously publishes multiple feeds (like John Cook who writes one of my recommended blogs)?

Now, we don't expect the analyst to describe fully how a "user" is defined but while interpreting this chart, we should ask appropriate questions of the data.

Next, the analyst establishes a reference level called "general population". Is this the right metric? This depends on what the chart is used for. If you are choosing between spending money on Twitter, and say spending the money on national TV advertising, then perhaps this comparison is valid. If you are selecting between Twitter and say Google, then absolutely not. For most readers, I think a more relevant point of comparison would be the general Web user, rather than the general population. This is an important distinction because the general Web user also earns higher incomes and has higher educational attainment than the general population, thus the current set of data exaggerates the "value" of Twitter exposure.

Finally, if you are a marketer looking to spend with Twitter, you are also worried about "reach". Say, 50% of website ABC's users are rich people compared to 10% of your reference population. That sounds like an amazing opportunity. Well, only if website ABC has sufficient number of users! If ABC is a niche website serving only 1% of your reference population, then despite the benefit of targeting, its scale is too small for your need.

Don't take a chart on its surface. Read behind the chart!

 

 

Experiments in simplification

Julien D. sent us to this link, where a design agency picks up everyday objects and investigates what happens if the designs were to be simplified. Here is Nutella simplified:

I reckon this should be wonderful inspiration for chart designers.

Take your final design, remove components and interrogate whether you need those components.

(Related posts: self-sufficiency.)

Rightsizing: the graph edition

Reader Alex C. alerted me to this sensible note from Allan Reese, complaining about a piece of marketing by a software vendor (Aptech Systems), shown here:

This graph purportedly demonstrates the power of 3-dimensional plots, which presumably is a feature of the software GAUSSplot. As Reese pointed out, it rather unfortunately demonstrated the weaknesses of 3D plots, and it did so amusingly well.

Reese: "Apart from a Daliesque charm as abstract art, I can see nothing professional or commendable in this graph."

Yes, every decision rasps. Reese noted that the chart is named a "3D contour plot" by the vendor and yet we see a paired column chart arranged in an L-shape. The diskettes indicate the average of the 3 column values, which serves to obstruct our view of the underlying data. The legend--which is a palette of colors--plays the redundant role of gridlines. The diskettes take on various colors but the label shows it in orange. The two identical axis labels run in opposite directions, with a third running horizontally atop the color legend. The title announces a comparison of natural and synthetic fabrics, which explains how the eight fabrics were divided into two groups but will be missed easily. We surmise that the chart designer typically reads from top to bottom, right to left given the orientation of the fabric categories.

What most disturbs Reese--and surely anyone who is informed of it--is the optical illusion rendered by the use of three different shades of gray for the three panels. The chart literally creates two duelling images, the cube with rainbow strips crawling all over it, and the corner with two walls with rainbow columns sticking up from the floor. The second image, the intended one, is unstable because it would be hard to create a lighting scheme that would render one wall dark while the other wall is lighted.

***

The following 2D dot plot has no razzle-dazzle but makes the point.

I realized the grouping by synthetic v. natural late and just decided to box the synthetic ones. One can certainly make this a two-panel chart with the synthetics on the left and the naturals on the right.

The use of only 3 samples is highly questionable. This chart shows that the only reasonable conclusion is that Acrylic and Nylon have higher burn times than the rest. With so few samples, it is hard to tell if the remaining fabrics are truly different.

***

There may be situations where a 3D chart is preferred to a 2D chart but this set of data is certainly not such a situation. The software may in fact produce great 3D charts but this particular chart does not show off the software as the marketers may have hoped. One of the designers most important task is to examine the structure of the data, and "right-size" the chart-- throwing in extra dimensions is often counter-productive.

 

 

 

Weekend reading

Lots of ideas from readers have been gathering dust in my mailbox. Here are a bunch of links, with a few comments of mine.

***

This first link I'm not sure what to make of. I think the architects and graphic designers amongst you may be able to make sense of it. Not me. It came with this description: "dr. dr. crash and dr. trash of m-a-u-s-e-r
analyzed worlds most junk magazines and visualized their data." For the intrepid (and I claim no liability):

    "Jetistics: The Analysis of Junk. The Junk of Analysis?"

***

This is yet another instance of the trend of infographics infiltrating PR releases.

This is yet another example of a map adding little or no value to the data. The presence of geographic data is not an excuse to give a lesson on maps.

It would be one thing if the geographic location helps the readers understand the data but in most such charts, the map merely says "Reader, I presume you are map illiterate, so let me tell you  South Africa is at the southern tip of the African continent..."

Also notice that the bar charts are sorted by average size of invoices, which is definitely less meaningful than total amount invoiced. This, I suspect, is the failure to ask the pertinent question, which is at the top of the Trifecta checkup.

  

  ***

"19 Must-See Biotech Infographics", according to Kelly Davis of the BioBlogging Project.

  #2 on this list is a chart (rather old data) on GM food, an issue of concern to me. In the Trifecta checkup, this addresses an important question, and displays very relevant data but uses a poor chart... too many colors, colors not carrying any meaing, hard-to-read labels.

Of the other links, these are more interesting: #10, #12, #17, #19, #8, #9.

 

 

***

Lessons from propaganda

Political wordsmith (euphemism) Frank Luntz's presentation is all over the Web. I saw it on Business Insider. In the debate between words and numbers, Luntz obviously takes the side of words.

He used a few simple charts in the presentation, which is interesting by itself since he fundamentally is a words guy, not a numbers guy.

The charts, while simple, are very instructive:

This bar chart sent me running to the maligned pie chart!  (Almost, read on).

While the total responses were almost evenly split between the three choices, the bar chart drew our attention to the first bar, which is inapt.

If plotted as a pie chart, I thought, the reader would see three almost equal slices. This effect occurs because we are much less precise at determining the areas of slices than the areas of bars.  Wouldn't that turn our usual advice on its head?

 

How the Bar Chart is Saved

The one thing that the pie chart has as a default that this bar chart doesn't is the upper bound.  Everything must add up to 100% in a circle but nothing forces the lengths of the bars to add up to anything.

We save the bar chart by making the horizontal axis stretch to 100% for each bar.  This new scaling makes the three bars appear almost equal in length, which is as it should be.

Another Unforgivable Pie Chart

On the very next page, Luntz threw this pie at our faces:

Make sure you read the sentence at the bottom.

It appears that he removed the largest group of responses, and then reweighted the CEO and Companies responses to add to 100%.

This procedure is always ill-advised - responders responded to the full set of choices, and if they were only given these two responses, they very well might have answered differently.

It also elevated secondary responses while dispensing with the primary response.

Infographing the cost of iPad

The cost of the iPad gets the infographics treatment here.

I feel a little weird about featuring this item.  Helen E., who created the chart/poster, urged me to write about it. The link seems to connect to a commercial site but doesn't look too commercial -- and since the iPad fever is upon us, I thought why not.

There are two elements on this poster that qualify it as data graphics.

The surreptitious blue bars that are sized to match the inflation-adjusted prices... except for the Apple Lisa when the bar was chopped prematurely.

And the image equation gimmick at the bottom.

***

The 43 iPads = 1 Lisa visualization is definitely effective. I'm not so sure if anyone should care about this particular comparison though.

The blue bars are a super, light touch on a chart that could be otherwise quite boring. The varying heights of the bars exaggerate the larger prices, and can cause confusion.  For example, Apple III, costing $11,412.88, is a midget next to Macintosh Portable, costing $11,358.59.

The choice of which products to feature images and bolded text appears somewhat arbitrary... are those keystone products? Nice use of foreground/background though.

***

I sent a note back to Helen about the (ab)use of decimals, indicating that dropping decimals and rounding off to the nearest $10 would improve readability.

She replied, saying "moving forward, they will be a little cleaner without such a paranoid focus on accuracy".

She explained further:

We were really concerned with accuracy as we knew that the Apple fan base would be tough on us if our calculations were even a little out.

So legendary Apple fanboys, take your decimals with you on your way out!

Off-sheet accounting

For mid-week entertainment, this full-page ad appeared in the Wall Street Journal recently:

The vertical axis says "% NYSE of All Market Share Volume".  The time-line is from July 05 to beyond July 08.  The text in the black box is "Matched Market Share: July 11, 2008".

When it's so obvious, it's probably not obvious.  The big story is off the chart: what happened to the other 50% of the volume over the years?

Faced with this, one reaches for the pie chart (... almost).

Small add (8/21/2008):

Water and wine

Marketers have always argued that price signals quality; this leads to the startling idea that one should just set a high price. 

If you don't believe it, note how Coca Cola and Pepsi turned tap water into a premium-priced $1.7 billion market.  As we now know, Dasani and Aquafina are just bottled tap water.

Even if one can turn water to wine, now researchers discovered the same rule applies.  Unlike most scholarly articles, they actually published a well-made chart to illustrate their experiment.

Testers were given the same wine but told either it cost $10 or $90.  Their brain activity is measured.  The chart showed that those thinking it cost $90 (green line) had much better sensation about the wine than those thinking it cost $10 (blue).

A standard way to display this information is a data table that spells out every estimate and its standard error, plus some asterisk or bolding scheme to indicate statistical significance.  Visualization is far superior.

For more examples, see Gelman's paper or Kastellec and Leoni's paper.

Reference: "Study: $90 wine tastes better than the same wine at $10", News.com, Jan 14, 2008.