Peek into beauty 2

Jeff W made some astute comments on the New York Times Netflix visualization, which I praised in the last post.  He pointed out that there is so much more to the underlying data than what can be shown within the confines of these maps.  For example, he wanted to know the relationship between Metacritic scores and Netflix ranks (or rentals), explore the heavy-tailed distribution of titles, expose regional differences, etc.

What he is hitting on is the shortcoming of the current approach to infographics... an approach which is about putting order to messy data, rather than summarizing, extracting and generalizing.  And it is also the difference between "data graphics" and "statistical graphics".

This is related to the modelers versus non-modelers dichotomy Andrew Gelman just discussed in this blog post.  (He cites Hal Stern as the source of the quote.)

Basically, non-modelers have the same philosophy as infographics designers - they want to make as few assumptions as possible, to rely exclusively on the data set.  By contrast, modelers want to reduce the data, their instinct is to generalize.  The stuff that Jeff wanted all require statistical modeling.  As I mentioned before (say, here), I believe infographics has to eventually move in this direction to be successful.

Take the correlation betwen Metacritic score and Netflix ranking... the designers actually thought about this and they tried to surface the correlation, in a way that is strait-jacketed by the infographics aesthetics.  What they did was to allow the movies to be sorted by Netflix ranking, or by Metacritic score, using the controls on the top right.  And when the Netflix ranking is chosen for sorting, the Metacritic score is printed next to the map, so as the reader scrolls along, he or she can mentally evaluate the correlation.  Of course, this is very inefficient and error-prone but we should give the designers props for trying.

Building a model for this data is no simple matter either because multiple factors are at play to determine the Netflix ranking.  A good model is one that can somewhat accurately predict the Netflix ranking (color) based on various factors included in the model, such as the type of movie, the cost of movie, the number of screens it's played, any affinity of a movie to a locale (witness "New in Town"), regions (at different levels of specificity), recency of the movie, whether it's been released on multiple format, etc. etc. 

Jeff's other point about ranking vs number of rentals raises another interesting statistical issue.  I suspect that it is precisely because the number of rentals is highly skewed with a long tail that the analyst chose to use rank orders.  If an untransformed number of rentals is used, the top few blockbuster films will dominate pretty much every map.

Keep the comments coming!

Comments

[Jeff Weir]

Thanks for picking up my points.
Sometimes it’s bad to be a chart junky, because our addiction forces us to focus on the merits of the chart as presented. We should make sure that we critique not just a chart but also how else the information might be presented, the pedegree of that information, and alternate sources of information that might add more insight.

There’s been 3 good examples on 3 of my favourite blogs in the last week:
*This example
*A chart at Ajay’s Databison blog over at http://www.databison.com/index.php/volatility-chart-in-excel-lessons-in-chart-design-from-ny-times/
* A chart critiqued in Chandoo’s PHD blog over at http://chandoo.org/wp/2010/01/15/flu-trends-chart-review/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+PointyHairedDilbert+%28Pointy+Haired+Dilbert%29&utm_content=Google+Reader

In all three cases, the charts look pretty cool. But on deeper reflection, the charts could be made either way more interesting and/or more easily interpretable. And there’s statistical problems with two of those charts that mean it could be dangerous to draw conclusions from them.

Regarding this particular chart, there’s a few great comments on the Times’ site that get to the heart of what I'm saying. My favourite : “How can the Times give so much bandwidth to this?” I agree. To paraphrase Tufte, the data is not the most interesting data.

Another comment reads “This is a strange feature for the Times, with no analysis or context to guide the reader. Would be nice to have some sort of commentary from a sociologist type. The conclusions to be drawn from this are....what?” I couldn’t agree more.

One commenter who's obviously a chart junky said “This is brilliant work”, followed immediately by “I cannot tell what numbers of rentals there are, or how significant a very-light color is. It would be useful to know how many rentals are at the lowest bounds: does one rental in a neighbourhood make it appear, or several? “

So what’s brilliant about it? To my mind, brilliant execution + crap message <> brilliant graphic. If the message is crap, the graphic is crap REGARDLESS of the execution.

On presentation, I’ve seen quite a few comments praising graphics/graphs saying stuff like “The colours fit in with the approach that Tufte/Few/Picasso advocates concerning colours/data ink/scale”. But very often those people miss the big picture: the graphic completely misses the APPROACH that the quoted expert would advocate.

I’m not against creative ways to express data…unless they are not the BEST way to display the data.

[Danny Dougherty]

The Society for News Design has a good interview with a designer who worked on the piece. He talks not just about process but some of the design trade offs made in putting together the graphic and about the slightly different print version that appeared in the Metro section of the NYT: http://www.snd.org/2010/01/nyt-netflix-graphic/

[Jeff Weir]

Stephen Few has a great post at http://www.perceptualedge.com/blog/?p=672 on accessing the effectiveness of a new dashboard's design. Many points are applicable not just to dashboards but to stand-alone graphics...certainly if graph/graphic designers followed the "Specific Parts of the Dashboard" part then we'd have much more meaningful graphs.

[Jeffrey Weir]

Thanks for the link, Danny.

I see that it was Netflix that withheld the "number of people renting" metric in favour of "rank" for commercial reasons. Fair enough, then. Still, this didn't preclude the designer from looking at the relationship between Metacritic scores and Netflix ranks. Or looking at how closely correlated the top 50 lists were between zip codes Although this would probably be beyond the scope of the NYT...it takes a lot more number crunching, and ultimately their mission is to provide fodder for the eyeballs of the masses.

I see that they ran a full-page graphic in print that had more statistical analysis. I’d be interested to see this.

Another thing I ponder on about this graphic is why you would need an ‘Alphabetical’ filter. Alphabetically makes no sense unless you want to find a particular movie by title rather than by popularity or metascore. But in that case, it would be easier if a user could select the movie title they are after directly from a dropdown list, rather than select the ‘Alphabetical’ filter and then move the slider to find the title they are searching for.

[Peter Holmdahl]

I'm looking for (to read tomorrow or before :) something about visualizing/comparing matrices. They could be for instance a month where each day have 5 values (date is one dimension, location is the second, and the actual value is the third). And I want to compare the values of 2-5 of these.
Any reading tips anyone? :)

[Kaiser]

You should consider a "heat map", sometimes called a "cell plot". Imagine a grid of squares, each square correspond to a given (date, location) pair. You just pick colors according to the third value. This is a great pattern recognition tool. Put several of these side by side and you can tell the patterns easily, if they exist.

[Peter Holmdahl]

Thanks Kaiser! Yeah, heat maps are nice, but kinda hard to compare value-to-value between two or more matrices, I think.

[curtis johnson realty]

This is a very interesting post. I have to say that focusing on the rental business must have a exact info and background pertaining to it. I really appreciate the effort of posting it so that many will be inspired and can get useful information about rentals. More power.

[shopping bags]

in contrast good old fashioned soap can be identified by containing saponified oil of ____, that is an oil (or fat) reacted with either sodium hydroxide (aka caustic soda or lye) or potassium hydroxide (caustic potash or potash lye.) while the oil in question is consumed by the saponification process, many small soap makers use extra fat beyond what will convert into soap to make a milder more moisturizing soap or shampoo. and of course they may also add oils and other ingredients for their beneficial qualities separately, after the reaction is completed.pilipalagaga 0307