Subjectivity

When I look at charts like this one, I ponder: Should graph designers adopt "objectivity" as practiced by American journalists?

Is it even possible to make "objective" charts?  Every design choice we make seem to chip away some of the detachment.  In this chart, the choice to order important web-site features by shopper -- rather than merchant -- ratings is a tacit preference for those ratings.  Bringing out key messages in the data is a subjective act, isn't it?

Are "objective" charts useful?  In our example, the design choices are kept to a minimum, and so it seems is its usefulness.  In comparing shopper and merchant ratings, one would be most interested in identifying the most effective web-site features as well as those features offered by merchants that find little resonance with shoppers-users.  These questions are better addressed by directly plotting the average rank and the ranking gap between merchants and shoppers (see below).

Notice that I said "ranking" rather than "rating".  The footnote discloses that the ratings were obtained from two different surveys conducted by two different companies at two different times.  How should we interpret the difference of 13% between the 89% of shoppers rating "Free Shipping" "very to extremely helpful" and the 76% of merchants rating "Free Shipping" "somewhat to very valuable"?

In the junkart chart, we can focus on three groups of features:

• the three top features ("Promo Discounts", "Free Shipping" and "Keyword Search") which attained the best average rank and least ranking gap;
• the three "orphan" features ("Recommended Products", "Top Sellers", "Gift Selection") created by loving web-site producers, abandoned by independent-minded shoppers;
• the three "neglected stepchildren" ("Shop the Catalog", "Store Locator", "Product Comparison") whose importance to shoppers were vastly underestimated by the merchants.

Unfortunately, while being "objective",  the data table fails to point out anything of interest to the reader.

Reference: "Consumers want one thing -- merchants are delivering another", Internet Retailer, Jan 2007.

Comments

[Gregor]

However, junkchart hides some data for points that are not so interesting for the creator. But reader might be interested in those points as well. Cluttering the plot with all labels is not an option. So, this plot really is subjective ;)

[SilentD]

Of course it's not possible to make "objective" charts, any more than its possible to write "objective" news stories.
In all communications, we make choices about what to put in, what to leave out, how to arrange items, etc., all of which have an impact on the resulting message. Even if it were possible to craft an objective message, the choice of subject matter brings us right back to subjectivity. To paraphrase O.B. Hardison, all maps are lies, what we need are maps that tell us the lies we need to know.

[Kaiser]

Gregor: that was the point. My plot draws attention to the data I found to be most interesting. Presumably, other people may be interested in the unstressed data.

SilentD: agree that true objectivity is a mirage. In this post, I want to bring up the question of whether we should make our charts "as objective as possible". In creating my version of the chart, I knowingly applied my subjective interpretation of the data. If objectivity was a worthwhile goal, a different chart would have been preferred!

[Chris]

What is worse to me is that they do not talk about the survey methodology. These look like stated importance, but it would be a whole lot more valuable to look at derived importance. After about 8 items in the list, my eyes just glaze over.

[Karl K]

Kaiser, of course we should try to make all of our charts, tables, memos, expository writings, ideas, and concepts as "objective" as possible -- or rather, as "truthful" as possible...while at the same time recognizing that we are not perfect, and perfect objectivity is beyond us.

No, what we want in our analysis of information is the exercise, to the best of our ability, of sound rational thinking to get us as close to the truth as we can. Make your ideas transparent, and then let the chips fall where they may as others review them and analyze them in turn.

Sort of what you do on this blog, eh?

[Georgia Sam]

Your critique of the table is spot-on, as usual. Regarding subjectivity vs. objectivity more generally, of course it is impossible to make a chart or anything else completely objective. On the other hand, it's invalid to argue that since total objectivity is impossible, all attempts to convey accurate, unbiased information might as well be abandoned. It is perfectly possible for a person to produce text or a picture that is far less biased than he or she is. Moreover, the impossibility of ever completely achieving a goal does not mean it isn't worth striving toward. The question is, which direction are we going: toward the goal or away from it?

[Kaiser]

The troubling thought is that if we live in the zone between absolute objectivity and absolute subjectivity, then any chart is open to criticism of bias because what I find informative or interesting is not likely to be the same as you, especially when faced with large data sets.

[Karl K]

Kaiser, I bet you only find it troubling because you used "bias" as a pejorative. I'll admit to a bias right now...a bias for the truth.

Heck, you know this better than anybody: good charts should be designed to illuminate information, and convey important, significant insights about information and data. They should aid in understanding, and then, in rational decison-making.

When you create a chart, you are sometimes creating an argument about the data. As such, charts are subject to the same basic rules of logic, and are prey to the classic logical fallacies if you're not careful. And some arguments are really really good (though not necessarily perfect). And some arguments are really really bad.

Charts, though, are often one step removed from an argument. You may have a sound argument, but really piss poor data presentation (think of Tufte's Challenger discussion). And the converse may be true -- you may have great charts, but lousy thinking.

Bottom line, I don't think the issue of subjectivity/objectivity is that big of a deal. Tell me and show me what you think; I'll do the same. Let's hammer it out. If we're both smart, and both after the truth, we're gonna be lots better off, even if we aren't totally objective.