Nick Cox commented on Aleks's post on Andrew's blog on whether chartjunk is better than Tuftian-style charts. Nick made the point that charts should be subjected to popularity contests (my paraphrase). He seemed to say all of statistical science, perhaps all of science, should be fair game for studies of popular opinion.
Interestingly, Aleks's post was about a popularity contest of a different kind: he alerted us to a research paper using Amazon's Mechanical Turk to assess different types of charts. I haven't had time to review this paper yet but it sounds more promising than the 20-sample experiment. When I read that paper, I will hold my doubts about "crowdsourcing", "prediction markets", and such like.
Back to Nick's comment. But a lot of science have already been turned into popularity contests in recent years - evolution, climate change, causes of autism, bird flu vaccines, etc. have all become politicized, with scientists kicked to the curbside. Based on these results, I wouldn't recommend it.
Nick also pointed out that the "chartjunk is better" paper won the best paper award at the conference where the researchers presented it. I think it said a lot about the lack of statistical expertise on the judging panel more than anything. I stated in my original review that this sort of research is useful and necessary, but it ought to be done with higher standards.
My prior posts on the chartjunk paper:
8 Red Flags ...