« Update: reviews, translations | Main | A significant mystery »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Cosma Shalizi

I can't bring myself to read the paper, but it sounds like they showed each subject multiple chart pairs, say k pairs, which would mean they had 20*k observations and not just 20. Still a hell of a multiple comparisons problem, though. Of course, the real problem here is that the experimental procedure can be fairly paraphrased as "someone with an ax to grind asked leading questions until he was happy with the answers".

Kaiser

Cosma: Not really. They are not randomizing treatment. Every one of 20 participants inspected both sets of charts. They should have used a paired-difference test but they didn't. The comparison is for the difference between the sum total of scores for chart type A and the sum total of scores for chart type B, replicated 20 times. So I think you have to call that 20 observations.

Jerzy

Are you sure? The paper says, "Each participant saw only one version of each chart, either Holmes or plain."

Kaiser

Jerzy: I know it's confusing because the way they set it up is super complicated. Each participant saw 14 charts, 7 are of the Holmes variety, 7 are of the plain variety. They alternate between Holmes and plain charts, and insert a blank chart to provide a "visual break".
So each participant is exposed to both "treatments" in alternating fashion.
However, for any given chart (better described as given data set), each participant sees only once. So in the above figure, you either saw the Holmes version or the plain version. But if you saw the Holmes version for the Diamond chart, your next chart would be the plain version.

Cosma Shalizi

So I think you have to call that 20 observations.

If that's what they did, then yes, it's just n=20. Wow (and not in a good way). "We can perhaps say what the experiment died of."

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Your Information

(Name is required. Email address will not be displayed with the comment.)

Marketing and advertising analytics expert. Author and Speaker. Currently at Vimeo and NYU. See my full bio.

Spring 2015 Courses (New York)

Jan 26: Business Analytics & Data Visualization (14 weeks) Info

Feb 23: Statistics for Management (10 weeks) Info

Mar 28: Careers in Business Analytics & Data Science (one-day seminar) Register

Apr 7: The Art of Data Visualization Workshop (6 weeks) Register

Next Events

Mar: 17 Finding and Telling Stories Using Data Visualization, Arlington, VA

Apr: 8 Princeton Association of New England, Boston, MA

Apr: 16-17 Data Visualization Workshop,, Digital Media Marketing Conference, St Louis, MO

Aug: 8-13 Joint Statistical Meetings, Seattle

Past Events

See here

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Search3

  • only in Big Data

Community