« Tip #1 for reading economic stats: pull apart threads | Main | Some comments on football statistics, why it is hard »


Feed You can follow this conversation by subscribing to the comment feed for this post.


Interesting post, and I agree with the conclusion. However, this would not work if the information is presented with a drill-down functionality. Drilling down will effectively reduce the sample size and will magnify the influence of errors/typos.
I frequently have to explain to customers that a report is produced from dirty data and they should be aware of it, to which they would typically remark that a few errors will note make a huge difference, and they are right. But a lot of BI tools allow easy drill-down, and that is where they have to be careful.


Dimitri: Absolutely, drilling down often presents problems, usually because the original research design did not anticipate analysis at those levels. This post does not deal with drilling down, however.


"Another is that statistical techniques by definition generalize the data, and thus are not very sensitive to individual values."

This statement is a little over the top. The robustness of a statistical technique to outliers varies a good deal between methods and across sample sizes. Least squares regression like you're doing here is actually quite sensitive to anomalous data points in many circumstances.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.


Post a comment

Marketing and advertising analytics expert. Author and Speaker. Currently at Vimeo and NYU. See my full bio.

Next Events

Mar: 26 Agilone Webinar "How to Build Data Driven Marketing Teams"

Apr: 4 Analytically Speaking Webcast, by JMP, with Alberto Cairo

May: 19-21 Midwest Biopharmaceutical Statistics Workshop, Muncie, IN

May: 25-28 Statistical Society of Canada Conference, Toronto

June: 16-19 Predictive Analytics World (Keynote), Chicago

Past Events

Feb: 27 Data-Driven Marketing Summit by Agilone, San Francisco

Dec: 12 Brand Innovators Big Data Event

Nov: 20 NC State Invited Big Data Seminar

Nov 5: Social Media Today Webinar

Nov: 1 LISA Conference

Oct: 29 NYU Coles Science Center

Oct: 9 Princeton Tech Meetup

Oct: 8 NYU Bookstore, NYC


Jul: 30 BIG Frontier, Chicago

May: 30 Book Expo, NYC

Apr: 4 New York Public Library Labs and Leaders in Software and Art Data Viz Panel, NYC

Mar: 22 INFORMS NY Student-Practitioner Forum on Analytics, NYC

Oct: 19 Predictive Analytics World, NYC

Jul: 30 JSM, Miami

Junk Charts Blog

Link to junkcharts

Graphics design by Amanda Lee


  • only in Big Data