« Reading guides for instructors, book clubs, etc. | Main | What can we learn from Freakonomics? »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Jonathan

Kaiser do you think this is a fundamental problem with RCTs? I am a huge supporter of them, but it seems if they are done in massive amounts of quantities you will continue to find (X, Y, and Z) as a cause for the outcome. I guess what I am really asking is, when do we stop investigating the claim?

Bruce Stephens

This wasn't, of course, a RCT. IIUC almost no such nutritional studies are RCTs. It was observational (epidemiological).

With RCTs I guess complaint 1 still stands: if you're studying one outcome then you're likely to end up presenting that one outcome and perhaps ignoring others. And the meta-complaint still stands, that the statistics tend to be presented in relative risk terms which (IIUC) are well accepted as being one of the worst ways to communicate such statistics. Complaints 2 and 3 would usually be avoided, presuming everything goes well (the study is effectively blinded, sufficiently large, etc.).

Kaiser

Let's talk about RCTs. RCT is typically used in clinical trials to test new drugs. As Bruce pointed out, complaint #1 applies to clinical trials; you keep hearing that the trials are not designed to detect potential harms because the primary goal is to detect potential benefit. Complaint #2 also applies because typically a drug cures one thing (say, reduce cholesterol) but the disease (heart disease) has multiple causes.

Complaint #3 is partially offset by the randomization - the point of randomization is to balance the unknown factors that may be skewing the outcomes. However, randomizing doesn't help the interpretation of regression coefficients. Controlling for other variables means that we have assumed the existence of someone who has average values of all the control variables. That person probably doesn't exist in the test sample.

Jeff Weir

RCT. IIUC. WTF?

We need a self-sufficiency test for comments, gents.

Jeff Weir

Red meat surely has some benefits. So we must balance both the benefits and the harms in order to decide how much to eat.

Fair enough. But I don't take issue with the headline "Study gives more reasons for passing on red meat", because it's not incorrect...the study did in fact give one more reason to pass. There's still plenty of reasons not to pass, like for instance the obvious enjoyment many readers of the study get from eating it (the meat, not the study!)

THis headline certainly doesn't say "Red meat is on balance bad for you". It's just prompting the reader to revise their intake to another indifference curve given this one more bit of information to hand tips the scale a little away from meat. That's how markets work, isn't it? And that's how the null hypothesis works, isn't it? It's obviously much more possible to quantify one well-defined harm than the entire possible range of harms and benefits to form a robust opinion of net benefit.

"Each extra serving was also tied to a 16 percent higher chance of dying from cardiovascular disease". If this was so, with 5 extra servings, we'd all be dead.. Without being familiar with the study, this isn't the only way you can read this stat...they might mean tha for each extra serving, your risk increases by 16% relative to the last level of risk.

Kaiser

Jeff:

RCT = randomized controlled trials (wiki)

this is a setup typically used to test the effectiveness of a new drug. in the simplest case, a group of patients known as controls are given a placebo or the prevailing treatment while a separate group gets the new drug. Assignment of any given patient to one of the two groups is by random lottery. The beauty of random lottery is that it equalizes all other factors including unknown factors. Real life is not so simple because many factors we'd like to study cannot be randomly assigned e.g. you can't force someone to smoke. Also, patients and/or doctors may decide to ignore the protocol, say if the traditional treatment is failing for someone in the control arm, there is a temptation to "cross over" and give the patient the new drug.

Jeff Weir

Cool. I just googled IIUC, and ironically now I do!

Hey, you got a comments RSS feed for this baby yet that readers can subscribe to? I'd like to follow the action in the Recent Comments bit by RSS...heaps of good learning here (and not just about internet slang).

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Marketing and advertising analytics expert. Author and Speaker. Currently at Vimeo and NYU. See my full bio.

Next Events

Mar: 26 Agilone Webinar "How to Build Data Driven Marketing Teams"

Apr: 4 Analytically Speaking Webcast, by JMP, with Alberto Cairo

May: 19-21 Midwest Biopharmaceutical Statistics Workshop, Muncie, IN

May: 25-28 Statistical Society of Canada Conference, Toronto

June: 16-19 Predictive Analytics World (Keynote), Chicago



Past Events

Feb: 27 Data-Driven Marketing Summit by Agilone, San Francisco

Dec: 12 Brand Innovators Big Data Event

Nov: 20 NC State Invited Big Data Seminar

Nov 5: Social Media Today Webinar

Nov: 1 LISA Conference

Oct: 29 NYU Coles Science Center

Oct: 9 Princeton Tech Meetup

Oct: 8 NYU Bookstore, NYC

Sep: 18 INFORMS NYC

Jul: 30 BIG Frontier, Chicago

May: 30 Book Expo, NYC

Apr: 4 New York Public Library Labs and Leaders in Software and Art Data Viz Panel, NYC

Mar: 22 INFORMS NY Student-Practitioner Forum on Analytics, NYC

Oct: 19 Predictive Analytics World, NYC

Jul: 30 JSM, Miami

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Search3

  • only in Big Data

Community