« Lessons for communicating science emerging from the vaccine campaign | Main | Real-world studies: limits of knowledge »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Antonio

"If truly randomized, we can see balance on any variable known or unknown to the investigators. "

We can hope for balance, or we may have faith in balance.

Kaiser

Antonio: Like most of statistics, we put faith in probability distributions :) Seriously, when running RCTs, I always check the cohorts on balance, just to make sure we are not a victim of the rare event. This is still a controversial practice.

But since you highlighted this sentence, I'd like to draw readers' attention to the word "unknown". What is so powerful about RCTs, and why they are so much better than observational data, is that even unknown variables will be balanced. If, say, a new variant of the virus started circulating around - something that is completely unknowable at the time of design - we expect both arms of the RCT to be exposed equally to it while for an observational study, we have to investigate on a case by case basis.

Lastly, the caveat to this, just like anything else in statistics, is that we are talking about large enough sample sizes. Randomizing will not balance really rare conditions.

Antonio

And I will raise you pointing to the word "any". :)
Could I state that, on average, among one hundred of unknown variables, I expect five of them to be not balanced (in "standard" terms)?

The comments to this entry are closed.

Get new posts by email:
Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

See my Youtube and Flickr.

Search3

  • only in Big Data
Numbers Rule Your World:
Amazon - Barnes&Noble

Numbersense:
Amazon - Barnes&Noble

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here

Principal Analytics Prep



Link to Principal Analytics Prep

Community