« Lessons for communicating science emerging from the vaccine campaign | Main | Real-world studies: limits of knowledge »


Feed You can follow this conversation by subscribing to the comment feed for this post.


"If truly randomized, we can see balance on any variable known or unknown to the investigators. "

We can hope for balance, or we may have faith in balance.


Antonio: Like most of statistics, we put faith in probability distributions :) Seriously, when running RCTs, I always check the cohorts on balance, just to make sure we are not a victim of the rare event. This is still a controversial practice.

But since you highlighted this sentence, I'd like to draw readers' attention to the word "unknown". What is so powerful about RCTs, and why they are so much better than observational data, is that even unknown variables will be balanced. If, say, a new variant of the virus started circulating around - something that is completely unknowable at the time of design - we expect both arms of the RCT to be exposed equally to it while for an observational study, we have to investigate on a case by case basis.

Lastly, the caveat to this, just like anything else in statistics, is that we are talking about large enough sample sizes. Randomizing will not balance really rare conditions.


And I will raise you pointing to the word "any". :)
Could I state that, on average, among one hundred of unknown variables, I expect five of them to be not balanced (in "standard" terms)?

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.


Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name is required. Email address will not be displayed with the comment.)

Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

See my Youtube and Flickr.
Numbers Rule Your World:
Amazon - Barnes&Noble

Amazon - Barnes&Noble


  • only in Big Data

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here


R Fundamentals, Principal Analytics Prep

Numbersense: Statistical Reasoning in Practice, Principal Analytics Prep

Applied Analytics Frameworks & Methods, Columbia

The Art of Data Visualization, NYU

Signed copies at McNally-Jackson, NYC

Excerpts: Numbersense Ch. 1, 7, 8. NRYW

Junk Charts Blog

Link to junkcharts

Graphics design by Amanda Lee