« When you don't need data | Main | Who's the richest person in the world right now? »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Dale

You say
"Andrew and Lauren suggest that the practice of using regression models without interactions is due to lack of data. It takes "16 times the sample size" to estimate one interaction effect relative to the main effect.

I get it when the observational data came from the professor's undergraduate class of 100 students but in Covid-19 studies, they have easily hundreds of thousands, if not millions, of subjects."

I get your point, but in reality those millions of data points are rarely available (especially for health data). In theory, the data exists, but any of the data sets I have available to me are much smaller than I like - and the inclusion of interaction effects is severely limited by that. With health data in particular, you would think interaction effects are especially important, so an unintended consequence of HIPAA is to limit the scope of interaction effects that can be included in observational studies.

Kaiser

Dale: Fair point. I may have a warped perspective as I've been reading Covid-vaccine-related studies and those seem to have access to government databases for the whole population.

In the paper, I also like this proposal: "When moving to analysis, interactions are important but hard to estimate with precision. So when we do include interactions in our model, we should estimate them using regularization and not demand that they attain statistical significance or any other threshold representing near-certainty."

Perhaps not recognized enough by practitioners is the fact that "not including interactions" is equivalent to "assuming no interactions".

Dale

I agree with your last sentence and it is well stated. Can you refer me to where you are getting Covid vaccine data at the individual level? I've been looking for that and can't find such data. In fact, the government was not collecting vaccination data on the case report forms until 8 months after vaccines were available - and the publicly available data sets do not include the vaccination data (even the case report data that is restricted does not include that). So, I've only seen aggregated vaccination data and I don't think you can estimate interaction effects from that - can you?

Kaiser

Dale: In our recently published JECP paper, we complained about this precisely. Unless the researchers release data, we can't explore these alternative models.

The comments to this entry are closed.

Get new posts by email:
Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

See my Youtube and Flickr.

Search3

  • only in Big Data
Numbers Rule Your World:
Amazon - Barnes&Noble

Numbersense:
Amazon - Barnes&Noble

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here

Principal Analytics Prep



Link to Principal Analytics Prep

Community