What popular baby names teach us about data analytics

In my latest piece for Harvard Business Review (link), I tackle this common problem in the interactions between data scientists and business managers:

A typical big data analysis goes like this: First, a data scientist finds some obscure data accumulating in a server. Next, he or she spends days or weeks slicing and dicing the numbers, eventually stumbling upon some unusual insights. Then, a meeting is organized to present the findings to business managers, after which, the scientist feels disgruntled or even disrespected while the managers wish they could take the time back.

Using analyses of the popular baby names dataset as an example, I contrast the kind of analysis that generates click bait (e.g. the most "poisoned" names, the most "trendy" names) with the kind of analysis that generates potentially real business value.

More here.

Comments

[Ken]

This is in many ways similar to what happens in medical research especially epidemiology. People keep inventing hypotheses, they get results and have them published but they don't answer the questions that clinicians are interested in, and so are disregarded, especially if there are so many spurious results that nobody trusts them.

[Maternity Clothes UK]

Wow fascinating. Thanks for the insights.