In my latest piece for Harvard Business Review (link), I tackle this common problem in the interactions between data scientists and business managers:
A typical big data analysis goes like this: First, a data scientist finds some obscure data accumulating in a server. Next, he or she spends days or weeks slicing and dicing the numbers, eventually stumbling upon some unusual insights. Then, a meeting is organized to present the findings to business managers, after which, the scientist feels disgruntled or even disrespected while the managers wish they could take the time back.
Using analyses of the popular baby names dataset as an example, I contrast the kind of analysis that generates click bait (e.g. the most "poisoned" names, the most "trendy" names) with the kind of analysis that generates potentially real business value.
More here.
This is in many ways similar to what happens in medical research especially epidemiology. People keep inventing hypotheses, they get results and have them published but they don't answer the questions that clinicians are interested in, and so are disregarded, especially if there are so many spurious results that nobody trusts them.
Posted by: Ken | 04/06/2015 at 09:19 AM
Wow fascinating. Thanks for the insights.
Posted by: Maternity Clothes UK | 06/10/2015 at 03:04 PM