Big Data, Plainly Spoken (aka Numbers Rule Your World)

Thank you Buzzfeed for noticing us humans

My friend John R. sent me this excellent Buzzfeed feature on music playlists.

Here are some choice quotes to whet your appetite:

In 2014, when Tim Cook explained Apple’s stunning $3 billion purchase of Beats by repeatedly invoking its “very rare and hard to find” team of music experts, he was talking about these guys. And their efforts since, which have pointed toward curated playlists (specifically, an industrial-scale trove of 14,000 and counting) as the format of the future, have helped turn what was once a humble labor of love for music fans into an increasingly high-stakes contest between some of the richest companies in the world.

The algorithm that can judge the merits of new Gucci Mane, or intuit that you want to sing “A Thousand Miles” by Vanessa Carlton in the shower, has yet to be written... the job has fallen to an elite class of veteran music nerds — fewer than 100 working full-time at either Apple, Google, or Spotify — who are responsible for assembling, naming, and updating nearly every commute, dinner party, or TGIF playlist on your phone.

Spotify says 50% of its more than 100 million users globally are listening to its human-curated playlists (not counting those in the popular, algorithmically personalized “Discover Weekly”), which cumulatively generate more than a billion plays per week.

Machines have always been great at repetitive tasks that follow set rules. But many problems do not fall into that category.

We’ve come to expect that virtually all of our problems can be solved with code, so much so that we summon it unthinkingly before doing almost anything...But what if music is somehow different? What if there’s something immeasurable but essential in the space between what is now called “discovery” and, you know, that old stupidly human ritual of finding and falling in love with a song? 

It's the revenge of the humans. Recommendation engines are not good enough. This doesn't mean "science" is not important. The article later explains:

Hypotheses, of course, are meant to be tested, and Spotify curators regularly make adjustments to playlists based on data that shows how people are actually interacting with them.

One frequently used application is a performance tracker called “PUMA,” or Playlist Usage Monitoring and Analysis, which breaks down each song on a playlist by things like number of plays, number of skips, and number of saves.

This is really the way forward for "machines". Machines and humans are both needed: the sum ought to be greater than its parts. Forget the idea that one replaces the other.

 

 

Quote of the month

In a post titled "GIGO" (for those who don't like acronyms, Garbage In, Garbage Out), Andrew Gelman wrote this gem:

as long as the “garbage out” gets media attention, there will always be somebody willing to supply the “garbage in.”

The general drift of that post, and the previous one that led me to it, is a critique of the management consulting industry. Having worked in that industry earlier in my career, plus having studied at the school at which Michael Porter is a famous dude, I am familiar with how this industry works.

We even had a term called "blank sliding." This is exactly what you think it means... consultants are trained to write a "draft" of a client presentation consisting of all blank slides that only have headers written on them. Headers taken together tell the story of the presentation. Eventually, the slides are filled in with content. In a nutshell, the story is created before the meat is ready.

While this sounds horrible, it is not really that bad, if used properly. It creates a culture of getting the story right and focusing a lot of attention on the story. If the analyst uses this method directionally, allowing rewriting of headers, or maybe even the whole story, as the contents are filled in, it could lead to a good product. Frequently, however, it creates the situation described by the Gelman quote.

 

PS. Andrew is probably right about Michael Porter being paid big bucks. He is one of the most expensive public speakers (link).

GMO labeling is good science

A GMO labeling law has arrived in the US, albeit one that has no teeth (link). For those who don't want to click on the link, the law is passed in haste to pre-empt a more stringent Vermont law. The federal law defines GMO narrowly, businesses do not need to put word labels on packages (they can, for example, provide an 800-number), and violaters will not be punished.

One of the arguments against GMO labeling is that it is unscientific because (some) scientists are 100% certain that GMO foods are safe. (e.g. this Boston Globe editorial)

Any good scientist knows that scientific "truths" are true until they are proven otherwise. Science is a continuous process of making hypotheses, and finding data to confirm or reject them. The Bayesian way of thinking is very useful here. Being true is a probability - more confirmatory data increases the probability that a given hypothesis is true.

So why is GMO labeling good science?

In fact, I'd go so far as to say that there is no science without GMO labeling.

How is nutritional science done today? What is the research that tells us coffee is good, butter is good, salt is bad, etc.? Granted, this is a shaky field that has issued lots of false results. But the usual form of analysis goes like this: conduct a large survey of consumers and ask them about their diet (e.g. how much red meat do you eat each week?); obtain information about their health status, either through the same survey, a different survey, or direct measurements if they are part of a research study; then correlate the dietary data and the health data.

Now, imagine you want to study if eating GMO foods affects your health, either positively or negatively. Your survey question will be something along the lines of "How much GMO foods did you eat last week?"

Without GMO labeling, there is no way to conduct such research. This is why GMO labeling is good science. Not labeling GMO is bad science - actually it mandates no science.

 

Statistical thinking on my subway commute

So I recently moved and needed to find the optimal subway ride up to Columbia. I have been go back and forth between my two choices to collect some data to help make up my mind. Both routes require two train exchanges but only the first leg differs. In other words:

Route 1 : A -> B -> C
Route 2 : X -> B -> C

Here, the "nodes" (A, X, B, C) are train lines and the first arrow is the Times Square subway station. There are two ways to get from my apartment to Times Square, after which the two routes are identical.

This means the problem reduces to comparing:

Route 1s : A
Route 2s : X

How long does it take to get to Times Square using line A versus line X? Based on my experience so far, A > X by 5 minutes. Under normal circumstances, X is the choice as I get to Times Square 5 minutes earlier. The entire trip takes 35-40 minutes - the 5 minutes don't seem like a world of difference.

So far, I have ignored perhaps the most important piece of data: how variable are the travel times on line A versus line X? Each of those legs consists of part walking, part waiting at the platform, and part riding on the train. The waiting is the key source of variability.

The 5 minutes' difference was based on smooth transitions on both lines. If you have used NY subways, you'll know that wait times of 5-15 minutes are very common. So if line X tends to require longer waits at the platform than line A, then the 5-minute advantage can easily be wiped out!

So my next data collection task is to figure out how likely it is to suffer the distribution of wait times at the respective platforms.

***

I cover the average versus variability concept in Chapter 1 of Numbers Rule Your World (link). This concept is related to the signal and noise concept that Nate Silver made famous. The normal difference of 5 minutes is the "signal". The "noise", that is to say, the variability of wait times on the platform, may be so strong that you cannot "see" the signal. This is what I mean by "wiping out the difference."

 

 

Know your data 19: don't tell me you are 2 blocks away when you are 20 blocks away

Well, it didn't take long but private investigators have found the next big thing: Big Data.

Bloomberg reported on a company called IDI, who sells our data to private investigators. (link)

Unfortunately, this article is short on details and long on sensationalized catchphrases ("Every move you make. Every click you take...")

The CEO of IDI boasted that "We have data on that 21-year-old who’s living at home with mom and dad."

***

We really ought to be clear about what we mean by "data".

The definition of "data" is very loose. Much of this "data" are not direct records but are "model outputs."

For example, you can't measure things like how much electricity the 21-year-old in that household is using, separate from the power used by the parents. The raw data is not available, not even at the utility company. However, it is possible to create a statistical model to allocate the electricity consumption among the household members.

For a long time now, data providers have been selling businesses things like the income of individuals. Usually, the raw data come from Census records but the Census Bureau does not ever release individual data - in fact, a lot of care is taken to make sure individual incomes are not made public. Any released data are aggregated. Then, analysts build models to project the incomes at the individual (or household) level. These models are only moderately accurate.

***

If you are a driver, you might take notice that companies are out there taking snapshots of your license plates everywhere you go, and selling your locations. This is mentioned in the Bloomberg article. I have discussed this on this blog as well - the technology burst on the scene with Google Maps, then the police forces everywhere started using it, and now we have data companies doing the same.

One good thing about this... next time your friend tells you "I am two blocks away and will be there in 5 minutes," you can say "according to IDI, you are 20 blocks away and won't be here for another 30 minutes!"