Big Data, Plainly Spoken (aka Numbers Rule Your World)

Youtube debut: Fung with Data, How does Facebook track you

Excited to unveil my little summer project - Fung with Data!

Through this video series, I hope to shine light on how data, algorithms and software are controlling and affecting many aspects of our lives.

The first episode addresses how Facebook is tracking your data, and what you can do if you want to avoid prying eyes.

If you want to see more episodes, make sure you like and share the video!

 

The side effect of automation is fake data and theft

There is hope in our business journalism! That's the message I'm getting as I delight in the current stream of articles that take a critical look at the inherent unaccountability of technology giants. These stories have been around for years, even decades, but better late than never.

Fake views/plays on Youtube videos (New York Times) This is a companion piece to a previous story about fake social-media followers (New York TImes magazine)

Uber drivers gaming the pricing algorithm (Wall Street Journal)

Google's Android tracks you everywhere whether you allow it or not (Bloomberg)

Finally some coverage of the shame of IBM's Marketing (Wall Street Journal)

As blog readers know, most companies in the Web/mobile ecosystem use an advertising-supported model. Even the most popular products like Youtube, Google Maps, and Facebook have not found it possible to charge users anything (when in theory, users derive value from such products). Such digital advertising, since the beginning, has been heralded as inherently more accountable because it was supposedly more measurable. Just look at the mountains of data that are generated by the very act of putting up ads on a website. You can even trace a user's final action, say the purchase, back to various interactions prior to the action. (This data is called the "click stream".)

The marketing on digital marketing centers on its "efficiency," achieved through automating the process, taking human beings out of the equation, and trusting the black box machines. This system is flawed even in its most theoretical form, assuming "complete data". The clickstream and other digital data have multiple problems. Assigning causality is highly suspect. I wrote about this years ago - for example, here.

The opacity of the black box machines offers not just efficiency but also shelter for fraud. Any kind of verification, quality checking, common-sense making slow the system down. The more efficient is the system, the easier and faster it is to commit fraud!

What kind of fraud are we talking about? Fake everything! Fake clicks, fake impressions, fake ads, fake viewers, fake plays, fake followers, fake accounts, fake likes, fake webpages, you name it. Since advertisers are paying money for clicks, impressions, ads, viewers, plays, followers, accounts, etc., and most advertisers have neither the incentives nor the nerves to look under the hood, they are paying real dollars for all the fakes.

The much-lauded efficiency of the black-box advertising model sits on a pile of fake data. The system is seriously flawed assuming complete data; it is worse when we admit that not only do we not possess complete data, but we are drowning in fake data.

Now, data scientists are hard at work building even more black-box machines that promise to suss out the fake stuff. To understand how this might work, think about spam emails, an analogous problem. In the machine-learning world, spam detection is thought of as a "solved" problem. Spam filtering algorithms have existed for decades. Think about how many fake emails continue to sit in your inbox, and in addition, how many not-fake emails land in your spam folder - and you can begin to see why technical solutions are not potent enough to drive out fraudsters.

***

Below are some comments on selected parts of the latest New York Times article on fake Youtube views (here).

• Youtube stars can make millions from advertisers by publishing viral videos. Google which owns Youtube makes tons of money by having people upload their content to the platform and charging rent for it. Advertisers ultimately pay for those views. Youtube stars, and by extension, Google, can make lots of money from fake views, i.e. bots (software) pretending to play the video. Third parties approach wannabes to sell them fake views and take a cut of the action.
• It's a cat and mouse game that the fraudsters are winning. They have the upper hand because the products are often designed without assuming nefarious players gaming the system. Adding software to deal preemptively with nefarious players slows down tech startups trying to build an initial market. Preventing bad players from acting badly imposes (the externality of) restrictions for good players, which is another disincentive to tackle the problem.
• Youtube officially claims: "fake views represent just a tiny fraction of the total." (later defined as 1 percent). This is a guess masquerading as fact. To know what fraction of the views on Youtube is "fake" requires knowing whether each view is fake or not. In theory, one can review a sample of the views, and then infer the fraction in the entire catalog from that sample. In practice, it is not easy to prove if a view is fake just from the data collected about the viewer's browser, OS, etc. A machine-learning algorithm will "predict" if a view is fake or not but such predictions should not be used as "ground truth".
• Legitimate organizations buy views, including marketing agencies, news organizations and musicians. As a former employee said, "View count manipulation will be a problem as long as views and the popularity they signal are the currency of YouTube." It's not just YouTube - we know that when teachers are compensated based on student test scores, they become complicit in abetting their students to cheat.
• This underground economy is quite well developed now, with wholesalers and resellers of fake views. If you are a reseller, you don't even need to know how to generate fake views. You just buy fakes from the wholesalers and charge a markup!
• The sellers of fake views are so sure they can evade Youtube's detection technologies that they are openly talking about their clients, business models, etc. to the press. Either they know those technologies don't work, or they know that Youtube isn't serious about cracking down.
• Several buyers interviewed by the Times said they truly believed the marketing spin of the fake-view sellers, that they were driving real users to watch their videos. This is a tad disingenuous. As one writer who purchased views pointed out, those purchased views led to no sales, and she couldn't get any data about who viewed the videos.

It's actually very easy to demonstrate fraud as the fake views are barely hidden. The article came with a great graphic, which does not appear to be addressed directly in the text. So let me explain.

The reporter purchased fake views from the vendor called "BuyYoutubViews". The top chart shows that close to 100% of those views ("in campaign") played the entire video (over 5 minutes long). The bottom chart shows that when they turned off the "campaign" i.e. stopped buying fake views, almost no one plays the video for more than a few seconds. (What is not shown is that the number of views probably plunged after the fake-views campaign was shut down.)

Raise your hand if you think "BuyYoutubViews" is finding real people to watch the video!

 

 

 

 

What about daughters of politicians?

Sometimes, the story is so good there is no need to embellish.

A medical school in Japan blatantly reduces test scores of women to reduce the proportion of admitted women.

It also awards bonus points to the politically well-connected.

The practice started in 2006, not 1906.

Read the story here.

Do we really know what the data are measuring? Hint: no

The tech industry has turned us into an omni-surveillance society.

Any shop that uses modern, digital, connected technologies is probably collecting, storing and selling your data to someone. The people receiving and analyzing the data form a much larger set than those collecting the data. These data analysts typically ingest the data as are, and write software that controls this or that aspect of our lives. However, such data are riddled with inaccuracies and bias, which is a form of inaccuracy.

While in Vancouver last week, I encountered the following two scenarios that illustrate the fragility of data collection.

***

I purchased a drink at the Dunkin Donuts store at JFK Airport, and I noticed that the receipt said "Dine In" even though the shop is a simple counter with no tables or seats, which means 100% of orders are take-out.

There can be a number of reasons for this mis-coding:

• the store manager doesn't care because it's obvious to anyone who works at the shop that every customer is take-out. Dine-in/take-out isn't a variable of any interest to management at this location since it is invariant.
• the software defaults to "dine in" and the staff is too lazy to toggle it for each transaction
• the employees may have been told to toggle the setting but they realize that no one cares and so do not follow the instructions
• the employees are not trained to toggle the setting
• there is a known bug that prevents the setting to be toggled

Now, imagine a data analyst who got a hold of this "data exhaust." This person likely has no expertise in fast food operations. Unlike the store manager or employees, s/he can't tell that when it says "Dine in" in that particular location, it actually means "Take out".

***

I went to a supermarket to buy a beverage for the road. The lady in front of me had only three items, so I thought it should be a short wait. Or so I thought!

The first item was scanned without issues. The second item was some Anjou pears, one of the most popular varieties in North America. See picture. Surprisingly, the worker didn't know what kind of pears those were, and had to ask around. That took a while.

The last item was a couple of zucchini. The worker scanned the bag, then cancelled the transaction. She apparently noticed that the system recorded those as "gray squash $1.18" when it should have said zucchini. So she unfolded a lookup table, found the line for zucchini and re-entered the number. It came out as "gray squash $1.18" again. She cancelled the item again.

The third time the charm. Maybe she read the wrong line but when it showed "zucchini," she was satisfied. The price? Exactly $1.18.

Hat #1: I'm the impatient customer waiting in line for my turn. It seems ridiculous that she spent 5 minutes figuring the code for zucchini and ended up charging $1.18, the same amount as the "error". She should have just let the mistake live.

Hat #2: I'm the data analyst possibly getting this data as "data exhaust", or perhaps the data analyst who works for this supermarket doing product sales projections. Kudos to this worker! She corrected the error at source, so that when the data flowed through the system, there was no miscoding!

Which side are you on? I have to say, in that moment, I just wanted the line to move faster, and would rather she left the data error in the system.

***

In my OCCAM framework for thinking about contemporary datasets, these are problems related to "adapting" data collected by other people for other purposes. You are never sure whether the data measure what you think they are measuring. Miscoding can be caused by management decisions, laziness, inattention, expediency, etc.

Most importantly, in both these examples, the data would come in looking fine. It's a miscoding, not a typo, not missing. Further, collecting more data does not solve the problem. It may even reinforce existing errors by having more samples of the wrong things.

One of the key differentiators of a good data analyst is whether s/he will diagnose tricky problems like this.

When should you leave the baseball stadium when the game is just too damn long

My new article, joint work with Pravin and Sriram, has appeared in FiveThirtyEight this week. It's motivated by this chart, showing the lengths of baseball games:

Since 2000, the average Major League baseball game lasts over 3 hours; one out of 10 games takes over 3.5 hours. The added time is almost all non-action time. In fact, there was an article in the Wall Street Journal in which they logged a bunch of games and discovered that less than 10 percent of the game time can be classified as "action." For the uninitiated, the "non-action" time includes pitchers staring at the batter, batters taking practice swings, managers slow-walking to the mound to chit-chat with the pitcher, etc.

To their credit, the League office has started to pay attention to this problem although it seems like an agreement with the players' union is hard to come by. Some fans are taking matters into their own hands - why fight with all others to get out of the stadium when you can leave in the middle of the game, with one team "comfortably" ahead?

In our article, we model this decision as a stay-or-leave decision, and use Wald's Sequential Probability Ratio Test to scientifically determine the optimal time to leave. Here's the bottom line:

A back-of-the-envelope estimate pegs the total amount of time saved at 1,612 hours for the season. The price to pay was the 61 mistakes — dramatic comebacks that were heard on the radio from the parking lot by the fans who left early.