Big Data, Plainly Spoken (aka Numbers Rule Your World)

What the Dear tech companies article says about predictive algorithms

Gillian Brockell at the Washington Post published a heart-felt essay pointing out the inhumanity of algorithmic (i.e. programmatic) advertising on social-media platforms like Facebook and Instagram (owned by Facebook). It's getting some attention in tech circles, which is a good thing. (Thanks to reader Antonio R. for tweeting it to me.)

Here's a tl;dr of her story: she works at the Post as an video editor, and is very active on social media, especially Facebook. Sadly, she recently suffered a stillborn. She found that Facebook continued to bombard her with advertising of pregnancy products, which kept bringing up sad memories. When she clicked on the "I don't want to see this ad" icon, the advertising did not stop but just switched gears, now assuming that she needs post-childbirth products.

She's social-media savvy enough to realize that the advertising platforms perfected by Facebook and Google are inhuman, driven by algorithms, based on silent, pervasive collecting of personal details that are either shared publicly, shared within semi-private communities, or transacted privately. Advertisers are relying on the scoop of data to auto-pilot their advertising campaigns, most of which define the goal as ad clicks.

***

The zeal of the tech industry to sell more servers, more processors, more boxes, more tools has led to a level of over-confidence in predictive algorithms that underly Gillian's dissatisfactory experience. Such enthusiasm is matched by the advertisers' hunger for more clicks, and more sales while ignoring the tenuous link between those clicks and sales. As data scientists, it's easy to get caught up in the adulation and, using Nassim Taleb's phrase, get fooled by randomness.

It is widely held that predictive algorithms are all-knowing, personalized, and omnipotent. The reality is not so clearcut. Let's take a deeper look at how algorithms work, and from there, maybe the data-science community will come up with alterations that can reduce the chance of dissatisfaction.

***

ALL-KNOWING

Algorithms are not all-knowing despite our tendency to believe they are so. Each algorithm is driven by a fixed set of data inputs. In Gillian's article, she suggested some possibilities, such as the tags on her Instagram, the contents of her Facebook posts, or her friends' posts, Google searches, "metadata" on Amazon wishlists, etc.

All of those could be inputs in someone's algorithm but each such item has to be meticulously captured by code. Collecting every additional item requires more coding. How much is collected depends on the culture of the development team. Some are extremely zealous, others draw the line.

Every algorithm then weighs the importance of different data inputs. These weights ultimately control the actions of the algorithms. Their determination is a guessing game. It's like what factors you'd use to decide who should be a Hall of Famer in sports. No two people can agree, neither can two algorithms.

***

PERSONALIZED

No algorithm can be truly "personalized", as in one-to-one personalized. One-to-one is incompatible with Big Data. If the algorithm tailors recommendations to you just based on what it knows about you, then it likely will make lots of mistakes. Data available at the individual level is sparse, and riddled with holes.

All algorithms leverage statistical averages. It's much easier to predict what movie the average teenager would watch than the preference of each particular teenager. Most algorithms work like this: if the teenager discloses all the movies that s/he watched in the past, then that personal history will form the basis for the recommendations, which can be quite accurate; but for most people, that level of detail is not present, so the algorithm falls back on statistical averages - what the average teenager watches.

Even within individuals, taste changes over time. So even for the data-hoarding teen, the recommendation derives from a "moving" average of his or her past record, in other words, the average over time of his or her past viewings. Because of this, the longer the history, the less sensitive is the algorithm to recent data. The GPA is a good analogy. It's much harder to move your GPA in your senior year than in your sophomore year because of the accumulation of grades.

(PS. This is a call for the industry to take up the issue of right to be forgotten. Deleting old data not only pays respect to your users but also removes a source of error from these algorithms!)

***

OMNIPOTENT

The most damaging myth about predictive algorithms is their supposed omnipotence. Almost every report in mainstream media about predictive algorithms makes assertions such as "Google can predict when you will die", and "IBM can predict who's a good employee". What do they mean by "can predict"? Readers are led to believe that "can predict" means "will predict", or "will predict accurately".

In reality, there is no black-or-white, no can or cannot, no accurate or not accurate. Accuracy is important, is much lower than advertised, and is not a binary state but a progression.

Predictive algorithms have been around for a long long time. Think weather forecasts. If I say, "the weather channel can predict the weather", you do not presume they will predict every day accurately. Some of us might even think "well, they can barely predict the weather. They have the ability to issue forecasts, but that doesn't mean the forecasts are accurate or useful."

***

There are several other important aspects of predictive algorithms, which I delve into in my two books. For example, accuracy is measured several ways, and always involves a trade-off between false positives and false negatives. The algorithms behave as per the incentives of their owners. The chapter on steroids testing, terrorist prediction, polygraphs in NUMBERS RULE YOUR WORLD (link) tackles this.

Also in NUMBERSENSE (link), the chapter on Target's model to predict pregnancy walks through how one measures the accuracy of predictive models, and how incentives play a role. Nate Silver's Signal and Noise book is also recommended - he has a section discussing the accuracy of weather forecasts, and how they are systematically biased (due to incentives).

 

 

 

 

Purdue's Math, Data Science and Industry Conference

This past weekend, I found my way to West Lafayette, Indiana, to speak at the Math, Data Science and Industry Conference, organized by Math Prof. Aaron Yip and Drew Swartz. I was very impressed with the quality and diversity of the talks there. They managed to strike a nice balance between academic talks and industry talks, and the BS quotient was minimal.

I will first outline my own talk, and then in a next post, I will highlight some things from the other talks I attended.

The goal of my talk is to paint a broad-brush picture of the scope of jobs that are part of the current Data Revolution, and to give a flavor for the nature of the work so that graduate students may decide for themselves whether this "data science" industry is a good fit for them.

One key takeaway is the distinction between research jobs and industry jobs. Research jobs lead to innovative research that can be published in scholarly journals. Most industry jobs demand short-term results that impact the business, and it does not matter whether the methods used are innovative. The boom in data jobs, however, is in industry jobs. Only large corporations in cash-rich industries can afford research jobs, and even at those firms, there are hundreds if not thousands of industry jobs for each research position. Math graduates can totally get hired for industry positions, if they put in a little effort to prepare for this career path.

Within industry jobs, I like to think of three job types.

1. Data science jobs - these are the headline-catching jobs because they are disproportionately found in the high-tech industry. Think of these as software developers with advanced database skills. The culture here is automation, removing human beings from the process.
2. Business analytics jobs - these jobs are tethered to business teams, such as marketing, finance, operations and customer service. They are the champions of embedding data analyses in the everyday decision-making processes. They interact constantly with business managers, providing a form of consulting service.
3. Data IT jobs - these people keep the data flowing in the organization, so to speak. They are also responsible for "data governance" and standardizing the formats, definitions, quality, etc. of the data. This sector is experiencing rip-roaring growth.

There is a huge need for scientific thinkers and data-savvy people in all three job types but at least half the open positions are in "business analytics." I discuss two particular gaps in skills that hiring managers often complain about in university graduates: (a) inability to develop the question and (b) not knowing how to question the data.

There is a clear reason why such gaps exist. A typical question we pose to students in a problem set first lays out the problem to be solved, then presents the set of data to be used, and finally challenges the students to plug the data into an appropriate method or framework so that the solution to the problem drops out.

The professor is not going to look kindly on the student if s/he criticizes or revamps the question or points out flaws in the data! (University classes teach theory, models and frameworks so this is not surprising.)

This brings me full circle to the distinction between research and industry jobs. In research, you can "choose your battles" by making certain assumptions to move past obstacles. For example, you assume that the (biased) dataset that you obtained is representative - lots of research papers that use observed social-media data do this. You just argue that bias correction is a separate problem to be tackled at some other time, perhaps by some other research team.

In industry, you don't have that luxury. A great solution to the biased problem may turn out to be a horrible solution to the unbiased problem. When I was at SiriusXM, we had some data on people's online listening patterns but almost nothing on their in-car listening. Building great models using the online data isn't going to do much good because most of the listening happens in the car, and people who listen online are quite different from those who listen in the car.

Towards the end of the talk, I pointed out that in order to do well in these data jobs, one must be comforable to live in the "gray" areas. There is the gray between science and social science, between models and heuristics, between data and intuition.

People were very friendly and we had some fine conversations at a bar after the day was over. I'm happy to report that at least a few people have indicated that they want to pursue these industry jobs.

 

 

 

 

 

 

Know your data 24: trust eroding

Big Tech is in serious danger of losing our trust when it comes to our data. For the longest time, they have sold consumers on a bargain: in return for providing convenience and services, they claim our personal data and sell them to the highest bidders.

That bargain requires consumers to trust Big Tech as good custodians of the data.

A number of recent revelations has eroded that trust. And I'm not talking about Cambridge Analytica.

Here are two recent examples that have not received the attention that they deserve.

Android phones found to track users even if they turn off Location History.

The Associated Press first broke this story, which was later verified by Princeton researchers. Android users who choose to turn off Location History obviously do so intending that they not be tracked.

In the same story, it is also disclosed that iPhone users who have installed Google apps are also being tracked at all times. When these users disable location tracking for the Google app, the app stops saving locations in the folder called Location History and saves the location data in a different folder instead. (Note also: the phone itself would not know how to do this; software engineers wrote code to enable this feature.)

This revelation is highly damaging. There have always been suspicion that your phone (or TV or other devices) is spying on you even if it is turned off. There are no technical obstacles to making this capability. The only reason why customers are not worried is that they trust Big Tech when they claim their phones (or other devices) do not have that capability.

Google does not deny this is happening, and in fact, argues that it is transparent in dealing with users.

***

Facebook found to have taken users' phone numbers provided for "two-factor" authentication and sold them to advertisers.

For a number of years now, Big Tech and small tech alike have bombarded us with security warnings, and made claims that "two-factor" authentication is the ultimate solution to online security issues. Setting up two factor requires the user to provide a cell phone number, which immediately removes any semblance of online anonymity (unless you get your hands on a burner phone).

It's been known in the marketing world forever that the key to unlocking anyone's data is your phone number. That's because the cell phone is almost always a personal device.

I don't typically consent to two factor for fear that the phone number could then become marketing fodder. This story, by Gizmodo, confirms that my worst fear is real. Similar to Google above, Facebook confesses to coopting those phone numbers, and even maintains that users have consented to such use.

This revelation is also highly damaging. In fact, it damages the reputation of the security industry. How are we to trust them when they tell us to use things like phone numbers, fingerprints, eye scans, etc. for security purposes when we cannot trust that such private data would not be transferred to other entities?

In the same story, it is disclosed that Facebook also harvests phone numbers for advertisers under a variety of other pretexts. For example, when you provide your phone number to receive alerts about new log-ins to your account, that number ends up on marketing lists. Further, when you upload your contact list to Facebook, you have exposed all of your friends' phone numbers, turning them into fair game for Facebook's advertising clients!

This further relevation is even more damaging. Lots of websites ask for our phone numbers under the pretext of servicing our accounts, but we can no longer trust any of them that they would not turn our personal data over to advertisers.

 

***

Judging by their subsequent pronouncements when caught red-handed, we fear that Big Tech is not sensing the importance of "trust" in their business model. If the trust goes away, these businesses will be in for a nasty surprise.

It's hard to tell apart business journalism and PR: the Wework edition

Most of business journalism reads like PR put out by companies. Still, some pieces are more blatant than others. The NY Times just printed an example of such gibberish, this opinion piece about Wework.

Wework is one of those "unicorn" startups that investors are hoping to sell to the public before the impending stock market collapse for gazillion dollars. Its main business is renting office space to mostly small businesses, entrepreneurs, or small divisions of large companies. The primary "innovation" is the short-term lease, as short as month-by-month. (Disclosure: my startup was a sub-tenant of Wework through the Harvard Business School Startup Studio. I am a satisfied customer.)

As with many successful startups, Wework offers a product to a segment of customers who find it irresistible. For a startup business like mine, it's much too risky to commit to a 10- or 20-year commercial lease. Most startups fail within a short few years, and the entrepreneurs are already taking on a lot of other risks. So they have an attractive product but does it have a viable business model to deliver that product profitably?

With few exceptions, Wework does not own the buildings it leases out - it has signed long-term leases with landlords, and then rent to startups and smaller businesses who sign short-term leases, thus the risk of the long-term lease shifts from the entrepreneurs to Wework.

The Times article presents an argument by analogy to refute an obvious weakness of Wework's business model. This same weakness has felled many bankrupt companies in the past. When you have long-term obligations and short-term revenue sources, you're highly vulnerable to the "credit crunch." This scenario played out in the Great Recession when many companies suddenly found that banks were unwilling to roll over their debt, which immediately put them out of existence. In the case of Wework, critics also worry that even the revenue sources will disappear since many startups will fold due to lack of customers and/or funding during a recession.

The reporter (Andrew Ross Sorkin) now builds a very flimsy argument based on "too big to fail." It cites some absolute numbers, such as square feet of space it has signed leases for, number of workers using the space, number of employees, etc. Then he arrived at the conclusion that: "when the next economic downturn comes — and it will — WeWork’s landlords will actually be less likely to evict the company if it doesn’t pay its rent."

For an argument based on big numbers, this one is particularly unmoored. None of those numbers are provided in context. What proportion of occupied commercial real estate has Wework leased out? What proportion of revenues does Wework contribute to the biggest landlords? The 1,300 employees may make Wework "one of the biggest architecture firms in the world" but the number does not qualify it as "too big to fail". (Fact check: the largest architecture firm is Gensler, according to Architectural Record, which employs over 5,000, according to Wikipedia. So Gensler is about 4 times the size of Wework.)

In addition, the argument misleads readers about what happened during the Great Recession. The banks who were the creditors did not renegotiate contracts with the debtors who no longer could pay. Far from it, they got bailed out by the U.S. government, using the argument that the failure of those loans would "bring down the entire global economy as we know it". 

This brings me to the final, and most salient, issue. What would those generous landlords be saving in this case? Not the global economy, but a business yet to find a self-sustainable business model - as Sorkin himself cited earlier in the article. Wework is currently running losses of over a billion dollars a year. According to Bloomberg, Wework made ~$800 million in 2017, and spent $1.8 billion. We also know Wework has sold junk bonds but I'm not sure we know the full amount of debt it is carrying on its books.

Wework has made some small steps to reducing its risk. It has indeed attracted some larger companies to rent space with longer leases. But those companies are doing it because the "longer" leases are still much shorter than what they could get from traditional landlords. Usually, the space is used by short-term, even experimental, business units.

***

Now on to something of interest only to number geeks. I have questions about this statement: "the company says its average customer has a lease of seven to eight months and new customers are signing leases that average 20 months."

I wonder how this "average" is computed. Note that most of the little guys are on month-by-month leases, which renew until cancelled. This is "right-censored" data in which we don't know the end date yet, so a lot of assumptions and projections go into that average. It's got to be an average of projections as opposed to an average of observed data.

Also, I expect that the distribution of the size of customers will be heavily "right-skewed": a vast majority of customers will be freelancers, and small offices (who sign very short leases), with a few larger tenants (signing longer leases). It's challenging to interpret an "average" unless we also know the "median".