Big Data, Plainly Spoken (aka Numbers Rule Your World)

Dumbing by numbers

The New York Times has been making waves this week featuring management practices at Amazon and workplace tracking practices at various companies (link). These are essential references for how data make us dumber.

I am going to ignore the shocking claim by the journalist who stated that GE is "long a standard-setter in management practices." To give him some credit, he did not say "good" management practice. It is true that business schools like to glorify GE managers. But the most famous GE doctrine is to line all employees up at the end of the year, and give the bottom 10% pink slips. (See Jack Welch's Wiki page.) This practice is of the same cloth as the "purposeful Darwinism" that was vilified in the article about Amazon.

What I want to focus on is the completely bonkers line of argument paraded by software vendors who sell workplace tracking (i.e. surveillance) tools.

1. The performance of your workers is completely measured by our continuous and usually stealthy tracking of data.

2. Because of the continuous and stealthy nature of tracking, the data are objective, unbiased, trustworthy, and accurate.

3. Because such data are available, managers will no longer let their own subjective feelings, personal friendships, idiosyncracies, nespotism, trivial feuds, vendettas, racism, sexism, ageism, etc. affect their decision-making.

***

The CEO of one of these startups was quoted by the Times:

I couldn’t imagine living in a world where I’m supposed to guess what’s important, a world filled with meetings, messages, conference rooms, and at the end of the day I don’t know if I delivered anything meaningful.

So what are the data that would allow each worker to know every day whether they "delivered something meaningful"? The article mentioned just two types of data: the usual tracking of how people spent their time at work; and little notes workers are encouraged to send to bosses to "nudge" or "cheer" each other.

Just because you can count "nudges" or "cheers", or you can count the words, or pairs of words, or triplets of words, most frequently associated with someone, doesn't mean you know anything meaningful about their performance.

In fact, a lot of this data are manipulated, and probably worthless.

Even within the Times articles, there are multiple examples of why employee notes are not to be trusted. "People wouldn't put something negative in a public forum, because it would reflect poorly on them," said one vendor. At Amazon, employees reported that the secret feedback system is "frequently used to sabotage others". I find it hard to believe that we even need proof of such behavior. In fact, that is one of the key points I made in Numbersense.

Counting emails, or minutes spent on the work computer, is similarly pointless. Someone who spent 20 minutes on the computer is not necessarily more productive than someone who spent 10 minutes working and 10 minutes web-surfing random things. The former employee might be slower, or confused, or learning on the job, or day-dreaming. Again, it's hard to believe that we even need proof of this point.

There is a tendency to believe that data have intrinsic value. One of the worrying trends in the age of Big Data is insufficient time spent understanding if the data collected measure the right things, and whether the analyses provide even marginally trustworthy answers to the questions being asked.

In a lot of ways, data have made us dumber.

 

 

 

 

Statbusters are back, taking on robots that hire people

In our newest column, we take on the recent media obsession with companies who make robots that hire people. (link)

As with most articles about data science, the journalists failed to dig up any evidence that these robots work, other than glowing quotes from the people who are selling these robots. We point out a number of challenges that such algorithms must overcome in order to generate proper predictions. We also discuss why measuring the outcomes of these predictions is so hard: one problem is we have no objective standard for someone being the "correct" hire; another is the action we take based on the predictions affects the outcome that was predicted.

Read the entire article here.

Careers in Data Science and Business Analytics: a new course

Data Science and Business Analytics are red-hot in the business world -- there are definitely more jobs than there are qualified people. Is this the right field for you? How do you find a job in this field? More importantly, how do you build a lasting career in analytics? I'll be addressing these questions in a new course I'm developing to be held in late March.

Here is the tentative outline of the one-day seminar. Please let me know if you have any suggestions. There is still time to change the contents.

Also, on my sister blog, I'm announcing the next installment of my data visualization workshop.

***

 

NEW YORK UNIVERSITY

SCHOOL OF PROFESSIONAL STUDIES

CAREERS IN DATA SCIENCE AND BUSINESS ANALYTICS

A One-day Seminar, Mar 28, 2015 (Saturday), New York City

By Kaiser Fung

 

Seminar Objectives

Data Science and Business Analytics have seen tremendous growth in the last few years, and there is fierce competition for the best talent. Numerous academic programs have sprung up, and many workers are making a career transition. The job market, while vibrant, is confusing because the field is fast evolving, and because the field encompasses different job types and career paths. The goal of this seminar is to help students develop a plan for finding a job in Data Science and/or Business Analytics, and building a lasting career in this field.

Specific objectives include: understanding the nature of the job, and the state of the job market; determining whether the job is right for you; creating a networking strategy; improving your resume; developing your pitch; making the decision to invest in further education, and building a lasting career.

The seminar is divided into four sessions. Interaction is encouraged throughout the day. Students complete hands-on exercises, especially in the third session; and receive handouts on tips and resources.

Course Outline

What is numbersense?

It's Spring Break at NYU, which for professors, is not a break. I have been marking midterms for my business analytics class. Since I like to set open-ended questions (are there anything else in statistics?), I get a variety of answers. One of the questions helps clarify what I mean by numbersense.

The question asks students to comment on the distribution of a variable (median income) in a dataset of customers. Every student should know how to generate a histogram and a boxplot, plus summary statistics and percentiles for this data. The figure below shows what each student was looking at. Before you read further, think about what features of this distribution attract your attention.

The responses I received fell into several categories. Let me list them out:

1. The mean is $40,369 and the median is $43,174. Most of the customers have median income between $26,083 and $56,897.
2. The mean is $40,369 and the median is $43,174. Most of the customers have median income between $26,083 and $56,897. There is a large range of incomes from $0 to $200,001, with a lot of high outliers.
3. The median is $43,174 about the same as the mean. Most of the customers have median income between $26,083 and $56,897. There are a lot of high outliers. Almost a quarter of the sample has $0. Based on the age distribution (skewing older people), I think these may be retirees.
4. The median is $43,174 about the same as the mean. Most of the customers have median income between $26,083 and $56,897. There are a lot of high outliers. There appears to be two types of customers, those with zero income and those with a standard distribution. Some of the entries with zero income may have been missing values coded as zeroes, because they correlate with unknowns or zeroes in other variables.
5. The median is $43,174 about the same as the mean. Most of the customers have median income between $26,083 and $56,897. There are a lot of high outliers. There appears to be two types of customers, those with zero income and those with a standard distribution. Since the data are not collected at the individual level but at the Zip+9 level, meaning it measures the median income of the residential blocks around each customer, $0 surely does not mean zero. The zero-income segment has average values of other variables not too different from the positive-income segment and so most likely, zero means unknown.

These answers are ordered from demonstrating least numbersense to most. Response types #1 and #2 make no mention of the spike of zeroes despite the strong hint in the question: "Give plausible explanations for any parts of the distribution that is not smooth". Response #2 notices but is not bothered enough to explain it.

Responses #3-#5 all attempt to explain the observed anomaly. Response #3 has a good theory ("retirees") but somehow looks past the fact that the zero-income segment spans a wide age range. (The highlighted parts of the histogram below are the zero-income customers.)

 In fact, this chart was used by several to prove that retirees accounted for the zero-income segment. This is a "strong priors" problem: it's all too easy to take weak evidence in the face of a strong theory.

One student divided the customers into zero-income versus not. This allows us to examine the distribution of other variables. For example, the median home value of those with "zero income" is almost the same as those with positive income.

***

Think about the people you hire to do analytics. While any of the answers above are acceptable, if you find someone who can give you Response #3-#5, you are in much better shape. That's what I mean by hiring for numbersense.