Dumbing by numbers

The New York Times has been making waves this week featuring management practices at Amazon and workplace tracking practices at various companies (link). These are essential references for how data make us dumber.

I am going to ignore the shocking claim by the journalist who stated that GE is "long a standard-setter in management practices." To give him some credit, he did not say "good" management practice. It is true that business schools like to glorify GE managers. But the most famous GE doctrine is to line all employees up at the end of the year, and give the bottom 10% pink slips. (See Jack Welch's Wiki page.) This practice is of the same cloth as the "purposeful Darwinism" that was vilified in the article about Amazon.

What I want to focus on is the completely bonkers line of argument paraded by software vendors who sell workplace tracking (i.e. surveillance) tools.

1. The performance of your workers is completely measured by our continuous and usually stealthy tracking of data.

2. Because of the continuous and stealthy nature of tracking, the data are objective, unbiased, trustworthy, and accurate.

3. Because such data are available, managers will no longer let their own subjective feelings, personal friendships, idiosyncracies, nespotism, trivial feuds, vendettas, racism, sexism, ageism, etc. affect their decision-making.

***

The CEO of one of these startups was quoted by the Times:

I couldn’t imagine living in a world where I’m supposed to guess what’s important, a world filled with meetings, messages, conference rooms, and at the end of the day I don’t know if I delivered anything meaningful.

So what are the data that would allow each worker to know every day whether they "delivered something meaningful"? The article mentioned just two types of data: the usual tracking of how people spent their time at work; and little notes workers are encouraged to send to bosses to "nudge" or "cheer" each other.

Just because you can count "nudges" or "cheers", or you can count the words, or pairs of words, or triplets of words, most frequently associated with someone, doesn't mean you know anything meaningful about their performance.

In fact, a lot of this data are manipulated, and probably worthless.

Even within the Times articles, there are multiple examples of why employee notes are not to be trusted. "People wouldn't put something negative in a public forum, because it would reflect poorly on them," said one vendor. At Amazon, employees reported that the secret feedback system is "frequently used to sabotage others". I find it hard to believe that we even need proof of such behavior. In fact, that is one of the key points I made in Numbersense.

Counting emails, or minutes spent on the work computer, is similarly pointless. Someone who spent 20 minutes on the computer is not necessarily more productive than someone who spent 10 minutes working and 10 minutes web-surfing random things. The former employee might be slower, or confused, or learning on the job, or day-dreaming. Again, it's hard to believe that we even need proof of this point.

There is a tendency to believe that data have intrinsic value. One of the worrying trends in the age of Big Data is insufficient time spent understanding if the data collected measure the right things, and whether the analyses provide even marginally trustworthy answers to the questions being asked.

In a lot of ways, data have made us dumber.

 

 

 

 

Comments

[Chris P]

As a GE manager, I can report that the drop the bottom 10% of the employees is not how ratings are calculated.

[Kaiser]

Chris: The article did have quotes from GE HR managers who said that the old system has been scrapped. Now, it is also possible that Welch and others described a simplified version of the process. The business climate 10-20 years ago, I recall, was such that the cut the bottom 10% policy wasn't controversial. There was also Chainsaw Al who was famous for a similar policy.

[Ken]

The General Electric purges are a nice example of the problems with both trying to measure things and looking at the past performance of policies. You start with a company with a bloated and not very efficient management and removing the bottom 10% is going to be a fairly reliable way to improve management. Some of them will be good managers but most are not. Anyone who works in a publicly funded business will know the feeling.

Move forward a few years and even if 10% of your managers are bad, then picking them will be an achievement. So getting rid of what are evaluated in the bottom 10% will take a lot of good managers, causing a lot of fear and resulting in people attempting to make themselves look better rather than doing useful work.

I think everyone has underestimated the problems of data errors and missing data for big data. How many of us when forced to give information like phone numbers have typed in something at random.