Big Data, Plainly Spoken (aka Numbers Rule Your World)

More shallow science on obesity measures

For those who read the New York Times's stories denigrating BMI as an obesity metric, you have heard only a small part of the story. Their latest senasationalist coverage says that 18% of BMI-obese people are not really obese, with 12% labeled as "healthy obese" and 6% called "skinny fat".

This is a major missed opportunity by an important newspaper in promoting deeper thinking about scientific data.

The first question to ask when faced with those numbers is how is "true" obesity defined? A claim that 18% of BMI-obese people are misclassified is a claim by someone that they know the true state of "obesity". But how is obesity defined? You know what, there is no objective measure. BMI is based on height and weight. Other measures are based on waist size, body fat, and so on.

Those numbers make an assumption that body-fat percentage as measured by the DXA method is God's word on obesity. Someone is making a claim that the 12 percent of people who are obese under BMI but not obese under DXA is "healthy". The claim is not even that these people are "not obese"--they are "healthy". This isn't science.

Further, DXA requires a body scan on an expensive machine while BMI is a measure that can be computed and monitored by anyone at home. That alone makes using DXA as a measure impractical.

Further, there is a huge literature that establishes the correlation between BMI and a variety of health problems. There is very little research that shows DXA as correlated with health problems. (Much of this is because DXA is not readily measured.)

Further, the chart is a bit misleading because officially, BMI-obese is BMI over 30. Between 25 and 30 is called "overweight". Almost all of the "misclassifications" are BMI-overweight so those "healthy obese" are just under the BMI-obese definition. In any case, the worst that could happen to these 12% is that they are asked to exercise, eat healthy, etc. What is the cost of such misclassification? The "skinny fat" group is a bit more concerning but where is the proper scientific evidence proving that this group faces abnormal death risk from fat?

Chapter 2 of Numbersense (link) has the full story on this misguided argument about DXA vs BMI. The bottom line is: our obesity crisis will not be solved by changing how we measure obesity.

 

 

 

Statbusters: To divide or not to divide by 365

In the newest column for the Daily Beast, Andrew and I look at the media's fascination with expressing large numbers as daily numbers. (link) In short, you should divide by 365 only when the metric actually scales with time, and be careful if the metric is not evenly distributed across time. We discuss the following headlines: "Air pollution is China is killing 4,000 per day" and "Periscope users view 40 years of video per day".

Read the whole thing here.

Dumbing by numbers

The New York Times has been making waves this week featuring management practices at Amazon and workplace tracking practices at various companies (link). These are essential references for how data make us dumber.

I am going to ignore the shocking claim by the journalist who stated that GE is "long a standard-setter in management practices." To give him some credit, he did not say "good" management practice. It is true that business schools like to glorify GE managers. But the most famous GE doctrine is to line all employees up at the end of the year, and give the bottom 10% pink slips. (See Jack Welch's Wiki page.) This practice is of the same cloth as the "purposeful Darwinism" that was vilified in the article about Amazon.

What I want to focus on is the completely bonkers line of argument paraded by software vendors who sell workplace tracking (i.e. surveillance) tools.

1. The performance of your workers is completely measured by our continuous and usually stealthy tracking of data.

2. Because of the continuous and stealthy nature of tracking, the data are objective, unbiased, trustworthy, and accurate.

3. Because such data are available, managers will no longer let their own subjective feelings, personal friendships, idiosyncracies, nespotism, trivial feuds, vendettas, racism, sexism, ageism, etc. affect their decision-making.

***

The CEO of one of these startups was quoted by the Times:

I couldn’t imagine living in a world where I’m supposed to guess what’s important, a world filled with meetings, messages, conference rooms, and at the end of the day I don’t know if I delivered anything meaningful.

So what are the data that would allow each worker to know every day whether they "delivered something meaningful"? The article mentioned just two types of data: the usual tracking of how people spent their time at work; and little notes workers are encouraged to send to bosses to "nudge" or "cheer" each other.

Just because you can count "nudges" or "cheers", or you can count the words, or pairs of words, or triplets of words, most frequently associated with someone, doesn't mean you know anything meaningful about their performance.

In fact, a lot of this data are manipulated, and probably worthless.

Even within the Times articles, there are multiple examples of why employee notes are not to be trusted. "People wouldn't put something negative in a public forum, because it would reflect poorly on them," said one vendor. At Amazon, employees reported that the secret feedback system is "frequently used to sabotage others". I find it hard to believe that we even need proof of such behavior. In fact, that is one of the key points I made in Numbersense.

Counting emails, or minutes spent on the work computer, is similarly pointless. Someone who spent 20 minutes on the computer is not necessarily more productive than someone who spent 10 minutes working and 10 minutes web-surfing random things. The former employee might be slower, or confused, or learning on the job, or day-dreaming. Again, it's hard to believe that we even need proof of this point.

There is a tendency to believe that data have intrinsic value. One of the worrying trends in the age of Big Data is insufficient time spent understanding if the data collected measure the right things, and whether the analyses provide even marginally trustworthy answers to the questions being asked.

In a lot of ways, data have made us dumber.

 

 

 

 

Reporting on Ferguson

In today's Daily Beast column, Andrew and I looked at the reporting of Ferguson and the attempts to place the event in the context of larger crime waves. We discuss issues of the narrative fallacy, cherry-picking of data, and publication bias.

The link is: http://www.thedailybeast.com/articles/2015/08/10/the-truth-about-post-ferguson-gun-deaths.html