Big Data, Plainly Spoken (aka Numbers Rule Your World)

Know your data 10: spying mannequins

Mashable (link) made a big fuss out of a new technology: mannequins that have cameras and data processing software, which allows the retailers to discern age, gender, and race of shoppers. This allows the shops to observe shopping patterns, and they will use this knowledge to change the store layouts, product placements, and so on to improve the shopping experience.

I see nothing wrong with this. The software as described is much more benign than the face recognition software deployed by Facebook for example. The Facebook version links a face to a name while this software only places one into broad demographic segments. A store has to be designed for the "average" shopper, and so it is useless for the shop to identify individuals. Moreover, shops have a large number of transient customers, say tourists, who will only show up once in their lives.

Mashable has a quote from Nordstrom, claiming that this technology "crosses privacy boundaries with customers" and they would never use it. That's a nice sound bite but I wonder if Nordstrom, or any large retailer for that matter, has owned up to spying on shoppers already. They have pretend-shoppers in their stores taking detailed notes on how individual shoppers behave. A number of books have been written on this topic, such as Paco Underhill's Why We Buy: The Science of Shopping (link).

I admire the work Underhill and associates have been doing. Store statistics can only tell you what but never why. In order to understand why shoppers behave in certain ways, these researchers are doing the legwork of painstakingly collecting data. This is one of the lessons of Chapter 2 of Numbers Rule Your World. In epidemiology, it's called shoe leather. In the world of Nate Silver, it's called polling -- what Nate does isn't possible without pollsters interviewing real people, gathering the data, compiling and analyzing them. One can say the new mannequin technology streamlines the data collection a bit.

In fact, these mannequins are much less intrusive than online or mobile shopping. In those arenas, retailers can absolutely link our every click to our name and address. There are those who claim they receive "permission" to collect data but no online retailer I know of would let me buy something if I reject their data sharing agreement.

***

I'm waiting for Mashable to correct their story. In a bid to find a villain, they claimed that Benetton is one of the big-name customers of this technology--both in the lead and in the article itself. However, if you click to the original Bloomberg article they cited (link), you'd be surprised to learn that "Benetton Group SpA said it’s not using EyeSee or comparable technology."

This lapse of reporting accuracy is much more serious than the alleged privacy intrusion.

Our ability to estimate simple statistics: a neglected research area

I recently came across a series of papers by Irwin Levin (link), about how well people estimate statistical averages from a given set of numbers. In contrast to the findings of Tversky and Kahneman, Gigerenzer, etc. on probability, it seems like we are able to guess average values pretty well, even in the presence of outliers.

It must be said the sample size used in Levin's experiments was tiny (12 students in one case but working with something like 75 sets of numbers). That said, the experimental setup was remarkable. Take this paper as an example. The numbers were either shown in sequence or at the same time. Levin created three types of tasks: a descriptive task in which the goal was to get the average of the numbers presented, including the outliers; an inference task in which the goal was to guess the average of the population of numbers from which the sample was drawn, in which case we expect the subjects to discount the outliers; and a discounting task, in which subjects were presented with data including outliers, but were asked to ignore them.

***

The reason for this post is that Levin's work was done in the 1970s (Levin himself retired this year according to his webpage). There doesn't appear to be much interest in this subject since then.

It seems like researchers may find the estimation of summary statistics like means, medians, etc. not interesting enough. All the new research that I know of concerns judging probability distributions, margins of error, variability, etc.

However, I'm more interested in point estimates, and I feel that the early research left the question still unsettled. I haven't found any research on how good we are at guessing the median of a set of numbers, or the mode, or trimmed means, or moving averages. If we see repeated numbers, are we likely to use the average, the median or the mode, or some other statistic to summarize that information? Given what we now know about irrationality and biases in judging probabilities, are we able to replicate Levin's finding? Or will we find that his experiment would not hold with better samples?

 

 

Use of statistics in setting insurance rates

Reader Mark Johnstone, from across the pond, points me to some fascinating materials that are highly irrelevant to those who have read Chapter 3 of Numbers Rule Your World, where I explain the statistical underpinning of insurance policies. It's unfortunate that our policy-makers do not understand the probabilistic nature of this business, and create rules that are self-defeating.

The EU Court recently decreed that insurers are not allowed to use gender as a factor in determining how much one pays for insurance. (WSJ article here; also PDF of the ruling). For example, life insurers sell policies to women at lower average prices than to men--well, women on average live longer than men. The argument against such differential pricing is that it discriminates between people. That fundamentally usurps the entire concept of insurance.

As I said in the book, insurance is an ingenious and extremely human (and humane) concept. It turns what seems like a bad thing (cross-subsidies) into a good thing (lower average prices for everyone). It protects the unlucky few. It is always the case that some people subsidize other people. But we accept that because it is not known beforehand who would suffer the early death and who would have the long life.

The insurance scheme fails if some customers feel like the subsidy has been set up unfairly, that is, you think you are a sure loser who subsidizes others in the scheme. This is why insurers would give women lower prices. Otherwise, the women would drop out of the insurance pool if and when they realize they would always be subsidizing the men.

***

This ruling may have broad implications on all kinds of businesses because any predictive models used in marketing or credit scoring or online ad targeting,etc. all contain gender as one of the factors. I have always understood that if gender is only one of many factors in a multi-factorial model, then we are acting in good faith.

In Chapter 2 of the book, I explained how a simple credit-scoring model works. If gender is only one of many factors in the model, then not all women receive the same credit score. Amongst women, other factors such as income, education, the kinds of magazines you read, etc. determine one's particular score.

That said, if someone were to remove all other factors from the model, that is to say, if you aggregate everything else and come up with the credit score for the average woman and that for the average man, they are certainly not identical. But that is a very poor use of statistical aggregation. That's when you throw out a lot of important data to arrive at an overly simplistic conclusion.

Here's an example of such aggregation (link):

In Chapter 2, I made the argument that for credit-scoring applications, statistical models based on correlations are sufficient. It doesn't matter whether it is gender that causes the lower propensity to be responsible for car accidents. If it is consistently the case that if someone satisfies a set of criteria (one of which may be female), one has much lower risk, then to me, it is enough to give such people lower rates. The Court rejects this argument and suggests that only causal models can be applied (which is as good as saying, no models can be used.)

***

Mark told me one of the proposals out there is "to fit 'black boxes' into cars so more individual data can be collected, as opposed to relying heavily on aggregates". Presumably, this suggestion is made to automobile insurers.

You know what, the statistics cannot be suppressed, and the result of using this method will not be materially different. If you now take the rates set by these "black boxes" and aggregate them up by gender, as I described above, it is again certain that the rate paid by the average man will be different from that paid by the average woman. The only way by which they would be the same would be that male and female drivers cause the same proportion of accidents, which is demonstrably false.

***

Would insurance premiums go up as a result of this?

It would seem to me that women will pay more and men may pay slightly less. Eventually, some women will realize that their premiums essentially subsidize men. If they start dropping out of the pool, then everyone's premiums would rise.

***

Would the EU Court stop at gender? Arguably, discriminating on age, on education, on incomes, on what journals you subscribe to, etc. are all bad.

One other point I made in Chapter 2 is that automated models are just super-charged versions of manual rate-setting. If a human being is asked to set rates for different customers, he or she would end up setting lower prices for the average woman anyway. In fact, from the very beginning, actuarial tables have gender breakdowns because longevity clearly differs by gender.