Use of statistics in setting insurance rates

Reader Mark Johnstone, from across the pond, points me to some fascinating materials that are highly irrelevant to those who have read Chapter 3 of Numbers Rule Your World, where I explain the statistical underpinning of insurance policies. It's unfortunate that our policy-makers do not understand the probabilistic nature of this business, and create rules that are self-defeating.

The EU Court recently decreed that insurers are not allowed to use gender as a factor in determining how much one pays for insurance. (WSJ article here; also PDF of the ruling). For example, life insurers sell policies to women at lower average prices than to men--well, women on average live longer than men. The argument against such differential pricing is that it discriminates between people. That fundamentally usurps the entire concept of insurance.

As I said in the book, insurance is an ingenious and extremely human (and humane) concept. It turns what seems like a bad thing (cross-subsidies) into a good thing (lower average prices for everyone). It protects the unlucky few. It is always the case that some people subsidize other people. But we accept that because it is not known beforehand who would suffer the early death and who would have the long life.

The insurance scheme fails if some customers feel like the subsidy has been set up unfairly, that is, you think you are a sure loser who subsidizes others in the scheme. This is why insurers would give women lower prices. Otherwise, the women would drop out of the insurance pool if and when they realize they would always be subsidizing the men.

***

This ruling may have broad implications on all kinds of businesses because any predictive models used in marketing or credit scoring or online ad targeting,etc. all contain gender as one of the factors. I have always understood that if gender is only one of many factors in a multi-factorial model, then we are acting in good faith.

In Chapter 2 of the book, I explained how a simple credit-scoring model works. If gender is only one of many factors in the model, then not all women receive the same credit score. Amongst women, other factors such as income, education, the kinds of magazines you read, etc. determine one's particular score.

That said, if someone were to remove all other factors from the model, that is to say, if you aggregate everything else and come up with the credit score for the average woman and that for the average man, they are certainly not identical. But that is a very poor use of statistical aggregation. That's when you throw out a lot of important data to arrive at an overly simplistic conclusion.

Here's an example of such aggregation (link):

In Chapter 2, I made the argument that for credit-scoring applications, statistical models based on correlations are sufficient. It doesn't matter whether it is gender that causes the lower propensity to be responsible for car accidents. If it is consistently the case that if someone satisfies a set of criteria (one of which may be female), one has much lower risk, then to me, it is enough to give such people lower rates. The Court rejects this argument and suggests that only causal models can be applied (which is as good as saying, no models can be used.)

***

Mark told me one of the proposals out there is "to fit 'black boxes' into cars so more individual data can be collected, as opposed to relying heavily on aggregates". Presumably, this suggestion is made to automobile insurers.

You know what, the statistics cannot be suppressed, and the result of using this method will not be materially different. If you now take the rates set by these "black boxes" and aggregate them up by gender, as I described above, it is again certain that the rate paid by the average man will be different from that paid by the average woman. The only way by which they would be the same would be that male and female drivers cause the same proportion of accidents, which is demonstrably false.

***

Would insurance premiums go up as a result of this?

It would seem to me that women will pay more and men may pay slightly less. Eventually, some women will realize that their premiums essentially subsidize men. If they start dropping out of the pool, then everyone's premiums would rise.

***

Would the EU Court stop at gender? Arguably, discriminating on age, on education, on incomes, on what journals you subscribe to, etc. are all bad.

One other point I made in Chapter 2 is that automated models are just super-charged versions of manual rate-setting. If a human being is asked to set rates for different customers, he or she would end up setting lower prices for the average woman anyway. In fact, from the very beginning, actuarial tables have gender breakdowns because longevity clearly differs by gender.

 

 

Comments

[Ken]

In Australia our private health insurance didn't allow any discrimination at all until a few years ago. The problem was that people were tending to join when they were older and most likely to have health costs. What we have now is a system where the cost is based on the age that the person joined the health fund. That way while I know that my premium is higher than it should be now, I will get a lower premium when I'm older.

[John]

How about based on how much you earn? Or what job your parents had? Your socioeconomic status? Or what are your political views? If any of these were statistically different groups, would that mean they should be allowed? Where is the line drawn? I'm not saying there IS an actual correlation between these and safe driving/lifestyle, but what if any of these allowed to differentiate statistically relevant groups?

Like you said. Any model has some pay for the others. From separating rates based on extremely detailed statistics (removing the many pay for the few), to no differentiation in pricing (which I would say is the actual concept of insurance); It's always a choice on what is allowed for differentiation and what is not.

What if there was a relationship with the color of your skin?

[Kaiser]

John: same price for all would only work if all have equal risk but whether it's health, auto, etc., real life is unfair, or risk is partly systematic, and risk is not equal, hence the problem

[Antonio]

I also was surprised when I came to know EU Court decision. Consider that current car insurance premiums are based mainly on gender, age, citizenship (state), place of residence (country) and number of accidents in the last n years. Some domestic court has already stated that basing premium on state is not allowed, and I think that in the next future the whole mechanism of premium determination in the car insurance market will be totally shocked by other court say-so(s). Only the last criterion will be save.
The rationale behind court decision seems to be that one person cannot be judged on characteristics that he is not guilty of (it is not my fault to be born there, to live here, to be a male/female, to be young/old) or on characteristics of people that look or behave like him, but only on his own behaviour, like the number of past accidents he is guilty of.
You write that "The insurance scheme fails if some customers feel like the subsidy has been set up unfairly, that is, you think you are a sure loser who subsidizes others in the scheme. [...] Otherwise, the women would drop out of the insurance pool if and when they realize they would always be subsidizing the men.".
Really I do not know what will happen in the life insurance market but in the car insurance market insuring oneself is mandatory for car owners, so nobody can escape from this obligation, subsidizing the others or not. And if all the insurance companies which act in the EU market are subjected to the same rules, maybe a new equilibrium will be reached. I repeat, I do not know, because I am not an expert of this field.
Mark observation is founded. EU Court is not a metaphysical entity but lives in an environment where pressures of power and economical lobbies are very strong. Passing from aggregate data to individual black box on cars would be an ingenious way to collect private data about nearly all the EU citizens and in the same time develop a new economic market (in a technological area) for black box producers and all the satellite activities.
On the other hand, I cannot avoid to note that more and more EU Court decisions appear to people as weird, schizoid and above all pejorative and oppressive. We live in a period of mounting tension, and I will not exclude some form of revolt versus an imposed system which, besides a presentable face refusing every discrimination, does not respect the different identities and nationalities.

[Antonio]

Sorry, please read "behind" in place of "besides".

[Kaiser]

Hi Antonio, thanks for the long comment. Government mandates are I think needed in some cases to make sure that insurance works as planned.

I don't think the black box idea flies. While it may appear that the decisions are based on individual data, as I pointed out, you just have to aggregate the data to reveal the inescapable statistic that women gets into fewer accidents, live longer, etc. That's because the accident data are correlated/confounded with the gender data. This sort of decision can only be made by a court that failed to listen to the statisticians.

[Antonio]

Kaiser, thank you your your reply. I hope to be not so boring as I am long. :)

* A point of matter.
Yes, the difference in the accident rates among men and women is inescapable. But it is not the same thing to decide or not to decide upon gender. I don't like analogies, but consider the following (hypothetic, I don't want to offend anybody) situation. Suppose that blacks commit crimes more than whites. Nevertheless, a judge has to pass judgement based on the evidences showed during the trial, and nobody would accept that the judge could found his decision on the race of the charged man. The statement "a black is more frequently guilty than a white" should be true anyway, but it is different if sentences are taken on each individual case or on the average behaviour of the group the accused person belong to.

So, even admitting the difference in accident rates, it's a matter of principle that with gender and county (and not countRy as I have written before, forgive me) of residence a driver has to pay according to the average behaviour of other people having the same sex and the same residence, while with black box data he has to pay only on his own behaviour.

And, by the way, it should be that black box data allow a premium determination more efficient than gender and county of residence.

It's an argument which is difficult to discuss for me, so I'm trying to clarify all the aspects for myself in primis. I have also read the following page about statistical discrimination:
http://filipspagnoli.wordpress.com/2010/07/24/statistical-discrimination/

[asigurare auto]

I don't want to offend anybody) situation. Suppose that blacks commit crimes more than whites. Nevertheless, a judge has to pass judgement based on the evidences showed during the trial,

[Lance Tankman]

I feel like although things such as someones circumstances affect their insurance rates, i think its necessary. I know its a little bit different in canada. I read about some insurance in vancouver and its a different set up for sure. Good post.