Big Data, Plainly Spoken (aka Numbers Rule Your World)

JSM panel on statistical literacy

I'll be presenting a talk on Monday afternoon at the Statistical Literary 2011 section at the Joint Statistical Meeting, organized by Milo Schield. The full program is here.

The abstract of my talk is:

Recently, pop-stats books have captured the public's favor, overcoming the negative perception of the subject of statistics. The best known examples include the Malcolm Gladwell series; the Freakonomics franchise; Ian Ayres's Super Crunchers; and the speaker's Numbers Rule Your World. Readers find these books highly accessible as each author finds a way to balance readability and rigor. What can educators learn from this publishing phenomenon? What is the role of pop-stats books in statistics courses? (Partly based on joint work with Andrew Gelman)

New developments in breast cancer screening

Mammogram screening for breast cancer has become controversial. It's an emotional topic and the science is persuasive but not definitive. The danger of over-screening is ever present not just for breast cancer but for all types of diseases. False positives do have victims although the medical establishment doesn't often talk about it.

Here are two recent articles of new studies:

Screening has little impact on breast cancer deaths: study (link)

It's very hard, and often impossible, to measure directly the impact of a health policy because we cannot randomly assign people into treatment and control groups for comparison. Once a policy is in place, everyone is subject to the policy. If, as in this case, there are women who elect not to have mammograms, it may seem like we have two comparison groups: those who did and those who did not have mammograms. The challenge is that the women who choose not to do them are different from those who choose to do them in many ways. For instance, uninsured women are probably less likely to do them, and women who don't have insurance are probably poorer, less educated, etc. Such factors could explain any change in breast cancer deaths, as opposed to the use of mammograms.

This new study does a sort of pre-post analysis. In different countries, the mammogram guidelines went into effect at different times. If such screening truly reduces breast cancer deaths, then one should observe a marked drop in deaths when comparing the period before to the period after adoption of those guidelines. Thus, we can look at the timing of the policy and the timing of drop in cancer deaths to test this hypothesis.

The analysis is tricky. Many other factors may be contributing to the decline in breast cancer deaths. So the researchers find pairs of countries that have similar health care systems but different timing of mammogram policies. Of course, the analysis would break down if the matching procedure is found to be flawed.

Their conclusion is that the timing of the drop in breast cancer deaths is unrelated to the timing of regular use of mammograms.

Mammograms: computer-aided detection doesn't help (link)

This study confirms earlier findings that having computers help radiologists read results does not lead to higher accuracy. Sadly, it leads to over-diagnosis. This reminds me of the issue in Chapter 1 of Numbers Rule Your World: this technology may be worthless but what if patients perceive them to be useful?

Neat reads 9

Two recent articles worth reading:

Decoding your email personality, NYT: (link)

This describes the field called "forensic linguistics", the use of statistics to link (or delink) writings from their writers. Mark Zuckerberg is being sued again, this time by one Paul Ceglia who has produced emails (saved as Word documents) that show that he may own half of Facebook. Mark hired a linguist who argues that he can show that those emails are "unlikely" to have been written by Mark. Language Log has some additional insights here.

This is another fascinating example of statistics used in real life. I find it hard to believe that in this case, it would be possible to say with acceptable confidence whether Mark is the real author or not.

Counting calories? Your weight-loss plan may be outdated, NYT (link)

Some new conclusions from a long-term study that tracks people for decades to try to understand the link between food consumption and weight gain/loss. I'm going to need to find time to read the underlying study but certain aspects of this class of health studies have always bothered me.

First, the results are often expressed as: milk is good, cookies are bad, butter is ok, etc. This presentation gives the impression that all components of a diet are independent of each other. If milk will rise my weight by 0.2 percent and cookies by 0.3 percent, then if I take both, my weight would go up by 0.5 percent. Unfortunately, things are not so simple. In this example, cookies and milk are often consumed together; someone who chooses to eat fewer cookies is likely to also drink less milk.

Second, these studies report averages for the entire population but surely there are subgroups that differ from the average person.

 

 

Know your data 7: they are watching you

The NY Times features a startup called Social Intelligence, and their business is to create profiles of you and me from publicly available information online. They sell these profiles to employers who want to screen job candidates.

Here are a few questions worth pondering:

• What does "public" information mean? Say, you have a Facebook account in which your "wall" is only readable by friends. Is that information public or private?
• What does the CEO mean when he says "[the current practice of] Googling someone is ridiculously unfair"? What is Social Intelligence if it isn't some form of Google?
• Should there be a reciprocal business, that is, that job candidates can gain access to profiles of their prospective co-workers and bosses? Would employers be as eager to use such a service knowing that their own profiles are also accessible?
• Will people be allowed to "buy silence"? If so, would the law consider this a form of discrimination?
• How can Social Intelligence (or anyone else or a computer for that matter) know whether the information gathered online is true or false? How can the startup prevent enemies from defaming people? How can the startup prevent job candidates from seeding the Web with positive spam about themselves?
• How does Social Intelligence build profiles on people with common names like John Smith, Peter Chan, etc.? How accurate is the matching to individuals? And how is such matching accomplished?
• How does Social Intelligence read sarcasm, irony, jokes, etc.? With online writing, even humans sometimes have trouble understanding each other. The types of things employers are looking out for are precisely those that could most easily be misconstrued. The article contains an example: “This Is America. I Shouldn’t Have to Press 1 for English.” That is interpreted as a negative indication of character. What if this person is citing a line from a famous standup comedian?

The most annoying sentence of the whole article is the one that tells people to stop worrying because "job candidates must first consent to the background check." They forgot to add that "this is a free country, and you can choose to not apply for our job." Because that is precisely what they are talking about. It's the same annoying attitude when Apple says you can use iPhone only if you consent to 100 pages of terms and conditions.

With just about everything, lawyers are the true winners.

Misdiagnosing Groupon's problem

The WSJ has an article about "Groupon's Boston Problem: Copycats" (here, paywalled). A better title for this article should be: "Groupon's Boston Problem: Dealseekers".

As I discussed before, Groupon's merchants benefits only if the coupons are targeted to those customers who are new to the business or existing customers who would increase their total spending. Dealseekers are the bane of (most) Groupon merchants but also the fuel for Groupon's growth. As these merchants point out, dealseekers show up and spend the minimum amount to qualify for the Groupon deals. If the coupon offers 50% off if you spend more than $50, then the dealseeker spends exactly $50.

This is a less obvious example of adverse selection in marketing. Credit card companies have long known about this problem - the smart ones don't compete on rates. Whoever offers the lowest rates will attract the worst-quality customers because other banks are not giving cards to them. Same thing with insurers: that's why Warren Buffet always says price to make a profit, not for market share.

In the case of competing offer-deal sites, i.e., the copycats, the winning site is the one that loses the merchant the most money.

***

In the same way that "Restaurant Weeks" are here to stay, Groupon will serve some merchants and some customers well. But it's important not to mistaken those for the average merchant or the average customer.

Structural unemployment in black and white

A popular argument is circulating out there. You may have heard this one:

college graduates have lower unemployment rate than non college graduates; therefore, if there are more college grads, there will be more jobs and fewer unemployed.

This argument has been made by many, including Bernanke as I discussed here.

Unfortunately, this is a case of mistaking correlation with causation. Producing more college graduates will produce more indebted young people for sure. Producing more college graduates will not by itself create new jobs, unless you're talking about university teaching jobs in certain departments. The argument would make sense if there were an under-supply of college grads today but reports tell us that the unemployment rate among newly-minted college graduates is about 50%.

***

Warning: I now provide an unsolicited reply to Michael Kogan, who left the following comment on a blog by Paul Krugman:

Can you elaborate on why the structural unemployment story is false? The link you posted assumes the rate of "progress" is constant. But why should it be so? Many new industries today require half-way literate computer-savvy people. The unemployment among people with Master's degrees is very low - yes it is double what it used to be during Clinton years but it is still very low. And if you ask a more specific question: What is the unemployment rate of people with M.S. in Electrical, Computer or Mechanical Engineering from a top 100 University during Clinton years versus today - I bet you will come up with a very similar number - zero or close to it

This seems like a reasonable question, and I'm sure Michael is not the only one wanting answers.

To paraphrase his question: does the fact that people with Ivy-league technical degrees have close to zero unemployment rate (assuming he is right) prove that we have a "structural unemployment" problem?

***

Our unemployment number is an average of all segments of the population, including the (tiny) segment of people with Ivy credentials and technical degrees.  (See diagram on the right.) 

In my book, I caution against carelessly throwing around labels (age groups, racial groups, etc.). This is a prime example.

The fallacy in this type of argument can be seen if we use different labels. Substitute "Ivy elite with technical degrees" with "white folks"; substitute "non elites" with "colored folks". The fact is still true: white folks have lower unemployment rate than colored folks. Now, are we going to say the key to solving unemployment is to make colored people white?

Pointing to the difference in unemployment rates between college grads and non college grads and concluding that we should produce more college grads is not logically different from wanting to make colored people white. One is impossible, so is the other.

Here is another view that shows why the math can't work out. The belief is that we can take people from the bucket labelled as "A LOT OF PEOPLE" and move them to the bucket called "A tiny Elite of people..." If we do so, these people will magically find jobs. The more people we shift over, the more people will find jobs!

However, the employment status is determined by a lot of factors, of which the supply of college degrees is but one. There is a fixed number (small) of Ivy-League schools and only so many technical degrees granted (even smaller) per year from these schools. The solution looks nice on paper but it is not practical. It also ignores the fact that many of those who aren't going to college probably chose not to go to college. Just ask Bill Gates.

As I said before, producing more college grads will have two predictable outcomes: lower average salary for all college graduates, and more young people burdened with school loans. I think these same economists will label these "unintended consequences". Should we be implementing policies that have predictable "unintended" bad outcomes? 

The Economist implicates historians for bad statistics

This piece by the normally sane Economist was nominated to me via Twitter for a place in the Junk Charts hall of fame (This HOF was also a suggestion, it doesn't exist yet). While the chart is indeed quite shocking, the quantitative reasoning behind this piece, ironically titled "Quantifying history", is what deserves ignomity. So, I am posting it on Numbers Rule Your World instead.

The entire chart can be summarized in one sentence: human population has exploded in the 20th century. Any other commentary can be interpreted as a different way of stating this fact.

I said I wouldn't comment on the graph but I can't resist: the "21st century" is represented, with proper seriousness as befits a footnote, by the decade 2000-2010. You could be let off on probation in ordinary situations but when the chart presents the data (economic output and years lived) as proportions of the total sum, then the failure to either project to 2100 or to omit the 21st century is an offense that shocks the conscience.

***

The accompanying text posits a "democratic view" that "one person's life is just as much a part of mankind's story as another's."

In statistical terms, this is stamping out diversity, and treating every individual as the average. As I pointed out in Chapter 1, ignoring variability is a horrible thing. Lots of individuals have disporportionately large influence on the course of history. To name a few that comes to mind, George Washington, Nobel, Newton, Einstein, Shakespeare, Beethoven, Lenin, Hitler, Mao, etc. etc. cannot be replaced by any randomly selected person. Today, in the US, we have a special day in celebration of selected few individuals who have changed the course of history.

History is the stories of outliers, and certainly not the story of the average. So, the entire concept behind the chart is rotten.

I also reckon that while the population has exploded in the 20th century, what has also exploded is the number of poor people. Again, one must pay attention to the distribution of people, not just the average person.

Finally, what is "economic output (1990 $)"? This metric is a recent human invention, and it measures specific conditions that are deemed important to measure by the current consensus opinion of economists and politicians. I think at least some explanation is on order before using this metric to compare people across centuries.