Love this xkcd comic (link). Would have been even better cut to the final third. Great food for thought for predictive modelers.
A few years ago, some economists wanted to change their last name because economists with last names beginning close to the front of the alphabet have greater chance of winning the Nobel (according to their research). (link to PDF)
Andrew Sullivan highlights the significance of early voting in U.S. Presidential Elections (link). His second quote (from The Fix) requires a visit to your statistical adviser.
Andrew considers this an "important point":
Democrats generally vote early more than Republicans. In the five 2012
swing states where a 2008 early voting breakdown is available, Democrats
voted early more than Republicans in all five. Even in a very good GOP
year in 2010, Democrats voted early more often than Republicans in North
Carolina, Iowa and Nevada.
I'm afraid many will misinterpret this statement. What really matters is which party succeed in pushing more wavering voters to vote for them early, and thus nail down those votes. It would give Democrats no advantage if those early voters are hardcore party-line voters. The hardcore Democrat can cast his/her vote on Election Day or any day prior but it's still one sure vote.
In other words, not every early voter is alike. Early voters who are independents, previously undecided, wavering, etc. are more useful than early voters who are party-line voters.
There may be an adverse selection effect, in the sense that undecided voters are less likely to vote early since, well, they haven't made up their minds. So the people who are voting early are really the ones you have in your bag.
For me, the significance of early voting are these two points :
Early votes skew later voting propensity (which is why I don't think it's a good idea). This is similar to the East Coast/West Coast issue: by the time people in the West Coast wake up, there would already be results from the East Coast, or at the minimum, results from exit polls. This can have the effect of causing some people to stay home or other people to rush out to vote. It also allows campaign managers to take remedial action before Election Day.
Early votes is an insurance policy. It guards against gaffes leading up to Election Day.
This WSJ reporter doesn't much like statisticians, growth charts, and CDC (link). Her headline is "Is Baby Too Small? Growth Charts Make It Hard to Tell." This is her lede:
Elias Thorsteinsson weighed 6 pounds, 11 ounces, when he was born in January 2010. That put him in the 25th percentile of U.S. babies, meaning 25% of babies were smaller than he was at birth. But he didn't gain weight as quickly as other infants and dropped to the 1st percentile on the pediatric growth chart when he was 6 months old...Elias gained weight rapidly once he started eating solid foods. His head grew even faster—to the 99th percentile on the head circumference chart. His pediatrician wanted to do an MRI to rule out brain abnormalities. But Elias was developing normally, so Mrs. Stebbins refused. "We'd been through the wringer with these percentiles," she says. "I said, 'I have a big head. His father has a big head. He'll grow into it.' "
Is this damned lies and statistics or is this calling everything that has four legs a pig?
She goes on to complain, among other things, the existence of CDC and WHO standards which are not identical, and the existence of pediatricians who don't "always take the time to fully explain what the growth charts mean". (my italics)
The history of growth charts (height and weight tables) is fascinating, and not widely known.
They were pioneered by actuaries rather than doctors. In other words, they were first created for commercial reasons -- to enhance corporate profits of insurers -- rather than for health reasons.
The first charts were descriptive. It merely gave the body shape for the "average" person of a certain age and gender. Eventually, these charts became prescriptive - people who are not average are rated as "overweight", "obese", "underweight", etc. It's a big difference between knowing you're 50% above the average and being told you have to slim down, which is to say, you should be average.
Now, it appears that the growth charts may become detached from reality. The reporter includes an unattributed quote from the CDC: "The CDC says it doesn't plan to adjust its charts because it doesn't want the ever-more-obese population to become the new norm." This sounds like the standards will now be detached from the data, and simply be whatever the CDC wants us to weigh.
PS. This is not an anti-CDC post. There is an interesting problem here relating to using indices. If we keep using average BMI as the "target", and we are failing to revert to the mean, then the "target" will keep moving.
Every election year, journalists have a feast over the hundreds of poll results that never seem to end. They frequently abuse language as they try to explain what the polls mean. Because polls are small samples of people, poll results can only say so much. Specifically, when races are tight, they don't tell us much. This lack of clarity creates a certain nervousness among the prognosticators.
It was refreshing to see this headline: "Among Republicans, Santorum in statistical dead heat with Romney". (link) When was the last time the news tell us two candidates are neck and neck? The actual result was 30% Santorum to 28% Romney. You'd expect a headline like "Santorum slightly in front of Romney", even though the difference between them is statistically meaningless.
Some other media called this a "virtual tie". I hate this term. A "virtue tie" is a tie that really isn't. Alternatively, a "virtual tie" is nearly a tie. Neither sense of the word is applicable here. It is a tie, nothing "virtual" about it. In fact, when the Washington Post prints "Polls show Rick Santorum virtually tied with Mitt Romney nationally" (link), it gives the impression that Mitt Romney is slightly ahead... that story certainly did not come from the Pew poll. By the way, the Post manages to print this article and mention two recent polls without actually citing any numbers!
I just have to re-print the following table from Pew's press release here. Reporters frequently ignore the margin of error (again, this Yahoo reporter did well to feature the margin of error in the 2nd paragraph). Because these are polls with very few respondents, the margin of error is plus/minus 5 percentage points or more (for any subgroup being analyzed).
The 30% to 28% comparison was made for respondents who are "Republican or leaning Republican registered voters". This means the margin of error is given by the fourth line up from the bottom, which is plus/minus 5 percentage points. This means that the poll can only tease apart differences of larger than 10 percentage points. Notice that for any result concerning Republican voters (excluding those who said they are leaning Republican), the margin of error is even larger, at plus/minus 6 percent.
Yes, if you think these polls are useless to measure tight races, you'd be right.
On Dec 28, Cristina Fernández de Kirchner, the President of Argentina, received some bad news. She has been diagnosed with thyroid cancer, and must take a leave of 20 days to undergo surgery (on Jan 4). The discovery of the cancer was a result of routine screening.
The Economist blog reported this unhappy occasion, telling readers that "Ms Fernández is the latest of a series of South American presidents to fight cancer." Perhaps we are supposed to infer that being leaders of South American countries is a hazard to one's life (otherwise, what's the point of writing about it?). The Council of Foreign Relations even picked as a "top trend of 2011" in Latin America "the Region's Presidents' Battle Cancer".
The next day, Reuter's reporter Daniel Wallis told us that Venezuela's President Chavez has taken the correlation as evidence that "the United States might have developed a way to give Latin American leaders cancer."
Back in October, Kirchner won re-election by a landslide. Her supporters within Argentina are numerous. On Jan 3, the day before she went under the knife, they held an overnight vigil. According to the Washington Post (link), "At 1 a.m. on the morning of her operation, supporters assembled in various plazas and squares around the country, and remained there until it was finished."
Also on Oct 3, on a Financial Times blog, Jude Webber suggested that President Kirchner has benefited and will continue to benefit in public opinion from her illness:
According to this poll published last month, four days before the routine exam which revealed she had a cancerous tumour, Fernández’s popularity was 67 per cent – well above the 54 per cent she secured in the October elections. After the op, the only way is up, at least initially, pollsters say.
The Washington Post piece mentioned above has another angle. The reporter describes "what’s interesting is the panic that her illness has inspired".
It was announced that the three-and-a-half-hour surgery to remove her thyroid gland went successfully. CNN found a supporter who opined: "The truth is I was praying a lot, with all my strength, and now I feel very happy. I think the Virgin has granted a miracle."
Good news arrived, confirming that the cancer has not spread, and the President could get back to work soon.
On Jan 7, the doctors conceded that the initial diagnosis was a "false alarm"; Kirchner did not have cancer after all. According to the FT, the false positive only occurs in 2% of cases. So Kirchner was very unlucky.
How does a false alarm occur? According to Dr. Bianco, cited in this ABC News blog, typically, a lump is identified, and if it is large enough, a biopsy is taken, which means cells from the thyroid gland are taken and examined for signs of cancerous growth.
Next, Dr. Braunstein specifically tells us that doctors deliberately cause "false positives":
If there’s about a 20 percent to 30 percent chance those cells are cancerous, many doctors — including Braunstein — recommend removing the thyroid gland.
In other words, there is no simple black-or-white test of whether cancer is present in the cells. This, I suspect, is also due to the fact that only a tiny sample of cells are examined -- and the diagnosis is a general statement not just of the examined cells but of the entire gland. So there is a margin of error.
Based on what Dr. Braunstein is telling us, about 70 to 80 percent of the removed thyroid glands would be cancer-free. The chance of a "false positive' is actually really high (if you have been asked to have the operation)!
How does one reconcile this with the other report that the false positive rate is 2%. I'm not so sure.
As a side effect of this surgery, Kirchner would need to take hormone pills for the rest of her life.
P stands for pandemic. And this article nicely describes the predicament of policymakers as they grapple with the early stages of a possible outbreak in a new strain of influenza. At best, they are basing their policies on educated guesses, with the emphasis on guessing since data is in short supply.
This situation is akin to the one described in Chapter 2 of Numbers Rule Your World. At this point of the investigation, they only identified one cluster of cases, in the U.S. that can be traced back to pigs. "We don't want to overplay or underplay... we're trying to get that right." according to an official at WHO. Nice goal but unfortunately, unrealistic. The reality is that there are winners and losers on either side of this zero-sum decision. Public health advocates have little to lose from false alarms - they have the "better safe than sorry" mentality. Powerful business lobbies have much to lose from false alarms - and their voices are being heard: WHO has been warned not to call this "swine flu", according to the journalist.
Andrew Gelman has two posts (link 1, link 2) about a pollster Doug Schoen who has been making news about his survey of Occupy Wall Street protestors. Doug claimed his firm interviewed 200 people in Zuccotti Park.
Gelman cites the work of Azi Paybarah, who complained that (1) Doug drew ridiculous conclusions that contradict his data; and (2) he seemed to have altered the wording a survey question after the fact. Of course, both offenses make a mockery of polling. If you've read Charles Seife's Proofiness (my review here and here), you'd already have lost all respect for political polls like these.
We should examine why Doug Schoen even conducted this poll. The headline of the results was:
On Oct. 10 and 11, Arielle Alter Confino, a senior researcher at my polling firm, interviewed nearly 200 protesters in New York’s Zuccotti Park. Our findings probably represent the first systematic random sample of Occupy Wall Street opinion.
Our research shows clearly that the movement doesn’t represent unemployed America and is not ideologically diverse. Rather, it comprises an unrepresentative segment of the electorate that believes in radical redistribution of wealth, civil disobedience and, in some instances, violence.
Judging from this, I would say the primary purpose of this poll is to compare protestors in Zuccotti Park to "unemployed America".
In other words, the concept is flawed beyond repair.
Firstly, comparison is only possible if you have A vs. B. There is no indication that Doug's firm reached out to the cross-section of "unemployed America" to assess what they believe in. Unless Doug is unemployed America, I'm not sure how they could come to such a conclusion.
Secondly, Occupy Wall Street can never represent "unemployed America". One doesn't need a poll to know that. New York City is not America. People who live in New York City are definitely richer and more Democratic than most parts of America. New York City, as the beneficiary of bailout money, also does not have the unemployment rate of say Detroit. So, a more interesting question is whether Occupy Wall Street represents "unemployed New York City".
This overarching issue is an extension of Andrew's complaint that most of the country are in favor of taxing the very rich, which makes Doug's point that these protestors - who also support taxing the very rich - are radicals look silly. Basically, this poll has no reference point, without which one can not draw any conclusions.
Thirdly - and this is a commentary on the general media coverage rather than Doug specifically, there is this strange idea that if you're not one of the "unemployed America", you have no standing to advocate for "unemployed America". That's like saying a white person cannot possibly support civil rights because you're not a colored person.
Krugman points to this plea from Robert Samuelson to save the U.S. Statistical Abstract. Under pressure from Congress to "save money", the Census Bureau will disband the small team that assembles publications on the statistics of the United States. Apparently, this move cuts 24 jobs and saves $2.9 million annually.
This development is disturbing in many ways:
The Abstract will simply vanish. They are not killing just the paper version. There will be no online version either.
Samuelson claims someone at the Bureau made this statement: "[The Bureau] has to choose between its basic job of devising surveys and collecting statistics about economic, social and governmental conditions and the less-important task of publicizing the results." I hope this is a mis-quotation. Presenting and publicizing data is "less important" than collecting data? The priority is totally backwards.
Terminating this statistical publication is an assault on our democracy. By not publicizing these statistics, the effect is to ring-fence this data so that only specialists will access it. By not organizing them into a publishable state, only specialists will have the time and the expertise to sift through the data and make sense of it.
Statistics is an annoying profession to those who find themselves being measured by the numbers. When things are not measured, they will be "guesstimated" with any number of assumptions. Policies will be put in place, and their effects will also be "guesstimated". Accountability becomes a vague concept, infinitely debatable. This does not bode well for effective governance.
There's a lot I can say about the ongoing salmonella outbreak investigation and the massive recall of 36 million pounds of turkey, much of which you'll find in Chapter 2 of Numbers Rule Your World.
One question you might want to ask is: how big a risk is it for you?
It should be clear that the level of risk is different for different people. For example, I don't eat ground turkey so my risk is the risk of CDC identifying the wrong culprit (a false positive), meaning that I might eat something else that would get me sick. If you do consume ground turkey, your risk is that of buying a pack of ground turkey that happens to be contaminated -- plus the risk of a false positive.
According to the official press release, about 10% of Americans consume ground turkey in any given week. This means about 1/3 of Americans consume ground turkey in any given 4 weeks (1 - (0.9)^4 assuming independence from week to week), 6/10 in any 8 weeks.
Because fresh food is perishable, the risk depends on time. Assuming a temporary source of contamination, the batches of bad food would make its way through the food cycle in a fixed amount of time. (Permanent sources would be much easier to identify because you can just test samples from each machine to find the culprit.)
The following time-line chart from CDC is instructive:
Notice a few features of this chart:
the vertical scale has some very tiny numbers, we are talking about a few cases per week across the entire country
there is a "background" level of so-called sporadic infections (maybe about 2 a week) that would be visible whether or not there is a specific outbreak
the horizontal axis plots the time of reported illness which means that the actual times of infection would be shifted to the left
it was on July 27 when CDC issued a public health warning: note that this is to the far right of the curve. Could it be that the contaminated food has mostly made it through the distribution system? That the outbreak is well past its prime?
Cargill recalled batches dated all the way back to Feb 20. Could it be true that such meat is somewhere still on sale? I hope not.
In the book, I discuss the perils of statistical modeling with so few cases, the chance of false positives, the difficulty of establishing a cause-effect relationship, the incentives of health agencies, the logic of food recalls, and so on.
David Ropeik wrote in the Guardian about our irrational fears and misperception of risks. Worth reading (link). He asks: Why, if the actual risk for any given person is so low, does it feel so scary to so many?
Russia tried to ban all vegetable imports from the EU, which is a horrible idea. It is almost for sure that by this time, the contaminated batch of greens has been completely deprecated so any such measures are no more than PR stunts. Apparently, the EU convinced Russia to lift the ban (link).
Because of fear and, I must say, the lack of leadership to tackle the fear, quite a bit of unnecessary economic losses have been suffered. The EU estimates that farmers will lose $300 million.
Also, don't forget the amount of uncontaminated produce that has been laid to waste.
One other point: the level of risk is not the same for everyone. Most E-coli fatalities in past outbreaks have been elderly women or children with already compromised immune systems. In this case, 13 of 19 deaths were adult women, a little unusual but still a concentration of risk among a subset of the population. (link)
Just like a lot of situations, the "average" risk is not useful here. It's important to know if you are in the high-risk subgroup or not.