In the aftermath of the Democratic victory in the 2006 mid-term election, the NYT published a column floating the idea that "it was the economy, stupid". For statistics buffs, this column provides much food for thought.
Suffice it to say, if you were my student, you would not want to hand this in as an essay. To the author's credit, he did backload the article with lots of disclaimers.
The key thesis of the piece is:
if your state wasn’t among the best economic performers in the last six
years, judged by the growth of personal income, it appears that you
were three times as likely to vote to throw the bums out.
(We'll just assume he didn't mean "you" but "your state".)
To help us understand the author's logic, I created a scatter plot, relating the change in state average personal income (2000-2006) to the change in percent of Republican seats.
He first segmented the states into two groups: the red dots had the top 10 income growth rates; the blue dots were the remaining states. Then for each group, he computed the average drop in % Republican. For the reds, it was 2%; for the blues, it was 7%. (These levels are indicated by the horizontal lines. My data are slightly different from his.) Case proven -- with disclaimers.
Some of you are already counting the dots. If you only find 42, you'd have counted correctly. The following explanation provided by the analyst is classic:
It’s easier to answer this question if you leave out the six states
that didn’t elect any Republicans in 2000; after all, they didn’t have
any to throw out. If you also remove New Hampshire and South Dakota,
where the percentage of Republicans elected dropped to 0 from 100 — New
Hampshire only has two seats in the House and South Dakota has one — a
pattern starts to appear.
I will leave the emergent pattern thesis to a future post. For this post, I am interested in the trouble with percentages. He is right to point out that for those 100% Blue states, the change in %Republican is constrained to be positive, from 0% up to 100%. For most other states, the change can be positive or negative.
Good observation but wrong remedy -- those six states with 0% Republicans in 2000 are not special; removing them from the analysis is wrong-headed. What about those states with 100% Republicans in 2000? There, the change in %Republican can only be 0% or negative. In fact, the possible range for the change in seats for each state is different, and it depends on the Republican proportion in 2000! For example, if in 2000 the Republicans held 30% of the seats, then in 2006, the change must be between -30% and +70%.
The situation is worse: the range of possible values also depends on the number of seats in each state. The fewer total seats there are, the fewer possible values that can be taken. As the author notes, with only 1 seat, you either lose it, gain it or retain it, so that the change will be either -100%, +100% or 0%. No other values are possible!
Both the above troubles arise because we use percentages to describe something discrete (number of seats). This is a difficult problem and I don't know of a general solution. 
However, in this example, because the change in seats is small across all states, regardless of the total number involved, I recommend that we avoid percentages and stick with positive, zero and negative changes.
The boxplot shows that there is little correlation between income growth and whether Republicans would win or lose House seats in 2006. Here, the states are divided into three groups depending on whether the Republicans gained, lost or retained seats in the 2006 mid-term election. The median income growth are similar in all three groups and the boxes overlap heavily.
Reference: "Maybe You Did Vote Your Pocketbook", New York Times, Nov 12 2006.
PS. If you like this post, consider sending me a holiday gift.
I agree with JF who suggested that this chart was mind-boggling. The chart accompanied a somewhat diffuse NYT article postulating that tax break or shifting medical practice or less apprehension about tired nurses or added labor-inducing stress from visiting relatives may have something to do with more babies being born in December, particularly at month's end.
.) If enough data is removed, one can produce any pattern as one pleases. One can find subsets of data to support a hypothesis of positive linear, flat linear or quadratic, as shown below.
