How to know when polls are jokes

Andrew Gelman has two posts (link 1, link 2) about a pollster Doug Schoen who has been making news about his survey of Occupy Wall Street protestors. Doug claimed his firm interviewed 200 people in Zuccotti Park.

Gelman cites the work of Azi Paybarah, who complained that (1) Doug drew ridiculous conclusions that contradict his data; and (2) he seemed to have altered the wording a survey question after the fact. Of course, both offenses make a mockery of polling. If you've read Charles Seife's Proofiness (my review here and here), you'd already have lost all respect for political polls like these.

***

We should examine why Doug Schoen even conducted this poll. The headline of the results was:

On Oct. 10 and 11, Arielle Alter Confino, a senior researcher at my polling firm, interviewed nearly 200 protesters in New York’s Zuccotti Park. Our findings probably represent the first systematic random sample of Occupy Wall Street opinion.

Our research shows clearly that the movement doesn’t represent unemployed America and is not ideologically diverse. Rather, it comprises an unrepresentative segment of the electorate that believes in radical redistribution of wealth, civil disobedience and, in some instances, violence.

Judging from this, I would say the primary purpose of this poll is to compare protestors in Zuccotti Park to "unemployed America".

In other words, the concept is flawed beyond repair.

Firstly, comparison is only possible if you have A vs. B. There is no indication that Doug's firm reached out to the cross-section of "unemployed America" to assess what they believe in. Unless Doug is unemployed America, I'm not sure how they could come to such a conclusion.

Secondly, Occupy Wall Street can never represent "unemployed America". One doesn't need a poll to know that. New York City is not America. People who live in New York City are definitely richer and more Democratic than most parts of America. New York City, as the beneficiary of bailout money, also does not have the unemployment rate of say Detroit. So, a more interesting question is whether Occupy Wall Street represents "unemployed New York City".

This overarching issue is an extension of Andrew's complaint that most of the country are in favor of taxing the very rich, which makes Doug's point that these protestors - who also support taxing the very rich - are radicals look silly. Basically, this poll has no reference point, without which one can not draw any conclusions.

Thirdly - and this is a commentary on the general media coverage rather than Doug specifically, there is this strange idea that if you're not one of the "unemployed America", you have no standing to advocate for "unemployed America". That's like saying a white person cannot possibly support civil rights because you're not a colored person.

Comments

[Cody L. Custis]

'New York City, as the beneficiary of bailout money, also does not have the unemployment rate of say Detroit.'

I was feeling feisty remembering the government intervention for GM, so I asked Uncle Google about bailout money by state. Turns out, Michigan got the third largest bailout at $83 billion, which is about the same, per capita, as New York. DeeCee with a population of half a million, recieved $104 billion in bailout money, or $208,000 for each individual.

http://projects.propublica.org/bailout/main/map

[Kaiser]

Cody: thanks for the link. Useful materials. Your point about autos being bailed out too is valid. Would be nice if there are city comparisons rather than state comparisons. I suspect the financial industry is much more concentrated in the city than the auto industry is in the city.

[anthony]

Hello,
The fast and convenient online store offering your favorite British Food right here in the USA.http://www.britishfooddepot.com/