Beyond the surface of poll numbers

This Upshot piece about increasing prevalence of tattoos in the U.S. linked to a Fox News Poll, summarized here by Fox News reporters.

The statistics say that 20% of voters haves at least one tattoo, up from 13% in 2007. Fourteen percent haves two or more tattoos, up from 8%.

The problem with poll result reporting are many. Let me count them:

How should we interpret those numbers? One thing not discussed is the increasing level of acceptance of what used to be a social taboo. This is happening at the same time that more people are getting more tattoos. A serious understanding has to start with how much more willing is the tattooed person to disclose their inked status today compared to years past.

A second related question is whether people with tattoos are less likely to be voters. If you compare the three sources, you'll notice an increasing imprecision in the reporting. The pollsters used the term "registered voters"; the Fox News writer turned that into "voters"; and finally, the Upshot writer replaced it with "Americans". This is probably what most readers would do anyway. For this poll question, there is reason to doubt whether people with tattoos are registered to vote in equal proportion to the population at large.

Then, as with any poll, you wonder whether a subgroup of the respondents are paying attention or just making things up. Fox News told us "Ten percent of tattooed Americans say they don’t like tattoos." This could be buyer's remorse, or it could be people not taking the poll seriously.

It is often pointless to ask questions like these: "By a 73-16 percent margin, most voters say they would hire someone who has a visible tattoo." In essence, the pollster is asking someone if they would discriminate against people with visible tattoos. It's like asking someone "would you not hire someone if they are black?" You're not likely to get an accurate answer.

The sample size is roughly 1000 people, and this was designed to achieve a margin of error of plus/minus 3 percent according to the press release. Blink and you may miss the qualifier that said "results based on the full sample". When you start to look at subgroups within the full sample, you get into trouble.

Credit to the pollsters, they disclosed this information. For example, the Fox News report on the Fox News poll said "A third of those under age 30 have a tattoo (34 percent)". The margin of error on this number is about 7.5 percent which means that there was only sufficient samples to nail the estimate down to a range of 15 percent.

***

Polls are everywhere but good reporting on polls is rare.

 

Comments

[MJS]

Picking nits...

Shouldn't the 2nd sentence contain the phrase "20% of voters HAVE at least one tattoo"?

Saying "20% has" is an awkward construction to say the least and may be incorrect.

I know in my day that would have got some serious red pen.

M

[Kaiser]

MJS: No problem.. it's fixed.

[Ken]

Apparently 95% of people who answer polls agree that polls are worthwhile.

[Berry Boessenkool]

37.4% of statistical results are arbitrarily made up

;-)