Spin, spin, spin away

From a purely graphical perspective, the following NYT chart (link) is well executed:

Labeling is always a challenge with scatter plots. Here, they have 54 points, and the chart still doesn't look too crammed. I like the axis labels, and the clear labeling of the four quadrants.

I also like the vertical scale that goes from 4 to 8, despite the scoring range going from 1 to 10. This trims unneeded whitespace and magnifies the differences between nations.

***

In the Trifecta checkup, we also care about what the key question the chart is designed to answer, and how it relates to the graphical element. According to the subtitle, this chart showed that "the nations with more progressive tax rates had happier citizens."

This conclusion certainly does not jump off the page. Reader Christopher L. who submitted this chart found "no obvious trend." (Given the source, I suspect it's the researchers who drew that conclusion.)

There are lots of unanswered questions in an international comparison of subjective results of this type:

• How were the 54 nations chosen?
• Is the year 2007 representative of the recent situation in every one of these countries? Were there any tax reforms in 2007 in any of these countries?
• How reliable is the Gallup poll in each of these countries? How large are the sample sizes? Is it the same survey?
• Why is the difference between the highest and lowest tax burdens the right measure of progressiveness? And are they using the marginal tax rates or the average tax rates? (Judging from the Wikipedia page, there is a lot of arbitrariness in determining a country's tax rate.)
• Are the two data sources comparable? Happiness is a personal question while the range of tax rate is an aggregate metric, with each individual only experiencing one tax rate.

These are not trivial questions. If the data is bad, no amount of graphical magic can save it.

Most vexing for a display like this is that it forces the reader to look for the impact of tax burden on happiness. That's how the question is framed. There is nothing in this chart, though, that suggests that tax rates can explain happiness, and certainly nothing to suggest that low tax rates cause more happiness.

I call this story time. Put up some data, then spin stories and spin away.

 

 

 

Comments

[Andy W]

There actually is a slight trend between the two variables, but the background quadrants disguise it! See some of the discussion and comments on Albert Cairo's blog, http://www.thefunctionalart.com/2013/04/trend-lines-in-scatter-plots.html

Of course that doesn't mitigate any concerns of data quality. Cross-national comparisons in the social-sciences(regardless of topic) almost always have some of these problems (non-random selection of nations, incomparable stats. between nations, etc.)

[loveactuary]

Also not clear to me why delta(highest, lowest) burdens is the right measure. I have seen "fraction of all tax revenue earned by top X%" used before, and under some of those definitions there aren't many more progressively taxed countries (http://taxfoundation.org/blog/no-country-leans-upper-income-households-much-us)

Or perhaps the skewness associated with the distribution ...

[jlbriggs]

I wouldn't have any idea what number would be the right one to use to determine how progressive a nation's tax strategy is.

But regardless what number you choose...you should make sure that it supports your headline (or, well, vice-versa) ;)

According to my count, that's 7 countries that fall clearly within the 'more progressive taxes, happier' quadrant, and 21 that clearly fall in the 'more progressive taxes, less happy' quadrant (and 19 in the 'less progressive, less happy').

I'm sure you can find a way to draw a trend line through that plot that makes it look like it supports the conclusion, but a muddled square block of data points is hardly a correlation.

That's an awful lot of spin, indeed. :)

[NR]

I found the labels very confusing. The problem is that the chart description says that "those with more progressive tax rates had happier citizens" but in the chart labels, "more progressive tax rates" means "more progressive [than the US]"...and like jlbriggs said, conflating the two uses of "more progressive" actually contradicts the message!

[Jeff]

The place where this fails from a graphical perspective is in the treatment of the quadrants. I also started to compare the number of points in each, but this doesn't have much to do with whether the two factors are correlated.

Perhaps because the claimed correlation is not strong enough to hit you over the head, the division of the space into four areas actually works against the perception of the trend. The point of doing this was presumably to facilitate comparisons in both dimensions with the U.S. However (and I acknowledge that this is subjective) the different shading suggests to me that the areas themselves are significant--that the divisions are somehow a property of the data set as opposed to a way to call out one point.

I wonder whether a plain white background with a pair of subtle dashed reference lines (or no reference lines at all) wouldn't have better served the point the author was trying to make.