Don't believe what you see

Mankiw's blog linked to a press release by the Congressman Jim Saxton, using CBO data to show "middle income tax burden at lowest level in decades".  The attached graph, as Junk Charts readers will immediately recognize, is classic chartjunk.  Every time the vertical axis does not start at zero,  one suspects something is amiss.  And what with the gridlines and data labels?

"Don't believe it? Check out the data source yourself."  I followed Mankiw's suggestion and was indeed surprised... but not by the great fortune of the "middle class".  The surprise was how the chart painted a dishonest picture of the CBO data.

The original chart plotted only the tax rate experienced by the middle 20% of the population.  The CBO provided data for all five quintiles; why not plot them all?  In this new chart (right), the "surprise" windfall to the middle 20% proved not to be anything special at all!  All five quintiles, especially the middle three, followed pretty much the same trend over time.  The effect of singling out the middle 20% is to deprive the context by which the data should be interpreted.

Further, what might be the result of the declining middle income tax burden?  The CBO data painted an unexpected picture.  Paradoxically, as the middle 20% see their tax rate decrease, they also earn a smaller share of the nation's after-tax income (black line at right).  At the same time, the top 1% saw their share of after-tax income double from about 8% to almost 16% (blue line).  The top 20% line is also upward-sloping although less pronounced.  So, the implication that the middle class have had it good is plainly wrong.

What is going on?  Two factors were at play and the Congressman presented only one side of the story (the tax rate).  What he omitted was that during this period, the nation's wealthy took home larger and larger shares of the pre-tax income.  This shift in pre-tax income more than offset any relative reduction in tax rate for the middle 20%.

This distortion can be traced back to the use of quintiles (or more generally, ranks).  We use them to cope with data having extreme distributions but a by-product is losing information about how extreme are the extreme values.  As demonstrated here, the quintiles from old are really different from the quintiles from today because the underlying distribution has become much more extreme.

Finally, another bit of mystery (to me) is how the middle 20% came to be considered "middle class".  Is there a widely accepted definition?

Reference: "CBO Data Show Middle Income Debt Burden At Lowest Level in Decades", Feb 21 2008.

Comments

[Anonymous Prof]

Hi Edward,

This is a great example of misrepresentation information. If you don't mind, I'd like to use your analysis in my "How To Lie With Statistics" series (looking at the ways statistics are used to misrepresent information) of posts.

Keep up the good work,
AP

[anonymous voter]

Excellent analysis.

Now only one question remains... was the misrepresentation of the data intentional or not? In other words, is our Congressman a liar or an idiot?

[nathan]

um, edward?

[Kaiser]

Some think Tufte blogs here when I'm just one who admires his work and use his chartjunk terminology in the tagline.

[nathan]

ahh, i see. that's kind of an odd assumption. although, i guess you could take it as a compliment.

[Grace]

Wow, totally fascinating and easy to understand, which makes it even more appalling how the data was manipulated. Thanks! I've been wondering about the definition of the "middle class" for awhile. Have you heard Mayor Bloomberg is working on a new definition of "poverty"? http://www.nytimes.com/2007/12/30/nyregion/30poverty.html?scp=1&sq=bloomberg+new+poverty
I'm curious to know who, if anyone, is doing similar work on "middle class."

[Another Nathan]

What exactly does the context change? I _still_ see sizable tax cuts (about 25%) in the middle class. Your discovery is that this is also true of the middle-class more broadly defined? Or that it is true for everyone but the top quintile? This seems totally consistent with Mankiw's post.

The claimed "paradox" of roughly constant income share for the middle quintile has nothing to do with tax rates. In fact, it is entirely consistent with a rise in income _levels_ at the middle quintile.

Finally, while I understand the question about scaling, it should be noted that the choice to fill the chart with the data follows the precedent of 200 years of graphics experts. (See Wainer's 2001 book Graphic Discovery pp 29-38.)

[Kaiser]

I don't know for sure what Mankiw's point is but I interpret his "Quick Quiz... don't believe it?" line to mean a tax cut boon for the middle class. In statistics, everything is relative to something else; while it is accurate to say that taxes went down for the middle 20%, it is more informative to know that this decrease is consistent with what happened to the rest of the population.

As for the paradox, I believe that when one hears that the middle class has received a tax cut boon, one is led to think that the middle class has received a bigger share of the income pie, and that hasn't been the case.

I don't have Wainer's Graphic Discovery but I do have Wainer's Visual Revelations. See pp.18 where he said "honesty requires that we start the scale at zero".

[Another Nathan]

I can't speak for your mis-impression as to the effects of a tax rate cut. Simply put, there is no reason in the world a tax rate cut would mean an increase in the income share. I don't see why I should hold Mankiw responsible for your mis-impression. Similarly, I think the reference point Mankiw had in mind was the middle-class tax burden in 1970. Indeed, he also posted the graph with all 5 quintiles so I don't think he's hiding anything.

Interesting Wainer quote. Here he is in 2005 (p. 39 of Graphic Discovery): "There is agreement among experts spanning two hundred years. The default option should be to choose a scale that fills the plot with data....It is important to remember that the sin of using too small a scale is venial; the viewer can correct it. The sin of using too large a scale cannot be corrected without access to the orignal data; it can be mortal."

Given 200 years of precedent, I think it civil to give Mankiw the benefit of the doubt rather than question his honesty or intelligence.

[Kaiser]

Another Nathan: There is serious "mis-impression" on your part as well. If you read my original post, I said I saw this chart via Mankiw's blog, and followed his suggestion to look at the data. I described what I found, and complained about how the Congressman presented the data. I never questioned Mankiw's "honesty or intelligence".

As for Wainer's quote, I'd include not starting the scale at zero as a "sin of using too large a scale".

[Another Nathan]

Kaiser: While you may use whatever language you like to describe the failure to include zero, it is clear that Wainer's definition puts it in "too small a scale" as in "the y axis has too little variation." The point Wainer is making is that when I include zero, it mutes the wiggles in my data to the point that they may no longer be visible. When I exclude zero, I can always infer where zero would have fallen. Thus Wainer's distinction between "venial" and "mortal" sins.

While you are correct that your blog technically ascribes the "dishonest picture of the CBO data" to the Congressman, my point remains. The Congressman has not mislead anyone (which presumably is why a highly esteemed economist felt comfortable posting a link). The conclusion that the middle class has seen substantial tax rate reductions--no matter how you define middle class--remains unchanged by any of the alternative evidence presented thus far.

I find it uncharitable to suggest, as you and others here do, that the presentation cited is dishonest. While I want my students to be critical readers of data, I prefer they not become cynics who suggest people are "liars" or "idiots" (see anonymous professor above) if they are interested in a different question (tax rates as opposed to income shares). The fact _is_ that tax rates have fallen in the last 30 years. Given the impending explosion in entitlement promises, that may be a very bad thing. But I see no reason to question that it is a very real phenomenon. I appreciate the additional information you provide in terms of tax shares. But it simply does not suggest that the Congressman (or, by extension, Mankiw) is dishonest.

[Charlie (Colorado)]

In fact, following up to Nathan's comments, it would appear that it's rather a classic use of junk reasoning, seeing that you present both a straw man and a red herring.

Shame.