Worst statistical graphic nominated
A good question deserves good data

Nothing is as simple as it seems

Thanks to reader Chris P. (again) for pointing us to this infographics about teacher pay. This one is much better than your run-of-the-mill infographics poster. The designer has set out to answer specific questions like "how much do teachers make?", and has organized the chart in this way.

This post is about the very first chart because I couldn't get past it. It's a simple bar chart, with one data series indexed by country, showing the relative starting salary of a primary-school teacher with minimal training. This one:

Sosashable_teacherpay

The chart tells us that the range of salaries goes from about $12,000 at the low end (Poland) to over $65,000 at the high end (Luxembourg), with U.S. roughly at the 67% percentile, running at $42,000 per year. The footnote says that the source was OECD.

The chart is clean and simple, as a routine chart like this should. One might complain that it would be easier to read if flipped 90 degrees, with country labels on the left and bars instead of columns. But that's not where I got stuck... mentally.

I couldn't get past this chart because it generated so many unanswered questions. The point of the chart is to compare U.S. teacher pay against the rest of the world (apologies to readers outside the U.S., I'm just going with the designer's intention). And yet, it doesn't answer that question satisfactorily.

Our perception of the percentile ranking of the U.S. is fully determined by the choice of countries depicted. One wonders how that choice was made. Do the countries provide a nice sampling of the range of incomes from around the world? Is Poland truly representative of low pay and Luxembourg of high pay? Why are Korea and Japan the only two Asian countries shown and not, say, China or India? Why is there a need to plot Belgium (Fl.) separately from Belgium (Fr.), especially since the difference between the two parts of Belgium is dwarfed by the difference between Belgium and any other country? This last one may seem unimportant but a small detail like this changes the perceived ranks.

Further, why is the starting salary used for this comparison? Why not average salary? Median salary? Salary with x years of experience? Perhaps starting salary is highly correlated to these other metrics, perhaps not.

Have there been sharp changes in the salaries over time in any of these countries? It's quite possible that salaries are in flux in less developed countries, and more stable in more developed countries.

Also, given the gap in cost of living between, say, Luxembourg and Mexico, it's not clear that the Mexican teacher earning about $20,000 is worse off than the Luxembourger taking home about $65,000. I was curious enough to do a little homework: the PPP GDP per capita in Luxembourg was about $80,000, compared to $15,000 in Mexico, according to IMF (source: Wikipedia), so after accounting for cost of living, the Mexican earns an above-average salary while the Luxembourger takes home a below-average salary. Thus, the chart completely misses the point.

***

  Jc_trifecta Using the Trifecta checkup, one would address this type of issues when selecting the appropriate data series for use to address the meaningful question.

Too often, we pick up any data set we can lay our hands on, and the data fails to answer the question, and may even mislead readers.

 

 

 

PS. On a second look, I realized that the PPP analysis shown above was not strictly accurate as I compared an unadjusted salary to an adjusted salary. A better analysis is as follows: take the per-capita PPP GDP of each country, and the per-capita unadjusted GDP to form the adjustment factor. Using IMF numbers, for Luxembourg, this is 0.74 and for Mexico, this is 1.57. Now, adjust the average teacher salary by those factors. For Luxembourg, the salary adjusted for cost of living is $48,000 (note that this is an adjustment downwards due to higher cost of living in that country), and for Mexico, the adjusted salary was inflated to $31,000. Now, these numbers can be appropriately compared to the $80,000 and $15,000 respectively. The story stays the same.

 

 

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Ken

The problem with teacher pay is that in most countries it is virtually all public, and the public systems act virtually as a cartel, and a cartel that is quite prepared to produce a poor product if it keeps the costs low. Probably the best comparison would be to find private sector employees with comparable level of skill and compare their salaries.

Bill

These are the countries that are members of the OECD.

Gary

There are good reasons to look at starting pay, as that removes demographic effects (older vs. younger populations of teachers) and focuses on the question - who is being attracted into the field?

But if that's the question, not accounting for cost of living is a fail. Rather than going into GDP/PPP comparisons, Ken's suggestion makes more sense: look at ratio of teacher salary to alternatives for college graduates.

Finally there's no way average starting salary for a teacher in the US is 42K either. Average overall salary, maybe. Here in very well paid Montgomery County MD teachers start at 46K. If you include all benefits 42K is within reason, but not salary. Is there some adjustment already in the OECD data?

Rick Wicklin

From a statistical point of view, this plot is lacking an important feature: confidence limits (or "error bars" as they are known in some fields). Now, to be clear, I don't recommend putting confidence limits on a poster that is intended for general consumption! However, the fact that the nations are RANKED by starting salary implicitly suggests that Korea is "better" than Germany which is "better" than Ireland, and so forth.

Replacing the bar chart with a dot plot that includes "error bars" for the first and third quartiles would help convey the message that teachers in these nations have a range of starting salaries, and would indicate how wide the range is. Presumably, Luxembourg has a narrow range whereas other countries (USA?) have larger variation.

For an idea of how the dot plot might look (using different data), see http://blogs.sas.com/iml/index.php?/archives/108-Ranking-with-Confidence-Part-1.html

Rahul

I think you are being unfair to the original graph source. You can always come up with many more questions than what is shown on the plot. I'm curious, could you come up with a graph that answered all the points you brought up?

I can count 6 to 8 different points you question. Let's see someone make a plot that answers all of that?!

jana

Good god, you have done our eyes a service by not including the full original graphic from Soshable! Precisely why STEMs Need Blooms:

http://vimeo.com/23055140

Vivien

To be honest I feel Ortiz will stop Diaz, though it'll be late, I just think the age of Diaz (and the poor quality he's been facing recently) and the speed and power or Ortiz will see him stopped. It just is smart that you would want to pay only for good information. The first reason for people using the online sites for football betting is that it is so convenient to place bets in this manner.

The comments to this entry are closed.