Bound to extremes

The New York Times continued to push the envelope by printing super-complicated data graphics (while the Economist regrettably seemed to have picked the USA Today route... more on that in a future post).  The following graphic was used to illustrate the relationship between CEO compensation and their company's stock performance.

The two dotplot lookalikes depicted the percent change in CEO pay and the change in companies stock price, in both cases, from 2006 to 2007.  The size of the dots indicates the relative value of the CEO's pay.  The gray dots depict "similarly sized" companies for comparability.

In this post, I will focus on the comparison between change in pay and change in stock price for a given CEO.  In particular, the calibration of the axis/scale is problematic.  The scale is automatically determined by an algorithm; as one switches from one CEO to another,  the graphs take on different ranges, use different axis labels, and the zero-percent points shift.

This means that the two charts have different scales.  In this example, each tick mark advances 6% in the top chart but 12% in the bottom chart.

Since the zero points do not line up, the distance between the zero and the orange dot loses meaning:  the 2.5x longer distance in the top chart actually represented the same percentage change as in the bottom chart (31% versus 28%).

In order to respect the grid-lines (white lines), the tick marks fall onto stray percentages (24%, 36%, 48%, etc.).  That's unfortunate.

What's the culprit?  This chart is "bound to extremes".  In other words, the range of the depicted data is used to determine the plot area.  The bottom chart had zero on the left edge because all the stocks depicted rose between 2006 and 2007.  It is often better to use domain knowledge to determine the plot area.  Extreme values should be omitted if they don't add to the message.  Oftentimes, by leaving extreme values in the picture, we squash the rest of the data.

This is also why programs like Excel do a poor job picking a scale.

As an aside, the use of bubbles is almost always troubling. Bubbles do not have a scale so the only information we get is relative size.  However, we can't estimate areas properly so we get the relative size wrong.  Sometimes, even the chart designer may get stumped.  In the chart of Steve Jobs, you would think his bubble (total compensation $1) would be dwarfed by all the other bubbles, as in the WSJ chart we showed the other day.  Not so.

Thanks to Todd B. for submitting this chart.

Reference: "Executive Pay: the bottom line for the those at the top", New York Times, April 5 2008.

Comments

[Jon Peltier]

The different scales don't bother me. What bothers me is that I'd like to see this data in a single chart with stock price along X and compensation along Y. When shown as orthogonal axes, the different scales wouldn't be a problem. Does the NY Times really believe we're too stupid to understand XY charts?

[Andreas]

Agree with Jon. We need small multiples of payrise vs. total pay and pay rise vs. stock price.

[Aleks]

What bothers me is that they compare compensation with stock price disregarding the CEO's ownership of the stock! Often, stock options are a lot more important than the salary.

They really should have used a scatter plot too, you don't want to spend a lifetime hovering the cursor over the 2D chart to read off the scatter plot.

[Kaiser]

Jon and Andreas: let me try it again. The primary point of the graphic is to make statements like "Mr. CEO got a 50% pay hike while his company's shareholder value increased by 10% only", implying that the CEO got overpaid. One can certainly find the information from this graphic but because of different scales, the graphical component is rendered redundant, in the sense that the distances were distorted. The distortion arose because the scales were determined by the inter-company comparisons.

An additional issue is what they mean by "similarly sized" companies, and why the size of the company matters, rather than, say, the industry they are in.

[Matt]

The change in stock price vs change in compensation is somewhat meaningless, IMHO, since for it to be meaningful, it would assume that all executives of comparable companies were paid at an equal rate in the previous year (the baseline year for the calculating the change in compensation.)

There's no real way to measure this, but if I'm a company, and by doubling the CEOs pay from a measly $1,000,000 to $2,000,000 (a +100% increase), I could bring in an extra $50 million in revenue and therefore up my stock price (but by no means double it), it's a no brainer that the extra million paid to the CEO is well worth it, since it brought in far more in extra revenue.

Meanwhile, a company who has overpaid their CEO with a huge salary the previous year and ups his compensation at a rate less than the change in stock price may appear to be doing well, but in actuality, is paying more more than they should, since the previous year's salary was way out of line.

It's interesting to have the two numbers side-by-side when judging a CEO's compensation, but I'd hesitate to try and read to much into assuming that there should always be a perfect correlation with change in stock price vs change in compensation.

[Penny Stocks]

The change in stock price vs change in compensation is somewhat meaningless, IMHO, since for it to be meaningful.Thanks for good information.