Junk Charts

Disseminating junk

Tuesday was an up day in the stock markets.  What did the headline writers think?

Wall Street resumes rally following housing report  (Associated Press; link)

What did this great housing report say?  Here was a version from Marketwatch, with my comments.

Housing starts surge 22% on apartment building

It was the largest percentage gain in 19 years and was the first increase in eight months in the sector that was at ground zero in the global economic recession.

At the minimum, they claimed to see a local maximum, perhaps even a global maximum

The housing data in winter months are especially volatile because of the weather.

Was someone hedging?  It was unclear at this point how important this piece of information was.  Some professionals expressed skepticism that the news was really positive:

"We're inclined to write this off as a weather-related fluke for now, " wrote economists for Wrightson ICAP.

Notice they hedged too, using the words "inclined to".  Perhaps they had a reason to doubt the news.  The article now cited less rosy numbers:

But despite February's gain, housing starts are down 47% from a year ago, and are down 74% from the peak in early 2006. Permits are down 44% in the past year.

And even more bad news:

The National Association of Home Builders reported Monday that its sentiment index was stuck at 9 on a scale of 1 to 100 in March.

Did that really say 9 out of 100?  Sounded really bad for a nation of optimists and grade inflators.  One had to start wondering about the headline.  Was that 22% growth a fluke?  Did it mean anything?

Now, two-thirds of the way down, the article revealed its essence:

The government cautions that its monthly housing data are volatile and subject to large sampling and other statistical errors. In most months, the government can't be sure whether starts increased or decreased. In February for instance, the standard error for starts was plus or minus 13.8%. Large revisions are common.

The key news about the sampling variability was a throw-away line ("for instance").

Here, the author was confused about standard error and margin of error.  Standard error, being a standard deviation, cannot be a negative number.  The range of 22% plus or minus 13.8% is a margin of error.  This is a gigantic range, between minus 8.2% to 35.8%.  While we conclude that the growth was statistically larger than zero, we have to wonder whether this number has historical significance (best in 19 years?)

Worse than that, in any month when the growth is less than 13.8%, this level of sampling error means the growth rate is not statistically different from zero.  Thus, the most newsworthy sentence of the entire piece was:

In most months, the government can't be sure whether starts increased or decreased.

In other words, this statistic of growth in housing starts is junk.  If we really want to examine this statistic, the survey needs to use a much larger sample.

MarketWatch is to be commended for noticing the volatility issue.  Of the dozens of articles that came up in a Google search about the housing starts data, none others mentioned this problem.

The original government press release is here (pdf), with all the fine print.

Eternal optimist

Chris P pointed us to the "Financial Comeback" calculator, surely a well-meaning joke from the folks at the Times.  Here is how one gets to make a 40% loss back in just six years!


Surely, someone has to tell them about simulation.  They have to assume a probability distribution on the annual returns, and show us some sample paths.  Using the average annualized historical return in essence wipes out all variability and no wonder it's smooth sailing upwards.  Eternal optimist.  

Here is Chris' comment:

The bad news is that the range of values it offers does not include the return on the market from last year (-31% to -36%).  I guess they are optimistic.

The interactive features of this chart, however, impressed me.  The smooth adjustment to the chart as one slides the control, including the automatic choice of appropriate axis labels, is very nice indeed.

Okay, they want to trademark the name of the calculator so perhaps this is serious.

Reference: "Calculate your financial comeback", Jan 6 2009.

The shackle of time 2

In the last post, we removed the time dimension in order to clarify certain aspects of the S&P 500 returns.  We found that with an investment horizon of five years, there was historically about a 25% chance of losing money and a 25% chance of more than doubling.

Even though we looked at cumulative returns, it was still the case that the data was serially correlated; in other words, it could be that the eventual return was not independent of the starting year of the five-year period.  To gauge this, we must return to the time dimension that was previously removed.

The chart on the right plots the five-year returns for all five-year periods starting from 1910.  What it shows is that even with longer time frames, timing or luck still plays a key role. 

For example, any such investment in the S&P 500 between late 1950s and 1980 did not double in five years no matter which year the investment was made.  Then again, if the investment was made in the 40s and 50s, no one lost money in a five-year period, similarly in the 1980s.

So the fact that we saw a 25% chance of doubling (or losing money) over history says much less about what might happen in the next 5 years than the simple number suggests.

In response to a reader's comment - the data series was described as "real total return" so these are inflation-adjusted.

A joke

For light entertainment, see this chart sent in by Chris P.  Original here.

Chris said: "his graph breaks a few rules, but it has a clear message".

The shocking, out of the box column certainly grabs attention, and it is probably true that football coaches earn too much money.  But the chart really falls down on this one issue:

What's the median salary of these football coaches?

Reference: "Academic salaries", PHD Comics.

Mind the gap

When comparing two time series, one typically wants to discuss the size of the gap as it changes over time.  This Business Week chart, for example, depicted for readers the expanding gap between intra-day high and low prices of the S&P 500 for 2008.


This chart construct is effective at pointing out large changes but lacks precision in conveying smaller differences, or trends.  It is always a good idea to plot the gap directly, as we will show below.

More importantly, a better choice of scale can help a lot.  By focusing exclusively on variability (extreme values), this chart hides the relevant information of the closing prices of the S&P.  A point spread of a 100 points means more when the index is at 800 than at 1200.  In order to capture this, we can divide the point spread by the opening price of that day so we say the gap is one-eighth or one-twelfth of the opening price. 

The junkart version makes both changes.  The top chart fixes the scale, plotting the point spread as a percentage of daily opening prices.  Relative to the original chart, the variability in the front part of 2008 was muted because the index was at higher levels back then. 

The bottom chart plots the gap sizes (lengths of the high-low lines).  It is without doubt that directly plotting the gaps showcases the key message.  The current level of volatility is more than double what occurred at the beginning of the year.

If one wants to illuminate the trend as opposed to daily fluctuations, a further improvement will be using moving averages.

For those interested, shown below is a scatter plot that compares the original point spread and the derived point spread, which shows that the change is not trivial.

 

Reference: "The Market: A Daily Roller Coaster", Business Week, Oct 27 2008.

The dog ate the margins

In his column on automated polls versus traditional telephone polls, the Numbers Guy at Wall Street Journal gave us a few entertaining quotes.

"The dog could be answering the questions, " Ann Selzer, a traditional pollster, said of automated polling, which occurs through automated voice messages to voter who record responses.  Also, WSJ cited a prominent textbook which labelled them as "Computerized Response Automated Polls -- insulting acronym intended."

Reader Mark A. brought this to our attention because of the following chart.  He wondered what the point of the vertical axis was. 

Aside from that cosmetic problem, the biggest issue is the lack of explanation.  Predictive power, pollster-introduced error, methodological error: what are these?  The article itself gives no clues.  To make sense of the chart, readers need to consult Nathan Silver's (excellent) site, fivethirtyeight.com.  (The gory details here.)

By the way, Nathan's site has a variety of nicely produced charts.  (Like this one, readers will need to dig around to collect background information to interpret some of those charts.)

Another improvement is to provide some sense of the variance in the data, either by showing more than the top five pollsters or by showing the range of errors.  Since the average pollster sits on the right edge, it is as if the right half of the chart was clipped.  In the version below, we found most polls hovering around the average, with two egregiously bad.

If we know which polls are automated and which aren't, then color the dots accordingly.

There are bench players on every chart: these are the titles, axes, labels, text and so on.  They provide background information required to interpret the chart.  They may sit in the margins but their value is not to be underestimated.

Don't let the dog eat the marginal information.

Reference: "Press 1 for Obama, 2 for McCain", Wall Street Journal, Aug 1 2008.

Proof of rampant U.S. deflation

From San Diego Tribune via the Big Picture, irrefutable proof of steady deflation due to enlightened government policy!

Fodder for thought for those curious about U.S. economic statistics.  While conventional wisdom (or publicized rationale) often claims that the recent adjustments to the core methodology remove components that are "more volatile", the evidence here suggests that the new method basically removes 3% from the previously computed rates.  Remarkable how stable this difference is over time.

This is a fantastic chart that makes its point clear and loud.  The secret is in picking the right comparables.  To vet the data analyst with shifting metrics, it is not necessary to prove that the new metric is or is not more accurate than the old.  Oftentimes, by tracking both the old and the new, the effect of the change is revealed.  Here, the adjustments just keep taking the rates down.

Reference: "The Fed's inflation guage isn't realistic, critics say", San Diego Union Tribune, April 17, 2008.

Rant of the day: Typepad's new editor continues to impress -- changing font size wipes out all hyperlinks in the text being formatted!

Wannabell

This graph, called the Bell Curve, is a wannabe.

The first hint is its asymmetry, the right tail being longer than the left tail.

Further, the helpful labelling of the "average" does not coincide with the peak of the curve.

The author of the annotation seemed to understand, calling the distribution "skewed".  A Bell Curve is not skewed.

This is a pity because the designer might have selected a different chart type if she wasn't so enamored by the bell curve object.

The data tells us about users of 30-day unlimited passes in the New York City subway system: how many trips do they typically make?  The card costs $81 while each trip costs $2 so anyone taking fewer than 40 trips in those 30 days would have been better off buying individual tickets.  The "average" user took 56 trips.  The range of trips taken was very wide, perhaps surprisingly so.

Several key pieces of information has been left off the chart.  What is the total number of riders?  Without this, there is no way for readers to understand 15,185.  What is the smallest (and largest) number of trips taken by any rider?  Visually, it appears that the horizontal axis does not start at zero.

It would have been better to show a cumulative distribution with percentages of riders on the vertical axis.  On such a chart, we can read off the median and any percentiles.  In other words, it would be much more informative.

As it stands, I like very much the annotation of the 56 trip and the 100 trip points: they are great aids to help decipher the chart.  It would be great to indicate the 40 trip point too.

For those more technically inclined: the graph also begs the question of whether it is an actual or modelled curve.  It looks too smooth to be actual data.  If it is a model, then it is definitely not a normal distribution.  What could it be?  A spline?

Reference: "In Decade of Unlimited Rides, MetroCard Has Transformed How the City Travels", New York Times, July 16 2008.

Divided nation

Professor Gelman generally believes the red state, blue state paradigm is too simplistic to describe the American electorate.  He has been sharing some of his work on his blog, and has just published a book about this topic.  Recently he produced the following chart, which is gimmick-looking but crystal clear in its message.

Here, economic and social ideology are plotted on a scatter chart, with positive values indicating conservatism and negative values liberalism.  Further, each state is represented twice on the chart, the red point for the Republicans and the blue for Democrats within the state.

This is a cluster analyst's dream data set.  The absolute separation of the Republican cluster and the Democrat cluster is astounding: imagine a diagonal line perfectly classifying all points.

We should not miss a host of details:

• as Andrew pointed out, "the big thing we see from the graph ... is that Democrats are much more liberal than Republicans on the economic dimension: Democrats in the most conservative states are still much more liberal than Republicans in even the most liberal states."  This is clear from the wide gap on the horizontal axis.
• there is a small degree of overlap on the social ideology axis so the nation is closer together on that front.
• but wait a minute, the scale on the social axis is not the same as that on the economic axis.  This means that the extremes are more extreme on the social axis: the difference between MS and VT is roughly 0.8 on the social scale while the largest difference on the economic scale is roughly 0.5.  (here, I am assuming that the scales are comparable to each other)
• there is high correlation between social and economic ideologies: the points are well-aligned along the 45-degree line
• especially on social issues, the Democrats are divided within (the elongated shape of the blue cluster).

Reference: Gelman, "Ranking states by conservatism/liberalism of their voters", June 30 2008.

Football rankings 1

The Times' sports pages made wise use of graphics in a series of NFL articles recently.  Here is a rank plot (below left) comparing Jaguars quarterback David Garrard to seven other quarterbacks who started the weekend of January 5.

Simple and effective, this chart does not fuss around in showing us where Garrard ranks relative to the others. 

The junkart revision (below right) plays with a different scale: the spacing between the tick marks represent proportional differences in the underlying metric.  This gives us a little more: for example, Garrard's second rank in completion percentage is less remarkable than first thought as he essentially tied with the 3rd and 4th best while the top six were bunched between 60 and 65 percent.

But Garrard's touchdown to interception ratio stands out as the next best quarterback attained only about half his ratio.  (Todd Collins who had not thrown an interception until that time was omitted; he also had only started four games.)

References: "Two Dreams (One Big, One Tiny) Come True", New York Times, Jan 4 2008; ESPN statistics.