Junk Charts

Turning the table

We recently showed an example of when data tables worked well to clarify the data.  Last week, there was an example from the Times which did the opposite.

The accompanying article boldly claimed that

the 40-yard dash stands above them all as having the strongest correlation to success in the NFL.  The three-cone drill, the shuttle run, the bench press -- none correlate to NFL success.  The 40 is king.

Further, it cited Bill Barnwell from FootballOutsiders.com who created an "index" using both 40 time and body weight that is "an even better predictor than 40 time alone".  In other words, this formula

does the trick.

The data table, shown above, presumably clinched the case.

We were mystified when we put the data to the test, however.  Among the set of 15 running backs, the Index did not predict the Yards Per Carry at all!  The Index explained only 8% of the variation in Yards Per Carry between the backs.

The data table obscures this bivariate relationship.  As it was sorted by the Index, we would look for the column showing Yards Per Carry to be naturally sorted in the same order.  But it is hard to tell the trend from the noise in a table.

What went wrong?  It turned out neither 40 Time nor Body Weight had any relationship with Yards Per Carry.

These variables did not explain the range of Yards Per Carry attained by this set of running backs.

Finally, we found strong correlation between 40 Time and Body Weight.  (The heavier you are, the slower you run!) This meant that both variables contained similar information and some unlikely formula involving the two would be unlikely to perform significantly better than each variable alone.

So we are left to turn the table on the table.  More pertinent evidence is needed to prove the case.

The entire analysis suffers from survivorship bias as only the top running backs are examined, and no adjustment is made to deal with wide-ranging tenures.  Apparently, there is more data available in a book.  There is no indication of how the model shown above was validated.

Reference: "The Race of Truth: 40-Yard Times Can Tell the Future", New York Times, April 27, 2008.

 

Running in the rain

Reader Eduardo is unhappy about the embellishments in this Nikeplus chart of miles ran by day; "pretty but misleading" he wrote us to say.  This is a clear case of more is less.

As a data graphic, it doesn't work.  The reflections don't work.  Perhaps Nike wants to remind all you super-dedicated Nano-wearing runners what it's like to run in mist or rain!  To quote Eduardo: "The bars start at -1! I guess it is motivation."  An extra mile for everyone.  The rounded corners make it harder to read the level.

Speaking of bar charts, I want to follow up on an exchange from March.  In that example, we claimed that not starting bars at zero misrepresented the relative lengths of those bars.  The chart showed counts of baseball players implicated in the Mitchell Report by position.

This distortion arises from taking the same length off each bar regardless of the data.  As a result, the ratios of the lengths between the bars have been changed drastically.

For example, the ratio of P/3B in the top chart is 31/9 = 3.4 but in the bottom chart, it is 23/1 = 23!

Believe it or not

Via Social Science Statistics blog, I found this article in the Times about baseball's longest hitting streaks.  The authors ran 10,000 simulations of "baseball seasons using historical data to come up with a probability distribution of the longest hitting streak in each season.  They showed the following chart.

The record was 56 consecutive games with hits in a season, which in some circles is seen as unbeatable.  These authors -- "in a fit of scientific skepticism -- found that in any season, the simulated longest streak ranged from 39 to 109, with the median at 53 games.  They concluded that "the unlikely becomes likely".

That is sure to turn some heads.  I have a question for them as I can't make sense of these numbers.  A median of 53 meant that 50% (or 5000 out of 10,000) simulated seasons ended up with a hitting streak exceeding 53 games.  Empirically, according to here, Dimaggio's was the only one to go over 53.  Using the authors' time line of 1871 to 2005, that would be 134 seasons.  One out of 134 is 0.75% probability.  0.75 versus 50... sounds like something has gone wrong.

The article doesn't give enough details on the simulation so it is hard to understand what is going on.  I hope I am not misinterpreting their analysis.

 

Source: "A Journey to Baseball's Alternate Universe", Samuel Arbesman and Steven Strogatz, Mar 30 2008.

PS. As readers pointed out, each simulation is of all the seasons.  So the histogram is saying that the particular sequence of 134 seasons that we lived to see is not a rarity considering all the possibilities.  I'm not sure this is telling us much.  It doesn't address the question of how likely the 56-game record would be beat in the future.  It can't address this question because the particular sequence is now already set; the alternative universes are irrelevant because we can't jump from one universe to another mid-stream.

Also, readers want to have each hitter's probability be modeled rather than using the historical average; in other words, factor in opposing pitcher, home/away, etc.

I'll throw in another... there must have been an assumption of independence between one game to the next.  One would think the pressure would be so much higher on the hitter once he gets to 45, 50, 53 etc. games and it would be inappropriate to assume the hitting probability would remain the same.

Along those lines, why should the hitting probability be treated as fixed, rather than modeled as a probability distribution, which would account for variance as one of the readers suggested?

For more discussion, see this Wall Street Journal discussion.
 

Amazing baseballs

Reader Jonathan S. submitted this entry.

USA Today chartjunk:

Recycled junkart (his chart):

Jonathan noticed that the scales were off (more likely, they began with an axis that did not start at zero!  This is precisely why most graphs should start at zero).

As an aside, pitchers used to point to their (frequently untoned) physique as proof that steroids could not help; now we know better.

 

Football rankings 1.1

Long-time reader Jon sent in a different view of the QB data.  He uses a nifty tool in Excel to generate a parallel coordinates plot (also called profile plot) on which pairs of QBs can be highlighted and compared.

This chart exploits the foreground background concept very nicely.  One way to deal with abundant data is to highlight only those bits that matter to the question at hand, and relegating the rest to the background.

The gray lines in the background provide context without grabbing undue attention. He also converted every metric to a scale between 0 and 1, similar to what we did with our version.

The Eli Manning / Philip Rivers comparison shows that both QBs were below average on most of these metrics, with Manning near the bottom of each.

Football rankings 2

The above chart is another one in the NYT series on the NFL playoffs.  It evaluates the mix of passing and rushing attempts by offense.  The convoluted way by which the caption strains to tell a story indicates trouble ahead:

Of the three playoff teams that threw the ball the most, two of them come from cities known for cold weather.  Conversely, of the three teams that ran the most, two of them play their home games in milder weather.

The implication is that teams from cold-weather cities are supposed to want to rush more, and vice versa.  And the data (total of six samples) pointed to the opposite.

This presentation suffers from low data-to-ink ratio:  too much ink is spilled over not much data.  The designer arbitrarily picks one of the two variables (passing attempts, rushing attempts) as the primary, sorting variable -- trace the orderly green diamonds on the right chart.  This makes it hard to see a pattern in the brown diamonds.  As usual, a scatter plot works much better with two data series.

In the junkart version, the raw numbers of attempts are converted into proportion of attempts that were passing versus rushing.  This easy move immediately collapses the two dimensions into one.  Now, we have room to include an extra variable which matters: the average amount of snowfall in these cities.

So what does the data say about the relationship between propensity to pass and cold weather?  There appears to be very little relationship as the dots are all over the chart.  In particular, the teams playing in cities with the highest snowfall span the range of passing percents; similarly, those playing in lowest-snowfall cities also span the range of passing percents. 

The caption ignores all the blue dots, focusing only on the gray ones.  A more direct examination of the relationship reveals the folly of the so-called "not so conventional wisdom".

References: "NFL Offences Undergo a Thaw in Thinking", New York Times, Jan 5 2008; government snowfall statistics.

Football rankings 1

The Times' sports pages made wise use of graphics in a series of NFL articles recently.  Here is a rank plot (below left) comparing Jaguars quarterback David Garrard to seven other quarterbacks who started the weekend of January 5.

Simple and effective, this chart does not fuss around in showing us where Garrard ranks relative to the others. 

The junkart revision (below right) plays with a different scale: the spacing between the tick marks represent proportional differences in the underlying metric.  This gives us a little more: for example, Garrard's second rank in completion percentage is less remarkable than first thought as he essentially tied with the 3rd and 4th best while the top six were bunched between 60 and 65 percent.

But Garrard's touchdown to interception ratio stands out as the next best quarterback attained only about half his ratio.  (Todd Collins who had not thrown an interception until that time was omitted; he also had only started four games.)

References: "Two Dreams (One Big, One Tiny) Come True", New York Times, Jan 4 2008; ESPN statistics.

Hits and misses 2

In the previous post, we discussed how charts need to address the key question posed by the data.  In this case, the journalist was trying to show that police shots often go errant, and are largely unpredictable even when the distance of the target is given.

In the comments, there is interest in seeing the hit rate v. distance chart.  Because the data came to us in buckets, we do not have enough to continue the analysis.  If one were to guess, the real curve would start out with 100% accuracy at distance 0, fall sharply to a plateau in the 20-40% range at modest distances, and then drop again at large distances, decaying to zero.

Andrew Gelman has conducted this analysis for a similar problem, that of predicting accuracy of golf putts based on distance from the hole.  Here are two key charts from his paper (joint with Deborah Nolan):

The left chart is our hit rate chart above, except the golf data set is larger, allowing a curve fitting.  The right chart is the fitted curve which is a "model" for the true relationship between accuracy and distance from the hole.  The model fitted the data well.

Gelman and Nolan didn't just find any best fitting line through the data.  They started out with a trigonometric model (shown on the right), with the angle of the putt as a random variable.  With this setup, they wrote down the formula for computing the probability that the putt will fall in, that is, the proportion of success.  The angle is assumed to follow a normal distribution with the standard deviation being an unknown parameter.  The standard deviation is estimated from the available data.

Of course, the human body is a bit harder to model than the hole in the ground but this procedure could very well apply.

For more details, check out the paper (PDF).  This example is also found in their book on teaching statistics.

Source: Gelman and Nolan, "A Probability Model for Golf Putting".

The absolutely meaningless pie chart

Simon J., from New Zealand, sent this in during the recent Rugby Cup but I didn't notice it till now.  As he stated, "they do a good job confirming our views of pie charts!"  Dropkicks is a site about rugby, and other sports popular in the south Pacific.

So here is our light entertainment for Thanksgiving week:

This chart accompanied a very serious statistical analysis to address the monumental question of whether some countries were borrowing strength from foreign players.  If this is your cup of tea, follow this link.

P.S. Today I started the Junk Charts Core Collection, which include books I recommend on graphics, statistics, data mining and related topics (top right).  Some categories are sparse right now as I build out the collection.  If you have favorites, let me know and I will include them.  (I am using the Amazon interface to organize the list; if you buy books, you are buying from them.  I am not becoming a bookstore.)

11/19: Amazon seems to be having problems serving up the images.  I have turned off the image for now.  You can follow the text link above to see the book collection.

11/20: the image is up again

Cheers

This is an exemplary chart from the NYT Sports page.  It provides a clear, informative and exciting way to visualize how the baseball season has gone for the Mets this and last year.  It's been mostly up and not much down. 

We can observe the more subtle differences: last season was a steady rise with only two prolonged down periods; this season's curve is driven by two up periods (including right now), outside of which the record has hovered around two levels (0, +3).

Especially commendable is the judicious use of axis labels.  However, I'm not clear on how some of the labels were chosen.  For example, 14 games ahead seem to me a rather arbitrary one.

All in all, a job well done.

Source: "Not Only Yankee Fans Cheering for Week 22", New York Times, Aug 27, 2007