The art of arranging bars

Twitter friend Janie H. asked how I would visualize a hypothetical third column of this chart that contains the change from 2016 to 2017:

Techpriorities_data_table

This table records the results from a survey question by eMarketer, asking respondents ("marketers") to identify their top 5 technology priorities in the next 12 months.

I suggested the following:

Redo_techpriorities_order1

A hype-chasing phenomemon is clearly at play. Internet of Things and wearable technology are so last year. This year, it's all about A.I. Interestingly, something like "Big data" has been able to sustain the hype for another year.

A design decision I made is to encode the magnitude of the change in the bar lengths while encoding the direction of the change in the colors. One can of course follow the more canonical design of placing the negative bars on the left side of the data labels. My decision is a subtle way of imposing the hierarchy - first I care about magnitude, then I care about direction.

Here is a third way:

Redo_techpriorities_order2

This design imposes a different hierarchy. Your eyes are drawn to the top/bottom of the chart.

Any of these designs beat the data table by a mile. It's just too much work for the reader to figure out the value of the changes from the table.


Some like it packed, some like it piled, and some like it wrapped

In addition to Xan's "packed bars" (which I discussed here), there are some related efforts to improve upon the treemap. To recap, treemap is a design to show parts against the whole, and it works by packing rectangles into the bounding box. Frequently, this leads to odd-shaped rectangles, e.g. really thin and really tall ones, and it asks readers to estimate relative areas of differently-scaled boxes. We often make mistakes in this task.

The packed bar chart approaches this challenge by allowing only the width of the box to vary with the data. The height of every box is identical, so readers only have to compare lengths.

Via Twitter, Adil pointed me to this article by him and his collaborators that describes a few alternatives.

One of the options is the "wrapped bar chart" introduced by Stephen Few. Like Xan, he also restricts the variation to legnths of bars while keeping the heights fixed. But he goes further, and abandons packing completely. Instead of packing, Few wraps the bars. Start with a large bar chart with many categories filling up a tall plotting area. He then divides the bars into different blocks and place them side by side. Here is an example showing 50 states, ranked by total electoral votes:

Umd_few_wrapped_bars

You can see the white space because there is no packing. This version makes it easier to see the relative importance of the different blocks of states but it is tough to tell how much the first block of 13 states accounts for. The wrapped barchart is organized similar to a small multiples, except that the scale in each panel is allowed to vary.

Another option is the "piled bars." This option, presented by Yalçın, Elmqvist, and Bederson, brings packing back. But unlike the packed bars or the treemap, the outside envelope no longer represents the total amount. In the "piled bars" design, the top X categories act as the canvas, and the smaller categories are packed inside these bars rather than around them. Take a look at this example, which plots GDP growth of different countries:

Umd_piledbars

 The inset on the left column is instructive. The green (smallest) and red (medium) bars are packed inside the blue (largest) bars. In this example, it doesn't make sense to add up GDP growth rates, so it doesn't matter that the outer envelope does not equal the total. It would not work as well with the electoral vote data in the previous example.

I wonder whether a piled dot plot works better than a piled bar chart. This piled bar chart shares a problem with the stacked area chart, which is that other than the first piece, all the other pieces represent the differences between the respective data and the next lower category, rather than the value of the data point. Readers are led to compare the green, red and blue pieces but the corresponding values are not truly comparable, or of primary interest.

This problem goes away if the bars are represented by dots.

***

What strikes me as the most key paragraph in the Yalcin, et. al.'s article is the following:

To understand graphical perception performance, we studied three basic tasks:

1) How accurately can we estimate the difference between two data points?
2) How accurately can we estimate the rank of a data point among all the rest?
3) How accurately can we guess the distribution characteristic of the whole dataset?

As a chart designer, we have to prioritize these tasks. There is unlikely to be a single chart form that will prevail on all three tasks. So if the designer starts with the question that he or she wants to address, that leads to the key task that the visualization should enable, which leads to the chart form that facilitates that task the best.

 

 

 


Much more to do after selecting a chart form

Brady_ryan_heads

I sketched out this blog post right before the Superbowl - and was really worked up as I happened to be flying into Atlanta right after they won (well, according to any of our favorite "prediction engines," the Falcons had 95%+ chance of winning it all a minute from the end of the 4th quarter!) What I'd give to be in the SuperBowl-winning city the day after the victory!

Maybe next year. I didn't feel like publishing about SuperBowl graphics when the wound was so very raw. But now is the moment.

The following chart came from Orange County Register on the run-up to the Superbowl. (The bobble-head quarterbacks also came from OCR). The original article is here.

Ocr_patriots_atlanta

The choice of a set of dot plots is inspired. The dot plot is one of those under-utilized chart types - for comparing two or three objects along a series of metrics, it has to be one of the most effective charts.

To understand this type of design, readers have to collect three pieces of information: first is to recognize the dot symbols, which color or shape represents which object being compared; second is to understand the direction of the axis; third is to recognize that the distance between the paired dots encodes the amount of difference between the two objects.

The first task is easy enough here as red stands for Atlanta and blue for New England - those being the team colors.

The second task is deceptively simple. It appears that a ranking scale is used for all metrics with the top ("1st") shown on the left side and the bottom ("32nd") shown on the right. Thus, all 32 teams in the NFL are lined up left to right (i.e. best to worst).

Now, focus your attention on the "Interceptions Caught" metric, third row from the bottom. The designer indicated "Fewest" on the left and "Most" on the right. For those who don't know American football, an "interception caught" is a good defensive play; it means your defensive player grabs a ball thrown by the opposing team (usually their quarterback), causing a turnover. Therefore, the more interceptions caught, the better your defence is playing.

Glancing back at the chart, you learn that on the "Interceptions Caught" metric, the worst team is shown on the left while the best team is shown on the right. The same reversal happened with "Fumbles Lost" (fewest is best), "Penalties" (fewest is best), and "Points Allowed per Game" (fewest is best). For four of nine metrics, right is best while for the other five, left is best.

The third task is the most complicated. A ranking scale always has the weakness that a gap of one rank does not yield information on how important the gap is. It's a complicated decision to select what type of scale to use in a chart like this, and in this post, I shall ignore this issue, and focus on a visual makeover.

***

I find the nine arrays of 32 squares, essentially the grid system, much too insistent, elevating information that belongs to the background. So one of the first fixes is to soften the grid system, and the labeling of the axes.

In addition, given the meaningless nature of the rank number (as mentioned above), I removed those numbers and used team logos instead. The locations on the axes are sufficient to convey the relative ranks of the two teams against the field of 32.

Redo_newenglandatlanta1

Most importantly, the directions of all metrics are now oriented in such a way that moving left is always getting better.

***

While using logos for sports teams is natural, I ended up replacing those, as the size of the dots is such that the logos are illegible anyway.

The above makeover retains the original order of metrics. But to help readers address the key question of this chart - which team is better, the designer should arrange the metrics in a more helpful way. For example, in the following version, the metrics are subdivided into three sections: the ones for which New England is significantly better, the ones for which Atlanta is much better, and the rest for which both teams are competitive with each other.

Redo_newenglandatlanta2

In the Trifecta checkup (link), I speak of the need to align your visual choices with the question you are trying to address with the chart. This is a nice case study of strengthening that Q-V alignment.

 

 

 

 

 

 


Political winds and hair styling

Washington Post (link) and New York Times (link) published dueling charts last week, showing the swing-swang of the political winds in the U.S. Of course, you know that the pendulum has shifted riotously rightward towards Republican red in this election.

The Post focused its graphic on the urban / not urban division within the country:

Wp_trollhair

Over Twitter, Lazaro Gamio told me they are calling these troll-hair charts. You certainly can see the imagery of hair blowing with the wind. In small counties (right), the wind is strongly to the right. In urban counties (left), the straight hair style has been in vogue since 2008. The numbers at the bottom of the chart drive home the story.

Previously, I discussed the Two Americas map by the NY Times, which covers a similar subject. The Times version emphasizes the geography, and is a snapshot while the Post graphic reveals longer trends.

Meanwhile, the Times published its version of a hair chart.

Nyt_hair_election

This particular graphic highlights the movement among the swing states. (Time moves bottom to top in this chart.) These states shifted left for Obama and marched right for Trump.

The two sets of charts have many similarities. They both use curvy lines (hair) as the main aesthetic feature. The left-right dimension is the anchor of both charts, and sways to the left or right are important tropes. In both presentations, the charts provide visual aid, and are nicely embedded within the story. Neither is intended as exploratory graphics.

But the designers diverged on many decisions, mostly in the D(ata) or V(isual) corner of the Trifecta framework.

***

The Times chart is at the state level while the Post uses county-level data.

The Times plots absolute values while the Post focuses on relative values (cumulative swing from the 2004 position). In the Times version, the reader can see the popular vote margin for any state in any election. The middle vertical line is keyed to the electoral vote (plurality of the popular vote in most states). It is easy to find the crossover states and times.

The Post's designer did some data transformations. Everything is indiced to 2004. Each number in the chart is the county's current leaning relative to 2004. Thus, left of vertical means said county has shifted more blue compared to 2004. The numbers are cumulative moving top to bottom. If a county is 10% left of center in the 2016 election, this effect may have come about this year, or 4 years ago, or 8 years ago, or some combination of the above. Again, left of center does not mean the county voted Democratic in that election. So, the chart must be read with some care.

One complaint about anchoring the data is the arbitrary choice of the starting year. Indeed, the Times chart goes back to 2000, another arbitrary choice. But clearly, the two teams were aiming to address slightly different variations of the key question.

There is a design advantage to anchoring the data. The Times chart is noticeably more entangled than the Post chart. There are tons more criss-crossing. This is particularly glaring given that the Times chart contains many fewer lines than the Post chart, due to state versus county.

Anchoring the data to a starting year has the effect of combing one's unruly hair. Mathematically, they are just shifting the lines so that they start at the same location, without altering the curvature. Of course, this is double-edged: the re-centering means the left-blue / right-red interpretation is co-opted.

On the Times chart, they used a different coping strategy. Each version of their charts has a filter: they highlight the set of lines to demonstrate different vignettes: the swing states moved slightly to the right, the Republican states marched right, and the Democratic states also moved right. Without these filters, the readers would be winking at the Times's bad-hair day.

***

Another decision worth noting: the direction of time. The Post's choice of top to bottom seems more natural to me than the Times's reverse order but I am guessing some of you may have different inclinations.

Finally, what about the thickness of the lines? The Post encoded population (voter) size while the Times used electoral votes. This decision is partly driven by the choice of state versus county level data.

One can consider electoral votes as a kind of log transformation. The effect of electorizing the popular vote is to pull the extreme values to the center. This significantly simplifies the designer's life. To wit, in the Post chart (shown nbelow), they have to apply a filter to highlight key counties, and you notice that those lines are so thick that all the other countries become barely visible.

  Wp_trollhair_texas

 


Mapping the two Americas

If you type "two Americas map" into Google image search, you get the following top results:

Google_twoAmericasmaps

Designers overwhelmingly pick the choropleth map as the way to depitct the two nations.

Now, look at these maps from the New York Times (link):


Nytimes_election2016_mapDem

and this:

Nytimes_election2016_mapRep

I believe the background is a relief map. Would like to see one where the color is based on the strength of support for Democrats or Republicans.

The pair of maps is extremely effective at bringing out the story about the splitting of the U.S. population. From a design standpoint, I really like it.

I love, love, love the cute annotations everywhere on the page. I imagine the designer had fun coming up with them.

Nytimes_election2016_mapRep_inset

Pittsburgh Puddle, Cleveland Cove, Cincinnati Slough, ...

***

There is an artistic (or data journalistic) license behind the way the data are processed. Most likely, a 50% cutoff is applied to determine which map a county sits atop. The analysis is at the county level so there is neccessarily some simplification... in fact, this aggregation is needed to make the "islands" and other features contiguous.

I am a bit sad that at this moment, we are so focused on what sets us apart, and not what binds us together as a nation.

 

PS. Via twitter, Maciej reacted negatively to these maps: "Horribly tendentious map visualization from the NYT makes the candidate who won more votes look like a tiny minority."

This is a good illustration of selecting the chart form to bring out one's message. If the goal of the chart is to show that Clinton has more votes, I agree that these maps fail to convey that message.

What I believe the NYT designer wants to point out is that the supporters of Clinton are clustered into these densely populated urban areas, leaving the Republicans with most of the land mass. (Like I said above, because of the 50% cutoff criterion, we are over-simplifying the picture. There are definitely Democrats living somewhere in Trump's nation, and likewise Republicans residing in Clinton strongholds.)


Here are the cool graphics from the election

There were some very nice graphics work published during the last few days of the U.S. presidential election. Let me tell you why I like the following four charts.

FiveThirtyEight's snake chart

Snake-1106pm

This chart definitely hits the Trifecta. It is narrowly focused on the pivotal questions of election night: which candidate is leading? if current projections hold, which candidate would win? how is the margin of victory?

The chart is symmetric so that the two sides have equal length. One can therefore immediately tell which side is in the lead by looking at the middle. With a little more effort, one can also read from the chart which side has more electoral votes based only on the called states: this would be by comparing the white parts of each snake. (This is made difficult by the top-bottom mirroring. That is an unfortunate design decision - I'd would have preferred to not have the top-bottom reversal.)

The length of each segment maps to the number of electoral votes for the particular state, and the shade of colors reflect the size of the advantage.

In a great illustration of less is more, by aggregating all called states into a single white segment, and not presenting the individual results, the 538 team has delivered a phenomenal chart that is refreshing, informative, and functional.

 Compare with a more typical map:

Electoral-map

 New York Times's snake chart

Snakes must be the season's gourmet meat because the New York Times also got inspired by those reptiles by delivering a set of snake charts (link). Here's one illustrating how different demographic segments picked winners in the last four elections.

 

Nytimes_partysupport_by_income

They also made a judicious decision by highlighting the key facts and hiding the secondary ones. Each line connects four points of data but only the beginning and end of each line are labeled, inviting readers to first and foremost compare what happened in 2004 with what happened in 2016. The middle two elections were Obama wins.

This particular chart may prove significant for decades to come. It illustrates that the two parties may be arriving at a cross-over point. The Democrats are driving the lower income classes out of their party while the upper income classes are jumping over to blue.

While the chart's main purpose is to display the changes within each income segment, it does allow readers to address a secondary question. By focusing only on the 2004 endpoints, one can see the almost linear relationship between support and income level. Then focusing on the 2016 endpoints, one can also see an almost linear relationship but this is much steeper, meaning the spread is much narrower compared to the situation in 2004. I don't think this means income matters a lot less - I just think this may be the first step in an ongoing demographic shift.

This chart is both fun and easy to read, packing quite a bit of information into a small space.

 

Washington Post's Nation of Peaks

The Post prints a map that shows, by county, where the votes were and how the two Parties built their support. (Link to original)

Wpost_map_peaks

The height represents the number of voters and the width represents the margin of victory. Landslide victories are shown with bolded triangles. In the online version, they chose to turn the map sideways.

I particularly like the narratives about specific places.

This is an entertaining visual that draws you in to explore.

 

Andrew Gelman's Insight

If you want quantitative insights, it's a good idea to check out Andrew Gelman's blog.

This example is a plain statistical graphic but it says something important:

Gelman_twopercent

There is a lot of noise about how the polls were all wrong, the entire polling industry will die, etc.

This chart shows that the polls were reasonably accurate about Trump's vote share in most Democratic states. In the Republican states, these polls consistently under-estimated Trump's advantage. You see the line of red states starting to bend away from the diagonal.

If the total error is about 2%, as stated in the caption of the chart, then the average error in the red states must have been about 4%.

This basic chart advances our understanding of what happened on election night, and why the result was considered a "shock."

 

 


Depicting imbalance, straying from the standard chart

My friend Tonny M. sent me a tip to two pretty nice charts depicting the state of U.S. healthcare spending (link).

The first shows U.S. as an outlier:

FtotHealthExp_pC_USD_long-1-768x871

This chart is a replica of the Lane Kenworthy chart, with some added details, that I have praised here before. This chart remains one of the most impactful charts I have seen. The added time-series details allow us to see a divergence from about 1980.

Lanekenworthy-250wi

The second chart shows the inequity of healthcare spending among Americans. The top 10% spenders consume about 6.5 times as much as the average while the bottom 16% do not spend anything at all.

Ourworldindata_nihcm-spending-concentration-titled

This chart form is standard for depicting imbalance in scientific publications. But the general public finds this chart difficult to interpret, mostly because both axes operate on a cumulative scale. Further, encoding inequity in the bend of the curve is not particularly intuitive.

So I tried out some other possibilities. Both alternatives are based on incremental, not cumulative, metrics. I take the spend of the individual ten groups (deciles) and work with those dollars. Also, I provide a reference point, which is the level of spend of each decile if the spend were to be distributed evenly among all ten groups.

The first alternative depicts the "excess" or "deficient" spend as column segments. Redo_healthcarespend1

The second alternative shows the level of excess or deficient spending as slopes of lines. I am aiming for a bit more drama here.

Redo_healthcarespend2

Now, the interpretation of this chart is not simple. Since illness is not evenly spread out within the population, this distribution might just be the normal state of affairs. Nevertheless, this pattern can also result from the top spenders purchasing very expensive experimental treatments with little chance of success, for example.

 


Brexit, Bremain, the world did not end so dataviz people can throw shade and color

Catching a dose of Alberto Cairo the other day. He has a good post about various Brexit/Bremain maps.

The story started with an editor of The Spectator, who went on twitter to make the claim that the map on the right is better than someone else's map on the left:

Spectator_brexitmaps

There are two levels at which we should discuss these maps: the scaling of the data, and the mapping of colors.

The raw data are percentages based on counts of voters so the scale is decimal. In general, we discretize the decimal data in order to improve comprehension. Discretizing means we lose granularity. This is often a good thing. The binary map on the left takes the discretization to its logical extreme. Every district is classified as either Brexit (> 50% in favor) or Bremain (> 50% opposed). The map on the right uses six total groups (so three subgroups of Brexit and three subgroups of Bremain.

Then we deal with mapping of numbers to colors. The difference between these two maps is the use of hues versus shades. The binary map uses two hues, which is probably most people's choice since we are representing two poles. The map on the right uses multiple shades of one hue. Alternatively, Alberto favors a "diverging" color scheme in which we use three shades of two hues.

The editor of The Spectator claims that his map is more "true to the data." In my view, his statement applies in these two senses: the higher granularity in the scaling, and also, the fact that there is only one data series ("share of vote for Brexit") and therefore only one color.

The second point relates to polarity of the scale. I wrote about this issue before - related to a satisfaction survey designed (not too well) by SurveyMonkey, one of the major online survey software services. In that case, I suggested that they use a bipolar instead of unipolar scale. I'd rather describe my mood as somewhat dissatisfied instead of a little bit satisfied.

I agree with Alberto here in favor of bipolarity. It's quite natural to underline the Brexit/Bremain divide.

***

Given what I just said, why complain about the binary map?

We agree with the editor that higher granularity improves comprehension. We just don't agree on how to add graularity. Alberto tells his readers he likes the New York Times version:

1times_brexitmap

This is substantively the same map as The Spectator's, except for 8 groups instead of 6, and two hues instead of one.

Curiously enough, I gave basically the same advice to the Times regarding their maps showing U.S. Presidential primary results. I noted that their use of two hues with no shades in the Democratic race obscures the fact that none of the Democratic primiaries was a winners-take-all contest. Adding shading based on delegate votes would make the map more "truthful."

That said, I don't believe that the two improvements by the Times are sufficient. Notice that the Brexit referendum is one-person, one-vote. Thus, all of the maps above have a built-in distortion as the sizes of the regions are based on (distorted) map areas, rather than populations. For instance, the area around London is heavily Bremain but appears very small on this map.

The Guardian has a cartogram (again, courtesy of Alberto's post) which addresses this problem. Note that there is a price to pay: the shape of Great Britain is barely recognizable. But the outsized influence of London is properly acknowledged.

  1guardian_brexitmap

 This one has two hues and four shades.  For me, it is most "truthful" because the sizes of the colored regions are properly mapped to the vote proportions.


Various ways of showing distributions 2

In a comment to the previous post, Evan pointed to this Washington Post graphic: (link to article)

Washingtonpost_ageolympians

This chart doesn't render properly in Firefox, nor in Safari. But we can see the designer's intention. It has an added dimension of gender.

This wasn't the chart that caught my eye before. The one I saw has the shades of blue that I used - I basically used the same design with a different set of data.

The chart above can be deconstructed in a similar fashion. It represents a set of collapsed histograms - two histograms, one for each gender, on each row. In other words:

Redo_olympic_ages_bygender

I don't think adding the gender variable adds much to the chart. (Note: the dataset I used did not have gender. I assigned gender randomly for illustrative purposes.)


Raining, data art, if it ain't broke

Via Twitter, reader Joe D. asked a few of us to comment on the SparkRadar graphic by WeatherSpark.

At the time of writing, the picture for Baltimore is very pretty:

Sparkradar

The picture for New York is not as pretty but still intriguing. We are having a bout of summer and hence the white space (no precipitation):

Sparkradar_newyork

Interpreting this innovative chart is a tough task - this is a given with any innovative chart. Explaining the chart requires all the text on this page.

The difficulty of interpreting the SparkRadar chart is twofold.

Firstly, the axes are unnatural. Time runs vertically, defying the horizontal convention. Also, "now" - the most recent time depicted - is at the very bottom, which tempts readers to read bottom to top, meaning we are reading time running backwards into the past. In most charts, time run left to right from past to present (at least in the left-right-centric part of the world that I live in.)

Location has been reduced to one dimension. The labels "Distance Inside" and "Distance from Storm" confuse me - perhaps those who follow weather more closely can justify the labels. Conventionally, location is shown in two dimensions.

The second difficulty is created by the inclusion of irrelevant data (aka noise). The square grid prescribes a fixed box inside which all data are depicted. In the New York graphic, something is going on in the top right corner - far away in both time and space - how does it help the reader?

***

Now, contrast this chart to the more standard one, a map showing rain "clouds" moving through space.

Bing_precipitationradar_baltimore

(From Bing search result)

The standard one wins because it matches our intuition better.

Location is shown in two dimensions.

Distance from the city is shown on the map as scaled distance.

Time is shown as motion.

Speed is shown as speed of the motion. (In SparkRadar, speed is shown by the slope of imaginary lines.)

Severity is shown by density and color.

Nonetheless, a panel of the new charts make great data art.