Is the chart answering your question? Excavating the excremental growth map

Economist_excrement_growthSan Franciscans are fed up with excremental growth. Understandably.

Here is how the Economist sees it - geographically speaking.

***

In the Trifecta Checkup analysis, one of the questions to ask is "What does the visual say?" and with respect to the question being asked.

The question is how much has the problem of human waste in SF grew from 2011 to 2017.

What does the visual say?

The number of complaints about human waste has increased from 2011 to 2014 to 2017.

The areas where there are complaints about human waste expanded.

The worst areas are around downtown, and that has not changed during this period of time.

***

Now, what does the visual not say?

Let's make a list:

  • How many complaints are there in total in any year?
  • How many complaints are there in each neighborhood in any year?
  • What's the growth rate in number of complaints, absolute or relative?
  • What proportion of complaints are found in the worst neighborhoods?
  • What proportion of the area is covered by the green dots on each map?
  • What's the growth in terms of proportion of areas covered by the green dots?
  • Does the density of green dots reflect density of human waste or density of human beings?
  • Does no green dot indicate no complaints or below the threshold of the color scale?

There's more:

  • Is the growth in complaints a result of more reporting or more human waste?
  • Is each complainant unique? Or do some people complain multiple times?
  • Does each piece of human waste lead to one and only one complaint? In other words, what is the relationship between the count of complaints and the count of human waste?
  • Is it easy to distinguish between human waste and animal waste?

And more:

  • Are all complaints about human waste valid? Does anyone verify complaints?
  • Are the plotted locations describing where the human waste is or where the complaint was made?
  • Can all complaints be treated identically as a count of one?
  • What is the per-capita rate of complaints?

In other words, the set of maps provides almost all no information about the excrement problem in San Francisco.

After you finish working, go back and ask what the visual is saying about the question you're trying to address!

 

As a reference, I found this map of the population density in San Francisco (link):

SFO_Population_Density

 


The state of the art of interactive graphics

Scott Klein's team at Propublica published a worthy news application, called "Hell and High Water" (link) I took some time taking in the experience. It's a project that needs room to breathe.

The setting is Houston Texas, and the subject is what happens when the next big hurricane hits the region. The reference point was Hurricane Ike and Galveston in 2008.

This image shows the depth of flooding at the height of the disaster in 2008.

Propublica_galveston1

The app takes readers through multiple scenarios. This next image depicts what would happen (according to simulations) if something similar to Ike plus 15 percent stronger winds hits Galveston.

Propublica_galveston2plus

One can also speculate about what might happen if the so-called "Mid Bay" solution is implemented:

Propublica_midbay_sol

This solution is estimated to cost about $3 billion.

***

I am drawn to this project because the designers liberally use some things I praised in my summer talk at the Data Meets Viz conference in Germany.

Here is an example of hover-overs used to annotate text. (My mouse is on the words "Nassau Bay" at the bottom of the paragraph. Much of the Bay would be submerged at the height of this scenario.)

Propublica_nassaubay2

The design has a keen awareness of foreground/background issues. The map uses sparse static labels, indicating the most important landmarks. All other labels are hidden unless the reader hovers over specific words in the text.

I think plotting population density would have been more impactful. With the current set of labels, the perspective is focused on business and institutional impact. I think there is a missed opportunity to highlight the human impact. This can be achieved by coding population density into the map colors. I believe the colors on the map currently represent terrain.

***

This is a successful interactive project. The technical feats are impressive (read more about them here). A lot of research went into the articles; huge amounts of details are included in the maps. A narrative flow was carefully constructed, and the linkage between the text and the graphics is among the best I've seen.


Scorched by the heat in Arizona

Reader Jeffrey S. saw this graphic inside a Dec 2 tweet from the National Weather Service (NWS) in Phoenix, Arizona.

Nwsphoenix_bars

In a Trifecta checkup (link), I'd classify this as Type QV.

The problems with the visual design are numerous and legendary. The column chart where the heights of the columns are not proportional to the data. The unnecessary 3D effect. The lack of self-sufficiency (link). The distracting gridlines. The confusion of year labels that do not increment from left to right.

The more hidden but more serious issue with this chart is the framing of the question. The main message of the original chart is that the last two years have been the hottest two years in a long time. But it is difficult for readers to know if the differences of less than one degree from the first to the last column are meaningful since we are not shown the variability of the time series.

The green line makes an assertion that 1981 to 2010 represents the "normal". It is unclear why that period is normal and the years from 2011-5 are abnormal. Maybe they are using the word normal in a purely technical way to mean "average." If true, it is better to just say average.

***
For this data, I prefer to see the entire time series from 1981 to 2015, which allows readers to judge the variability as well as the trending of the average temperatures. In the following chart, I also label the five years with the highest average temperatures.

Redo_nws_phoenix_avgtemp_2


Finding meaning in Big Blue California

Via Twitter, Pat complained that this Bloomberg graphic is confusing:

Bloomberg_electriccars

The accompanying article is here. The gist of the report is that electric cars are much more popular on the West coast because the fuel efficiency of such cars goes down dramatically in colder climates. (Well, there are political reasons too, also discussed in the article.)

What makes this chart confusing?

Our eyes are drawn to big blue California, and the big number 25,295. The blue block raises three questions: first, how do we interpret that 25,295 number? How big is it? To what should we compare the number? Second, we notice a blending of labels--California is the only label of a state while all other labels are of regions. Third, the number under West is 31,783, even larger than 25,295 although it gets a smaller font size, a black-and-white treatment, and a seemingly small allocation of space.

It takes a little time to figure out the structure of the graphic. That the baseline is a treemap with the regions, and big blue California is a highlight that sits within the West region.

Tufte would not love the "moivremoire"  patterns, nor do I. I'd have left the background of the entire right side plain white.

I fail to see why this treemap form is preferred to a simple bar chart.

***

As I play around with the data, basically playing with stacking the data, I found a way to make a more engaging graphic. This new graphic builds off an insight from this data: that the number of electric cars sold in California is more than all other states combined. So here you go:

Redo_bloomberg_electriccars

Since the article attributes the gap in sales to regional temperature, an even better illustration should bring in temperature data.

 


Rethinking the index data, with modesty and clarity in mind

I discussed the rose chart used in the Environmental Performance Index (EPI) report last week. This type of data is always challenging to visualize.

One should start with an objective. If the goal is a data dump, that is to say, all you want is to deliver the raw data in its full glory to the user, then you should just print a set of data tables. This has traditionally been the delivery mechanism of choice.

If, on the other hand, your interest is communicating insights, then you need to ask some interesting questions. One such question is how do different regions and/or countries compare with each other, not just in the overall index but also in the major sub-indices?

Learning to ask such a question requires first understanding the structure of the data. As described in the previous post, the EPI is a weighted average of a bunch of sub-indices. Each sub-index measures "distance to a target," which is then converted into a scale from 0 to 100. This formula guarantees that at the aggregate level, the EPI is not going to be 0 or 100: a country would have to score 100 on all sub-indices to attain EPI perfection!

Here is a design sketch to address the question posed above:

Redo_epi_regional

For a print version, I chose several reference countries listed at the bottom that span the range of common values. In the final product, hovering over a stripe should disclose a country and its EPI. Then the reader can construct comparisons of the type: "Thailand has a value of 53, which places it between Brazil and China."

The chart reveals a number of insights. Each region stakes out its territory within the EPI scale. There are no European countries with EPI lower than 45 while there are no South Asian countries with EPI higher than 50 or so. Within each region, the distribution is very wide, and particularly so in the East Asia and Pacific region. Europe is clearly the leading region, followed by North America.

The same format can be replicated for every sub-index.

This type of graph addresses a subset of the set of all possible questions and it does so in a clear way. Modesty in your goals often helps.

 


A not-so-satisfying rose

At the conference in Bavaria, Jay Emerson asked participants to provide comments on the data visualization of the 2014 Environmental Performance Index (link). We looked at the country profiles in particular. Here is one for Singapore:

Singapore

The main object of interest here is the "rose chart." To understand it, we need to know the methodology behind the index. The index is a weighted average of nine sub-indices, as shown in the table at the bottom. In many cases, the sub-index is itself an average of sub-sub-indices. These lower-level indices measure the distance between a country's performance and some target performance, typically set at the international level. But those distances are converted into a scale between 0 and 100 so the country with a score of zero did the worst in terms of meeting the target while the country with 100 did the best.

In the rose chart, the circle is divided evenly into nine sectors, each representing a sub-index. The data are encoded in the radius of the sectors. Colors map to the sub-index, and the legend is provided in two ways: a hover-over on the Web, and the table below.

Here is the equation that connects the data (EPI) to the area of the sectors:

Rose_formula

There are a number of issues with this representation. First, because of the squaring of the EPI, the area is distorted. If one country is twice the EPI of another, the area is four times as large. Another way to see this is to notice that as the EPI increases, the curved edge of the sector moves outwards, tracing a larger circumference.

Another issue is the one-ninth factor, which implies that each of those nine sub-indices are equally important. The diagram below shows that interpretation to be incorrect. (The nine sub-indices are shown in the second layer from the outside in.)

Epi_index_components

 A third issue is illustrated in the Singapore rose. Notice from the table below that Singapore scored zero on Fisheries. But in the rose, Fisheries has a non-zero area. Think of this practice as coring an apple. The middle circle of radius k should be ignored. If the sector that has the color of Fisheries has zero area, then the entire red circle shown below should have zero area.

Rose-core

With these three adjustments, the encoding formula becomes rather more complicated:

Rose_formula2

where x depends on the weight of the sub-index, and k is the radius of the sector that represents value zero.

***
The rose/radar/spider type charts are more useful when placed side by side to compare countries. But even then, this chart form doesn't work well for this dataset. This is because the spacing of countries within each sub-index is not uniform.

 The site has a visualization of the distribution of sub-index scores by issue:

Epi_by_issue

We can see that in cases of water resources, most countries are not doing very well at all. In terms of air quality, most countries except for those in the right tail have performed quite well. It is hard to interpret the indices unless one has an idea of the full distribution.

***

Finally, one wrinkle that the EPI people did makes me happy. They have created PDF and images of their data visualization so it is quite easy to save and keep some of this work. All too often, browser-based technologies create visualization that can't be saved.


Designers fuss over little details and so should you

Those who attended my dataviz talks have seen a version of the following chart that showed up yesterday on New York Times (link):

Arctic_sea_ice

This chart shows the fluctuation in Arctic sea ice volume over time.

The dataset is a simple time series but contains a bit of complexity. There are several ways to display this data that helps readers understand the complex structure. This particular chart should be read at two levels: there is a seasonal pattern that is illustrated by the dotted curve, and then there are annual fluctuations around that average seasonal pattern. Each year's curve is off from the average in one way or another.

The 2015 line (black) is hugging the bottom of the envelope of curves, which means the ice volume is at a historic low.

Meanwhile the lines for 2010-2014 (blue) all trace near the bottom of the historic collection of curves.

***

There are several nice touches on this graphic, such as the ample annotation describing interesting features of the data, the smart use of foreground/background to make comparisons, and the use of countries and states (note the vertical axis labels) to bring alive the measure of coverage volume.

Check out my previous post about this data set.

Also, this post talks about finding real-life anchors to help readers judge size data.

My collection of posts about New York Times graphics.

 

PS. As Mike S. pointed out to me on Twitter, the measure is "ice cover", not ice volume so I edited the wording above. The language here is tricky because we don't usually talk about the "cover" of a country or state so I am using "coverage". The term "surface area" also makes more sense for describing ice than a country.


Sheep tramples sense

Merry Christmas, readers.

***

A Twitter follower pointed me to this visual:

Wtf_sheep

I have yet to understand why the vertical axis of the top chart keeps changing scales over time. The white dot labelled "Peak 1982" (70 million) is barely above the other white dot for "2007" (38 million). This chart hides a clear trend: the population of sheep in New Zealand has plunged by 45% over 25 years.

To address the question of sheep versus human, one should plot the ratio of sheep-to-human directly. In this case, the designer probably faced a problem: because of the plunging population of sheep, the ratio has plunged steeply in 25 years. To make a point that "people are outnumbered more than 9 to 1", the designer didn't want to show a plunging trend. (Could this be the reason why the human population in 1982 was not printed?)

This is a case of too many details. Instead of manipulating the scale to distort the data, one can simply show the current ratio, or the average ratio in the last five years.

***

As the reader scans to the bottom set of charts, a cognitive wedge is encountered, as the curved scale of the New Zealand chart gave way to the normal uniform scale. These smaller charts are no less confusing, however.

  Australia_iceland_sheep

The two lines on these two charts appear almost the same and yet, the Australian chart (on the left) shows a ratio of 4 to 1 while the Icelandic chart (on the right) shows a ratio of 1.5 times. Makes you wonder if each one of the small-multiples have a dual axis.

Again, I'm not convivned that the time series adds anything to the message.

 


Cloudy and red

Note: I'm traveling during the holidays so updates will be infrequent.

 

Reader Daniel L. pointed me to a blog post discussing the following weather map:

Vane_temp_anomaly

The author claimed that many readers misinterpreted the red color as meaning high temperatures when he intended to show higher-than-normal temperatures. In other words, the readers did not recognize a relative scale is in play.

That is a minor issue that can be fixed by placing a label on the map.

There are several more irritants, starting with the abundance of what Ed Tufte calls chartjunk. The county boundaries do not serve a purpose, nor is it necessary to place so many place names. State boundaries too are  too imposing. The legend fails to explain what the patch of green in Florida means.

The article itself links to a different view of this data on a newly launched site called Climate Prediction Center, by the National Oceanic and Atmospheric Administration (link). Here is a screenshot of the continental U.S.

Cpc_temp_anomaly

This chart is the other extreme, bordering on too simple.

I'd suggest adding a little bit of interactivity to this chart, such as:

  • Hiding the state boundaries and showing them on hover only
  • Selectively print the names of major cities to help readers orient themselves
  • Selectively print the names of larger cities around the color boundaries
  • Using a different background map that focuses on the U.S. rather than the entire North American continent 

This is a Type V chart.