the wtf moment

You're reading some article that contains a standard chart. You're busy looking for the author's message on the chart. And then, the wtf moment strikes.

It's the moment when you discover that the chart designer has done something unexpected, something that changes how you should read the chart. It's when you learn that time is running right to left, for example. It's when you realize that negative numbers are displayed up top. It's when you notice that the columns are ordered by descending y-value despite time being on the x-axis.

Tell me about your best wtf moments!

***

The latest case of the wtf moment occurred to me when I was reading Rajiv Sethi's blog post on his theory that Kennedy voters crowded out Cheney voters in the 2024 Presidential election (link). Was the strategy to cosy up to Cheney and push out Kennedy wise?

In the post, Rajiv has included this chart from Pew:

The chart is actually about the public's confidence in scientists. Rajiv summarizes the message as: 'Public confidence in scientists has fallen sharply since the early days of the pandemic, especially among Republicans. There has also been a shift among Democrats, but of a slightly different kind—the proportion with “a great deal” of trust in scientists to act in our best interests rose during the first few months of the pandemic but has since fallen back.'

Pew produced a stacked column chart, with three levels for each demographic segment and month of the survey. The question about confidence in scientists admits three answers: a great deal, a fair amount, and not too much/None at all. [It's also possible that they offered 4 responses, with the bottom two collapsed as one level in the visual display.]

As I scan around the chart understanding the data, suddenly I realized that the three responses were not listed in the expected order. The top (light blue) section is the middling response of "a fair amount", while the middle (dark blue) section is the "a great deal" answer.

wtf?

***

Looking more closely, this stacked column chart has bells and whistles, indicating that the person who made it expended quite a bit of effort. Whether it's worthwhile effort, it's for us readers to decide.

By placing "a great deal" right above the horizon, the designer made it easier to see the trend in the proportion responding with "a great deal". It's also easy to read the trend of those picking the "negative" response because of how the columns are anchored. In effect, the designer is expressing the opinion that the middle group (which is also the most popular answer) is just background, and readers should not pay much attention to it.

The designer expects readers to care about one other trend, that of the "top 2 box" proportion. This is why sitting atop the columns are the data labels called "NET" which is the sum of those responding "a great deal" or "a fair amount".

***

For me, it's interesting to know whether the prior believers in science who lost faith in science went down one notch or two. Looking at the Republicans, the proportion of "a great deal" roughly went down by 10 percentage points while the proportion saying "Not too much/None at all" went up about 13%. Thus, the shift in the middle segment wasn't enough to explain all of the jump in negative sentiment; a good portion went from believer to skeptic during the pandemic.

As for Democrats, the proportion of believers also dropped by about 10 percentage points while the proportion saying "a fair amount" went up by almost 10 percent, accounting for most of the shift. The proportion of skeptics increased by about 2 percent.

So, for Democrats, I'm imagining a gentle slide in confidence that applies to the whole distribution while for Republicans, if someone loses confidence, it's likely straight to the bottom.

If I'm interested in the trends of all three responses, it's more effective to show the data in a panel like this:

***

Remember to leave a comment when you hit your wtf moment next time!

 

Dizziness

Statista uses side-by-side stacked column charts to show the size of different religious groups in the world:

It's hard to know where to look. It's so colorful and even the middle section is filled in whereas the typical such chart would only show guiding lines.

What's more, the chart includes gridlines, as well as axis labels.

The axis labels compete with the column section labels, the former being cumulative while the latter isn't.

The religious groups are arranged horizontally in two rows at the top while they are stacked up from bottom to top inside the columns.

The overall effect is dizzying.

***

The key question this chart purportedly address is the change in the importance of religions over the time frame depicted.

Look at the green sections in the middle of the chart, signifying "Unaffiliated" people. The change between the two time points is 16 vs 13 which is -3 percent.

Where is this -3 percent encoded?

It's in the difference in height between the two green blocks. On this design, that's a calculation readers have to do themselves.

One might take the slope of the guiding line that links the tops of the green blocks as indicative of the change, but it's not. In fact the top guiding line slopes upwards, implying an increase over time. That increase is associated with the cumulative total of the top three religious groups, not the share of the Unaffiliated group.

So, if we use those guiding lines, we have to take the difference of two lines, not just the top line. The line linking the bottoms of the green blocks is also relevant. However, the top and bottom lines will in general not be parallel, so readers have to somehow infer from the parallelogram bounded by the guiding lines and vertical block edges that the change in the Unaffiliated group is 3 percent.

Ouch.

***

I generally like to use Bumps charts (also called slopegraphs) to show change across two points in time:

What's sacrificed is the cumulation of percentages. I also am pleased that Christian and Muslim, where the movements are greatest, are found at the top of the chart. (There isn't a need to use so many colors; I just inherited them from the original chart.)

Election coverage prompts good graphics

The election broadcasts in the U.S. are full-day affairs, and they make a great showcase for interactive graphics.

The election setting is optimal as it demands clear graphics that are instantly digestible. Anything else would have left viewers confused or frustrated.

The analytical concepts conveyed by the talking heads during these broadcasts are quite sophisticated, and they did a wonderful job at it.

***

One such concept is the value of comparing statistics against a benchmark (or, even multiple benchmarks). This analytics tactic comes in handy in the 2024 election especially, because both leading candidates are in some sense incumbents. Kamala was part of the Biden ticket in 2020, while Trump competed in both 2016 and 2020 elections.

In the above screenshot, taken around 11 pm on election night, the MSNBC host (that looks like Steve K.) was searching for Kamala votes because it appeared that she was losing the state of Georgia. The question of the moment: were there enough votes left for her to close the gap?

In the graphic (first numeric column), we were seeing Kamala winning 65% of the votes, against Trump's 34%, in Douglas county in Georgia. At first sight, one would conclude that Kamala did spectacularly well here.

But, is 65% good enough? One can't answer this question without knowing past results. How did Biden-Harris do in the 2020 election when they won the presidency?

The host touched the interactive screen to reveal the second column of numbers, which allows viewers to directly compare the results. At the time of the screenshot, with 94% of the votes counted, Kamala was performing better in this county than they did in 2020 (65% vs 62%). This should help her narrow the gap.

If in 2020, they had also won 65% of the Douglas county votes, then, we should not expect the vote margin to shrink after counting the remaining 6% of votes. This is why the benchmark from 2020 is crucial. (Of course, there is still the possibility that the remaining votes were severely biased in Kamala's favor but that would not be enough, as I'll explain further below.)

All stations used this benchmark; some did not show the two columns side by side, making it harder to do the comparison.

Interesting side note: Douglas county has been rapidly shifting blue in the last two decades. The proportion of whites in the county dropped from 76% to 35% since 2000 (link).

***

Though Douglas county was encouraging for Kamala supporters, the vote gap in the state of Georgia at the time was over 130,000 in favor of Trump. The 6% in Douglas represented only about 4,500 votes (= 70,000*0.06/0.94). Even if she won all of them (extremely unlikely), it would be far from enough.

So, the host flipped to Fulton county, the most populous county in Georgia, and also a Democratic stronghold. This is where the battle should be decided.

Using the same format - an interactive version of a small-multiples arrangement, the host looked at the situation in Fulton. The encouraging sign was that 22% of the votes here had not yet been counted. Moreover, she captured 73% of those votes that had been tallied. This was 10 percentage points better than her performance in Douglas, Ga. So, we know that many more votes were coming in from Fulton, with the vast majority being Democratic.

But that wasn't the full story. We have to compare these statistics to our 2020 benchmark. This comparison revealed that she faced a tough road ahead. That's because Biden-Harris also won 73% of the Fulton votes in 2020. She might not earn additional votes here that could be used to close the state-wide gap.

If the 73% margin held to the end of the count, she would win 90,000 additional votes in Fulton but Trump would win 33,000, so that the state-wide gap should narrow by 57,000 votes. Let's round that up, and say Fulton halved Trump's lead in Georgia. But where else could she claw back the other half?

***

From this point, the analytics can follow one of two paths, which should lead to the same conclusion. The first path runs down the list of Georgia counties. The second path goes up a level to a state-wide analysis, similar to what was done in my post on the book blog (link).

Around this time, Georgia had counted 4.8 million votes, with another 12% outstanding. So, about 650,000 votes had not been assigned to any candidate. The margin was about 135,000 in Trump's favor, which amounted to 20% of the outstanding votes. But that was 20% on top of her base value of 48% share, meaning she had to claim 68% of all remaining votes. (If in the outstanding votes, she got the same share of 48% as in the already-counted, then she would lose the state with the same vote margin as currently seen, and would lose by even more absolute votes.)

The reason why the situation was more hopeless than it even sounded here is that the 48% base value came from the 2024 votes that had been counted; thus, for example, it included her better-than-benchmark performance in Douglas county. She would have to do even better to close the gap! In Fulton, which has the biggest potential, she was unable to push the vote share above the 2020 level.

That's why in my book blog (link), I suggested that the networks could have called Georgia (and several other swing states) earlier, if they used "numbersense" rather than mathematical impossibility as the criterion.

***

Before ending, let's praise the unsung heroes - the data analysts who worked behind the scenes to make these interactive graphics possible.

The graphics require data feeds, which cover a broad scope, from real-time vote tallies to total votes casted, both at the county level and the state level. While the focus is on the two leading candidates, any votes going to other candidates have to be tabulated, even if not displayed. The talking heads don't just want raw vote counts; in order to tell the story of the election, they need some understanding of how many votes are still to be counted, where they are coming from, what's the partisan lean on those votes, how likely is the result going to deviate from past elections, and so on.

All those computations must be automated, but manually checked. The graphics software has to be reliable; the hosts can touch any part of the map to reveal details, and it's not possible to predict all of the user interactions in advance.

Most importantly, things will go wrong unexpectedly during election night so many data analysts were on standby, scrambling to fix issues like breakage of some data feed from some county in some state.

Criminal graphics graphical crime

One of my Twitter followers disliked the following chart showing FBI crime statistics for 2023 (link):

If read quickly, the clear message of the chart is that something spiked on the right side of the curve.

But that isn't the message of the chart. The originator applied this caption: "The age-crime curve last year looked pretty typical. How about this year? Same as always. Victims and offenders still have highly similar, relatively young ages."

So the intended message is that the blue and the red lines are more or less the same.

***

What about the spike on the far right? 

If read too quickly, one might think that the oldest segment of Americans went on a killing spree last year. One must read the axis label to learn that elders weren't committing more homicides, but what spiked were murderers with "unknown" age.

A quick fix of this is to plot the unknowns as a column chart on the right, disconnecting it from the age distribution. Like this:

***

This spike in unknowns appears consequential: the count is over 2,000, larger than the numbers for most age groups.

Curiously, unknowns in age spiked only for offenders but not victims. So perhaps those are unsolved cases, for which the offender's age is unknown but the victim's age is known.

If that hypothesis is correct, then the same pattern will be seen year upon year. I checked this in the FBI database, and found that every year about 2,000 offenders have unknown age.

In other words, the unknowns cannot be the main story here. Instead of dominating our attention, it should be pushed to the background, e.g. in a footnote.

***

Next, because the amount of unknowns is so different between the offenders and victims, comparing two curves of counts is problematic. Such a comparison is based on the assumption that there are similar total numbers of offenders and victims. (There were in fact 5% more offenders than there were victims in 2023.)

The red and blue lines are not as similar as one might think.

Take the 40-49 age group. The blue value is 1,746 while the red value is 2,431, a difference of 685, which is 40 percent of 1,746! If we convert each to proportions, ignoring unknowns, the blue value is 12% compared to the red value of 15%, a difference of 3% which is a quarter of 12%.

By contrast, in the 10-19 age group, the blue value is 3,101 while the red value is 2,147, a difference of about 1,000, which is a third of 3,101. Converted to proportions, ignoring unknowns, the blue value is 21% compared to the red value of 13%, a difference of 8% which is almost 40% of 21%.

It's really hard to argue that these age distributions are "similar".

As seen from the above, offenders are much more likely to be younger (10-29 years old) than victims, and they are also much more likely to be 90+! Meanwhile, the victims are more likely to be 60-89.