Big Data, Plainly Spoken (aka Numbers Rule Your World)

The Swedish mirage: the verdict is already written

What's happening in Sweden?

As shown in the above chart from OurWorldinData, a second wave has not spared Sweden, the country that is famous for pursuing "herd immunity through community spread."

Despite the complete lack of confirming data, the idea that Sweden's strategy may work refuses to die. The talk is that "in time" we will see the wisdom of the Swedish way.

But we don't need more time. The strategy has already failed. There is almost nothing that could happen in the future that could rescue it.

***

Look at the trend of Covid-19 deaths in Sweden compared to its Scandinavian neighbors:

(This chart was adapted from one first published by ArsTechnica, which I discussed on Junk Charts last week.)

The death rate per capita in Sweden from the start of the pandemic is 10 times worse than in Norway and Finland, and 5 times worse than in Denmark.

If the "herd community through community spread" strategy works, the Swedish outcome must improve over time relative to Norway and Finland. But as the chart above shows, counting only the last four months, we find Sweden doing even worse - the fatalities per capita is now more than 10 times higher than all three neighbors.

***

In spite of those numbers, some commentators say we've got to wait. What might happen in the future that would rescue this policy?

A devastating second wave would have to hit Norway and Finland while sparing Sweden, inflicting deaths per capita 10 times higher than in Sweden. The data from OurWorldinData tell us this isn't happening at all:

Finland and Norway have kept infections down while Sweden has not. Denmark has also suffered a spike but one that is not as bad as Sweden's.

***

Let's suspend our disbelief, and assume that somehow in the next six months, Finland and Norway would suffer 10 times the deaths per capita as Sweden, thus evening out the cumulative outcome. Does that constitute a validation of the "herd immunity" strategy?

Absolutely not!

All that show is that Sweden's policy accelerated the deaths of a lot of people. A proportionate number of Finns and Norwegians eventually also died but they lived longer, and during those extra months, there was a chance that an effective treatment or vaccine might arrive and save them.

***

When people say they need more time to evaluate the Swedish way, what they are waiting for is the extremely unlikely scenario that Finland and Norway would suffer substantially more than 10 times more deaths than Sweden in the next six months. There is zero indication that this "miracle" will materialize. Not only are recent fatality rates moving in the wrong direction but the case rates are also running away from this scenario.

I'm imagining a football (soccer) match in which one team falls behind 10-0 in the first 10 minutes, and the other team's coach is asking supporters to be patient and wait for the comeback.

Netflix, the data pioneer, gets flagged for data offence

The headline says it all. The Hollywood Reporter reported that "'The Witcher' Breaks Netflix Series Viewership Records -- with One Big Catch."

This should come as no surprise to my readers. It proves yet again you can use data to prove anything you want. Apparently, Netflix switched how it counts "viewers" in 2020.

Those not knee-deep in data work might not appreicate how measuring viewership can be so controversial. How hard is it to count how many people viewed a show?

In this case, Netflix announced 76 million households watched the series during the first four weeks of release. This seemingly simple statement includes a load of unspoken assumptions:

• Does each houshold have to watch all four weeks or one out of four weeks? (Nothing prevents one from requiring each household to watch two or three out of four weeks, either!)
  
  
• How much of each show must be watched in order to count as a "view"? (Hint: if the show is 60 minutes, absolutely no one will count 60 minutes. If you want to guess, select a number closer to 0 than to 60. Correct answer is disclosed below.)
  
  
• How many episodes must be be "viewed" to consider the series "viewed"? (The Witcher Season 1 has eight episodes, all released at the same time. I don't think you need a hint for this one.)
  
  
• Should each device and each sub-account be aggregated or not?

The above list isn't even comprehensive; there are even more subtle issues:

• What is being measured is not "viewing" but "streaming". The video player sends back regular signals to Netflix saying the player is still streaming a particular show. To measure actual viewing, the TV would have to track the viewers' eyes. (This is technically feasible although I think these companies aren't doing this.)
  
  
• Do repeat minutes count? If the rule is 10 minutes of at least two episodes, how do we deal with someone who watched 10 minutes of the same episode, twice?
  
  
• How to deal with fast forward, skipping, etc.?

***

As you look at those questions, think about whether there is a principled way to make these decisions. How does one argue that one out of four weeks is not a view but two out of four is? How does one argue that 10 minutes constitute viewing but 5 minutes don't?

It's pretty clear why you can use data to prove anything you want.

Still, what Netflix did is jaw-dropping. The previous definition of a view is "watching at least 70 percent of one episode of a series or 70 percent of a feature film". The new definition of a view - watching at least 2 minutes of one episode. You are reading this right; the 76 million households have "viewed" one Watcher episode for at least 2 minutes.

The first definition is already quite generous. The rule of the game is to get enough people to turn on the first episode, and it doesn't matter that they never return. Since Netflix has the famous recommendation engine, it's got the tool to drive traffic to the first episode of pretty much anything. The audience have to stick around for 70 percent of the running time, though.

How does Netflix justify dropping the bar from 70 percent of one episode to 2 minutes?

From the Hollywood Reporter:

Our new methodology is similar to the BBC iPlayer in their rankings based on 'requests' for the title, 'most popular' articles on The New York Times, which include those who opened the articles, and YouTube view counts," the company notes in its earnings report. "This way, short and long titles are treated equally, leveling the playing field for all types of our content including interactive content, which has no fixed length."

Netflix is right about Youtube. Youtube counts 30 seconds of run time as "viewing". Also, Youtube is autoplay by default. Once you start playing one video, it performs a version of "doom scrolling" so it keeps playing the next clip. I'm surprised people actually suggest that such "autoplay" views are not counted as views. Doing so would disqualify a ton of legitimate viewing as not views - and Youtube makes money from generating views!

The most objectionable part of the quote is "short and long titles are treated equally". According to Netflix, treating someone who watched 2 minutes of a 10-minute clip the same as someone who watched 2 minutes of a 240-minute movie demonstrates fairness.

You know it's a problem when the agreeable press not only noticed the data offense but also decided to stick it in the headline rather than burying it two paragraphs from the end of the article.

***

The issue of definitions is everywhere in data analysis. Covid19 tests and deaths. Flight delays. Mean tweets. Twitter users. Facebook video view time. Wind chill.

In Chapter 2 of Numbersense (link), I featured some doctors who believed that changing the definition of obesity will solve the obesity problem in the U.S.

P.S. [10/20/2020:

Mark Palko alerted me to his take on this news also. I think what he's saying is Netflix knows (of course) how much time each device "viewed" each minute of each episode, it just chooses not to publicize that data. Great point!

Here is another post by Mark that covers how Netflix uses "deceptive messaging and suspicious statistics". 

]

I don't like the term herd immunity

(Image credit: Sven van Echelpoel/Flickr)

"Herd immunity" keeps popping up as a public health policy even though most epidemiologists dismiss it as an unwise prescription. I previously wrote about herd immunity in a post titled "The shift to antibody testing is a choice of death over life" which highlights the high cost in human lives of such a policy pursued through community spread.

Let's review a few other implications of a "herd immunity" policy. There are two proposed paths to reaching herd immunity: through vaccination or through community spread.

Fairness

Herd immunity through natural infections is inherently unfair. The consequent deaths are unevenly distributed across the population. I'm not just talking about the underlying characteristics of the illness, which attacks certain demographics, such as the elderly, while apparently sparing other subgroups, such as the very young.

I'm referring to the differential access and quality of healthcare - which is a major contributor to differential mortality in minorities and lower-income communities. Moreover, I'm pointing to the heightened exposure to the virus due to economic conditions and the nature of one's work. Essential workers, workers who must serve others in person, and people who cannot afford temporary unemployment bear the burden of a "herd immunity" policy. There hasn't been much discussion about how these subgroups should be compensated for the "sacrifice" society demands of them, if the government chooses such a policy.

Fatalities

The only proven way to achieve herd immunity is by broad-based vaccination. This is also the safe path since a 50% effective vaccine cuts the death rate by half relative to achieving herd immunity through community spread. (It is also not clear how much immunity one gets from an infection.)

Attempting to reach "herd immunity" via natural infection not only doubles the number of fatalities (relative to vaccination) but also inflicts disproportionate harm on disadvantaged subgroups.

Rewarding the wrong behavior

Herd immunity via vaccination is widely held as desirable, even though we are praising the wrong entity. What we desire are the vaccines that protect us from the virus. Herd immunity is actually not an indicator of success but of failure. Let me explain.

If we have an effective and safe vaccine, everyone should take it. In practice, when 75 percent of the population have been vaccinated, we have defeated the virus, in the sense that the community spread will have been curtailed, whether or not the remaining 25 percent get the vaccine. (The proportion that needs to get vaccinated depends on how fast infections are occurring; I'm using 75 percent for illustration.) It's the 25 percent who benefit from "herd immunity" while the other 75 percent are protected by the vaccine. The 25 percent are the bystanders who get windfall from the actions of the other 75 percent.

A society does not have to, and probably should not, stop at 75 percent. If everyone gets vaccinated, we have comprehensive immunity, rendering herd immunity immaterial. That's what I mean when I say herd immunity is a sign of failure.

Herd immunity rewards non-cooperative behavior.

Tying your health to others' actions

If you are considering being a bystander, think twice. The more people who intend to sit out, the higher the chance that the society will not reach the 75 percent threshold to get herd immunity. In that world, the death rate of unvaccinated bystanders will be double that of the vaccinated (or worse, if the vaccine is better than 50% effective).

By sitting out, you have tied your own health to the actions of other people. Worse, you have tied your health to the actions of people who hold opposite views as you.

Immunity in theory

If you are still considering holding out, think again. Perhaps no one has told you that herd immunity is an abstract mathematical construct. The bystanders do not have immunity in the biological sense of the word.

If they come in contact with the virus, they may get sick, and even die from Covid-19. The mathematical theory predicts that once 75 percent of the population is vaccinated, the odds of an unvaccinated person contacting an infectious person is small enough that the epidemic will not grow further. Such implied mathematical immunity is different from getting immunity from a vaccine that stimulates our bodies to manufacture antibodies that fight the coronavirus.

The unfortunate wording of "herd immunity" fosters the misunderstanding that it gives individuals "immunity" from getting sick.

***

tl;dr

Herd immunity draws attention to non-cooperative behavior away from the scientific achievements of vaccines. The people who stand to benefit from herd immunity don't even have immunity, as they will get sick if they come in contact with the virus. That's why I don't like the term herd immunity.

If people would buy the real thing, there would be no counterfeits

In Chapter 3 of Numbersense (link), I told the story of the software industry complaining that pirates stole billions worth of software each year. This estimate of the value of software embeds the fantasy that people who use pirated software would have purchased legal software at list price should piracy be eradicated. It's a failure to think counterfactually. Many people who use pirated software cannot afford, or do not want, to pay for the legal version.

The tech industry invokes this logical flaw frequently. Earlier this week [a month ago by the time this post is published], news broke that the U.S. Customs and Borders Protection (CBP) intercepted "counterfeit" Apple Airpods but quickly, tech-savvy observers pointed out that those earbuds are made by OnePlus for Android phones, an Apple competitor. The photos included in the CBP press release even showed the seized products in OnePlus packaging.

CBP estimated that the seized products had a value of $389,000, by multiplying the number of products with the list price of Apple Airpods. Such an estimate assumes that if OnePlus earbuds were banned, consumers would have all purchased the much more expensive Apple product. The list price for OnePlus earbuds is less than half that of Airpods.

In this ArsTechnica article, the CBP claims $1.5 billion worth of counterfeit products are seized in a year. This number appears to suffer from the same counterfactual fallacy. What would consumers do if OnePlus earbuds were banned? A proportion of them will go back to wired headphones. Another group will migrate to other earbuds that are in a similar price range as OnePlus. These are not merely speculation - we do know that these customers elected to buy OnePlus instead of paying double for Apple.

If everyone who purchased the counterfeits would buy the real thing at list price, why didn't they buy the real thing in the first place?