There are two types of people when it comes to using data. I call them "data first" and "story first". The natural mode seems to be "story first". Calling this "narrative fallacy" or "confirmation bias" creates the impression that the default mode is "data first", and being "story first" is an aberration.
I believe it's the opposite. Being "data first" requires training, and conscious effort. This viewpoint may be related to Dan Kahneman's classification of System 1 and 2 thinking. Story first is quick, System 1 while data first is slow, System 2.
***
The recent reporting of first-time unemployment claims data by Marketwatch captures "story first" thinking. The reporter's story is the federal unemployment compensation (the $600 payout) hurts the economy.
On August 13, Marketwatch published an article with the following graphic:
The headline is the story: "when you pay people more to sit at home... they sit at home." What is the evidence (data)? The reporter said "a big drop in people seeking or receiving benefits in the past two weeks after the end of a $600 federal stipend hints the answer might be yes".
So he's pointing to the last two columns, and saying that those two columns are below 1 million while the previous columns exceed 1 million. Simply put, (a) from last week of July to first week of August, federal unemployment payout ended (b) from last week of July to first two weeks of August, initial jobless claims went down, therefore, (a) must be the cause of (b).
In order to sharpen part (a) of the argument, he established a bar of 1 million weekly claims as some kind of magical level separating good from bad. Above 1 million, bad; below 1 million, good. Prior to August, above 1 million; after August, below 1 million.
The reporter proceeds to quote multiple economists all claiming that this conclusion is a slam dunk, "not rocket science", "hear it all the time from business people", etc.
Our System 1 thinking might accept such a logic but given time to consume the data, it's easy to note the fatal flaw in the argument.
The federal benefit lasted from April through end of July (reference). For every single week during this period, the initial jobless claims number has dropped. In fact, the weekly drops were a lot bigger in April than in August. So we have months of counterexamples on the same chart that is used to prove his case.
Besides, the number of initial jobless claims in the second week of August is still five times higher than the level seen in January and February before the pandemic went out of control.
***
Fast forward one week to August 20. The same reporter reported in MarketWatch on initial jobless claims, and updated his chart:
The general trend of this chart mirrors the first chart. The number of initial jobless claims has declined from the peak in early April.
Take a closer look at this chart versus the previous chart. You will notice they are not the same thing. Here are the alterations:
- The first chart shows seasonally adjusted data while the newer chart shows unadjusted data. The footnote even discloses that the seasonally adjusted data are higher (i.e. worse) than the unadjusted data
- The newer chart added a second category of jobless claims (federal)
Despite the data changing, the story hasn't changed.
In the first paragraph, he writes "Initial weekly jobless benefit claims rose in mid-August and topped 1 million again, potentially pointing to an increase in layoffs after a summer surge in the coronavirus epidemic or perhaps to more people applying for benefits after President Trump temporarily added $300 in extra federal payouts."
So, the story remains the same. The previous week, removing the $600 caused the drop in jobless claims. This week, the "addition" of $300 caused the increase in jobless claims.
The President did not "add extra federal payouts". The $300 payout will be half the previous amount. Further, almost no one received a $300 check during the third week of August.
Once again, the contradictory evidence is found on the same graphic. Back in April, when the $600 checks started getting paid, the new jobless claims number was decreasing, not rising.
Now, let's talk about the 1 million claims threshold that appeared in the first chart. That 1 mllion level was established for the state-level new jobless claims, which is the lower part of the columns in the newer chart. If you look at just the lower parts, all three numbers for August sit below 1 million. So nothing "topped 1 million again." Magically, the 1 million threshold is now applied to the state-plus-federal new jobless claims data. Why? Because it fits the story.
Except... all three numbers for August sit above 1 million.
***
The economists cited in the August 20 article no longer think the link between federal payout and initial jobless claims is a slam dunk. The quotes are a mixed bag.
But the story isn't wrong. What is wrong? It's the data, stupid. "MarketWatch is reporting select jobless claims data using unadjusted figures to give a clearer picture of unemployment. The seasonally adjusted estimates typically expected by Wall Street have become distorted by the pandemic and appear to overstate new claims at times."
MarketWatch presents zero evidence that the seasonal adjustment is incorrect, except their belief that any adjustment that increases the unadjusted figures is wrong.
According to the journalist, "the increase in new claims wasn't as bad as it seemed... based on actual or unadjusted figures. They rose a much smaller 52,776 to 891,510 and remained below 1 million for the third straight week." [As noted, he uses 1 million regardless of whether the data include federal claims.]
This argument is entirely backwards. If the unadjusted change is smaller than the adjusted change, that means the current situation is worse than normal. Hard to argue with that.
The point of seasonal adjustment is to establish the expected weekly fluctuations in the data during normal times. (I wrote a tutorial on seasonal adjustment years ago, and also in Chapter 6 of Numbersense (link), on employment statistics.)
***
What do we think should happen the following week? Since more states will be paying the "extra $300", the data better behave! We want to see the initial jobless claims at the state level (unadjusted) to rise again since such payout disincentivizes work.
Drum roll.
On August 27, MarketWatch pushed the graphic to the bottom of the page.
The first sentence of the report reads: "New applications for jobless benefits fell again in late August to just above 1 million and resumed a downward trend, perhaps signaling the resumption of a gradual if painfully slow recovery in the U.S. labor market."
The theory says when more people are receiving the $300 checks, they should be staying home pushing up the jobless claims. (One problem all along is the focus on initial jobless claims rather than total jobless claims or continuing jobless claims.) Don't worry: the journalist is holding out hope for next week's data. He said: "most states still aren't offering the money and the extra cash is just starting to be sent out." The week before, he suggested that the $300 checks caused the rise in claims.
The cited statistic in the first paragraph is the seasonally-adjusted, state-level number (the lower sections of those columns) so the reporter reverted to the metric from two weeks before. He goes on to explain that the data problem is limited to that one week which broke the trend. "New claims rose sharply last week.... but most economists suspect problems with the seasonal adjustments or other temporary factors."
***
Story-first thinking frequently leads to cherry-picking specific conforming data, ignoring contradictory data, explaining away or pouring doubt on non-conforming data, branching the story to accommodate non-conformng data, and creating episodes that work individually but do not cohere.
***
There is another jobless claims report out today but I'll save this for a different post because the Bureau of Labor Statistics suddenly changed the methodology used for seasonal adjustment, which at first glance, will make every single seasonally-adjusted number between now and November look better.
Comments