Modeler exceptionalism

Americans exceptionalism is act as we say but not as we do.

It turns out a subclass of modelers also claim exceptionalism.

I'm reacting to several news items from recent weeks. This past post is highly relevant. The post is titled "False belief in true models".

***

The first news item is the giddy response by Wall Street and economists to the latest jobs report in the U.S. Here is the typical news headline:

The consensus forecasts by economists was 2.9 million. If you put out a projection of 2.9 million and the number turns out to be 4.8 million, you'd think the forecasting model is horribly off. But not if you are talking to economists! Every time an economic indicator is reported, the news describes it as being over or below expectations. In other words, the model is always right.

Now, during the last few months, you'd be living under a rock if you have not heard at least one economist criticize Covid-19 projections as wrong. Here is a Vox headline that is typical:

You might be scratching your head. These researchers also build forecasts. When the numbers come out, they differ from the predictions. When the actual number of deaths is below the forecast, shouldn't we rejoice that public health policies have over-performed expectations? And if deaths come in above forecast, the virus is a terrible adversary. Except that the media now tell readers that the modelers are crap.

Some economists went so far as to question the intelligence of epidemiologists who build such models (link).

What do we call modelers who believe their own models to be always true and other people's models to be usually wrong? I like to call them exceptionalists.

 

 

Comments

[John Hall]

"Every time an economic indicator is reported, the news describes it as being over or below expectations. In other words, the model is always right."

You misunderstand. Participants in stock market value stocks based on the expected present value of future earnings. Forecasts of future earnings are then dependent on the expected future path of economic variables. If some economic indicator comes out significantly different from expectations (and it actually has a lot of information content, like non-farm payrolls), then that has an impact on stock prices. If the economic indicator comes in consistent with expectations, then it shouldn't have any impact on stock prices. In reality, it's not the model that's right, it's that the market prices need to be revised when the model is significantly wrong.

[a]

are you condoning criticisim which is good. I see many models actually are wrong.

[Kaiser]

JH: Your description seems to be a malfunctioning clock that is reset on the hour. During each hour, the clock always moves out of sync with reality, either running too fast or too slow. My observation in this post is the parallel between economic models and epi models, and how the business media are treating them differently when they both resemble malfunctioning clocks. Where is the article that explains why the jobs expectation models were so wrong?

a: All competent modelers accept that their models are imperfect, and I have written many posts here that point to the weaknesses of some of the epi models. Many of these models actually got the larger picture right. The discussion about modeling should be focused on whether the model assumptions are reasonable or not.