Wednesday , November 6 2024
Home / Real-World Economics Review / Coronavirus and the “replication crisis” in social science

Coronavirus and the “replication crisis” in social science

Summary:
From an article by Nassim Nicholas Taleb and Yaneer Bar-Yam in yesterday’s Guardian Dominic Cummings [Chief Adviser to UK Prime Minister Boris Johnson] loves to theorise about complexity, but he’s getting it all wrong. . . . The error in the UK is on two levels. Modelling and policymaking. First, at the modelling level, the government relied at all stages on epidemiological models that were designed to show us roughly what happens when a preselected set of actions are made, and not what we should make happen, and how. The modellers use hypotheses/assumptions, which they then feed into models, and use to draw conclusions and make policy recommendations. Critically, they do not produce an error rate. What if these assumptions are wrong? Have they been tested? The answer is often no. For

Topics:
Editor considers the following as important:

This could be interesting, too:

Merijn T. Knibbe writes ´Fryslan boppe´. An in-depth inspirational analysis of work rewarded with the 2024 Riksbank prize in economic sciences.

Peter Radford writes AJR, Nobel, and prompt engineering

Lars Pålsson Syll writes Central bank independence — a convenient illusion

Eric Kramer writes What if Trump wins?

from an article by Nassim Nicholas Taleb and Yaneer Bar-Yam in yesterday’s Guardian

Dominic Cummings [Chief Adviser to UK Prime Minister Boris Johnson] loves to
theorise about complexity, but he’s getting it all wrong.

. . . The error in the UK is on two levels. Modelling and policymaking.

First, at the modelling level, the government relied at all stages on epidemiological models that were designed to show us roughly what happens when a preselected set of actions are made, and not what we should make happen, and how.

The modellers use hypotheses/assumptions, which they then feed into models, and use to draw conclusions and make policy recommendations. Critically, they do not produce an error rate. What if these assumptions are wrong? Have they been tested? The answer is often no. For academic papers, this is fine. Flawed theories can provoke discussion. Risk management – like wisdom – requires robustness in models.

But if we base our pandemic response plans on flawed academic models, people die. And they will.

This was the case with the disastrous “herd immunity” thesis. The idea behind herd immunity was that the outbreak would stop if enough people got sick and gained immunity. Once a critical mass of young people gained immunity, so the epidemiological modellers told us, vulnerable populations (old and sick people) would be protected. Of course, this idea was nothing more than a dressed-up version of the “just do nothing” approach. ……

Second, but more grave, is the policymaking. No 10 appears to be enamoured with “scientism” – things that have the cosmetic attributes of science but without its rigour. This manifests itself in the nudge group that engages in experimenting with UK citizens or applying methods from behavioural economics that fail to work outside the university – yet patronise citizens as an insult to their ancestral wisdom and risk-perception apparatus. Social science is in a “replication crisis”, where less than half the results replicate (under exact same conditions), less than a tenth can be taken seriously, and less than a hundredth translate into the real world.

So what is called “evidence-based” methods have a dire track record and are pretty much evidence-free. This scientism also manifests itself in Boris Johnson’s chief adviser Dominic Cummings’s love of complexity and complex systems (our speciality) which he appears to apply incorrectly. And letting a segment of the population die for the sake of the economy is a false dichotomy – aside from the moral repugnance of the idea.

Nassim Nicholas Taleb is distinguished professor of risk engineering at New York University’s Tandon School of Engineering and author of The Black Swan. Yaneer Bar-Yam is president of the New England Complex System Institute.

Leave a Reply

Your email address will not be published. Required fields are marked *