Wednesday , November 6 2024
Home / Real-World Economics Review / So much for ‘statistical objectivity’

So much for ‘statistical objectivity’

Summary:
From Lars Syll Last year, we recruited 29 teams of researchers and asked them to answer the same research question with the same data set. Teams approached the data with a wide array of analytical techniques, and obtained highly varied results … All teams were given the same large data set collected by a sports-statistics firm across four major football leagues. It included referee calls, counts of how often referees encountered each player, and player demographics including team position, height and weight. It also included a rating of players’ skin colour … Of the 29 teams, 20 found a statistically significant correlation between skin colour and red cards … Findings varied enormously, from a slight (and non-significant) tendency for referees to give more red cards to light-skinned

Topics:
Lars Pålsson Syll considers the following as important:

This could be interesting, too:

Merijn T. Knibbe writes ´Fryslan boppe´. An in-depth inspirational analysis of work rewarded with the 2024 Riksbank prize in economic sciences.

Peter Radford writes AJR, Nobel, and prompt engineering

Lars Pålsson Syll writes Central bank independence — a convenient illusion

Eric Kramer writes What if Trump wins?

from Lars Syll

Last year, we recruited 29 teams of researchers and asked them to answer the same research question with the same data set. Teams approached the data with a wide array of analytical techniques, and obtained highly varied results …

All teams were given the same large data set collected by a sports-statistics firm across four major football leagues. It included referee calls, counts of how often referees encountered each player, and player demographics including team position, height and weight. It also included a rating of players’ skin colour …

So much for ‘statistical objectivity’Of the 29 teams, 20 found a statistically significant correlation between skin colour and red cards … Findings varied enormously, from a slight (and non-significant) tendency for referees to give more red cards to light-skinned players to a strong trend of giving more red cards to dark-skinned players …

Had any one of these 29 analyses come out as a single peer-reviewed publication, the conclusion could have ranged from no race bias in referee decisions to a huge bias.

Raphael Silberzahn & Eric Uhlmann

Research that strongly underlines that even in statistics, the researcher has many degrees of freedom. In statistics — as in economics and econometrics — the results we get depend on the assumptions we make in our models. Changing those assumptions — playing a more important role than the data we feed into our models — leed to far-reaching changes in our conclusions. Using statistics​ is no guarantee we get at any ‘objective truth.’

Lars Pålsson Syll
Professor at Malmö University. Primary research interest - the philosophy, history and methodology of economics.

Leave a Reply

Your email address will not be published. Required fields are marked *