by The causal revolution in econometrics has gone too far Kevin Lewis points us to this recent paper, “Can invasive species lead to sedentary behavior? The time use and obesity impacts of a forest-attacking pest,” published in Elsevier’s Journal of Environmental Economics and Management, which has the following abstract: “Invasive species can significantly disrupt environmental quality and flows of ecosystem services and we are still learning about their multidimensional impacts to economic outcomes of interest. In this work, I use quasi-random US county detections of the invasive emerald ash borer (EAB), a forest-attacking pest, to investigate how invasive-induced deforestation can impact obesity rates and time spent on physical activity … Results are
Topics:
Lars Pålsson Syll considers the following as important: Statistics & Econometrics
This could be interesting, too:
Lars Pålsson Syll writes The importance of ‘causal spread’
Lars Pålsson Syll writes Applied econometrics — a messy business
Lars Pålsson Syll writes Feynman’s trick (student stuff)
Lars Pålsson Syll writes Difference in Differences (student stuff)
The causal revolution in econometrics has gone too far
Kevin Lewis points us to this recent paper, “Can invasive species lead to sedentary behavior? The time use and obesity impacts of a forest-attacking pest,” published in Elsevier’s Journal of Environmental Economics and Management, which has the following abstract:
“Invasive species can significantly disrupt environmental quality and flows of ecosystem services and we are still learning about their multidimensional impacts to economic outcomes of interest. In this work, I use quasi-random US county detections of the invasive emerald ash borer (EAB), a forest-attacking pest, to investigate how invasive-induced deforestation can impact obesity rates and time spent on physical activity … Results are supported by many robustness and falsification tests and an alternative IV specification. This work has policy implications for invasive species management and expands our understanding of invasive species impacts on additional economic outcomes of interest.”
Seeing this sort of thing makes me feel that causal revolution in econometrics has gone too far … The problem I have is with the second part of the analysis, on obesity and time spent on outdoor sports and exercise. It just seems too much of a stretch, especially given that the whole analysis is on a county level.
To put it another way: there are lots and lots of things that could affect obesity and time spent on exercise, and invasive species reducing forest cover seems like the least of it.
From the other direction: the places where invasive species are spreading is not a random selection of U.S. counties. Places with more or less invasive species will differ in all sorts of ways, some of which might happen to be correlated with time spent on exercise, obesity, all sorts of things.
In short, I see no reason to believe the causal claims made in the article …
Remember that quote from Tukey, “The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data”? …
As I see it, there’s a misplaced empiricism going on here, an idea that by using proper econometric or statistical techniques you can obtain a “reduced-form” estimate. The trouble, as usual, is that:
1. Realistic effect sizes will be impossible to detect in the context of natural variation,
2. Forking paths allow researchers to satisfy that “aching desire” for a conclusive finding,
3. P-values, robustness tests, etc. help researchers convince themselves that the patterns they see in these data provide strong evidence for the stories they want to tell.
4. Given an existing academic tradition, researchers don’t notice 1, 2, and 3 above. They’re like the proverbial fish not seeing the water they’re swimming in.
Most work in econometrics is done on the assumption that the researcher has a theoretical model that is ‘true.’ Based on this belief of having a correct specification for an econometric model, one proceeds as if the only problem remaining to solve has to do with measurement and observation.
The problem is that there is little to support the perfect specification assumption. Looking around in social science and economics we don’t find a single econometric model that lives up to the standards set by the ‘true’ theoretical model — and there is nothing that gives us reason to believe things will be different in the future.
To think that we are able to construct a model where all relevant variables are included and correctly specify the functional relationships that exist between them is not only a belief with little support but a belief impossible to support.
The theories we work with when building our econometric models are insufficient. No matter what we study, there are always some variables missing, and we don’t know the correct way to functionally specify the relationships between the variables.
Every econometric model constructed is misspecified. There is always an endless list of possible variables to include and endless possible ways to specify the relationships between them. So every applied econometrician comes up with his own specification and ‘parameter’ estimates. The econometric Holy Grail of consistent and stable parameter values is nothing but a dream.
The theoretical conditions that have to be fulfilled for econometrics to really work are nowhere even closely met in reality. Making outlandish statistical assumptions does not provide a solid ground for doing relevant social science and economics. Although regression analysis and econometrics have become the most used quantitative methods in social sciences and economics today, it’s still a fact that the inferences made from them are of strongly questionable validity.