by The presumed advantage of the experimentalist approach Here, I want to challenge the popular view that “natural experiments” offer a simple, robust and relatively “assumption free” way to learn interesting things about economic relationships. Indeed, I will argue that it is not possible to learn anything of interest from data without theoretical assumptions, even when one has available an “ideal instrument”. Data cannot determine interesting economic relationships without a priori identifying assumptions, regardless of what sort of idealized experiments, “natural experiments” or “quasi-experiments” are present in that data. Economic models are always needed to provide a window through which we interpret data, and our interpretation will always be
Topics:
Lars Pålsson Syll considers the following as important: Statistics & Econometrics
This could be interesting, too:
Lars Pålsson Syll writes What statistics teachers get wrong!
Lars Pålsson Syll writes Statistical uncertainty
Lars Pålsson Syll writes The dangers of using pernicious fictions in statistics
Lars Pålsson Syll writes Interpreting confidence intervals
The presumed advantage of the experimentalist approach
Here, I want to challenge the popular view that “natural experiments” offer a simple, robust and relatively “assumption free” way to learn interesting things about economic relationships. Indeed, I will argue that it is not possible to learn anything of interest from data without theoretical assumptions, even when one has available an “ideal instrument”. Data cannot determine interesting economic relationships without a priori identifying assumptions, regardless of what sort of idealized experiments, “natural experiments” or “quasi-experiments” are present in that data. Economic models are always needed to provide a window through which we interpret data, and our interpretation will always be subjective, in the sense that it is contingent on our model.
Furthermore, atheoretical “experimentalist” approaches do not rely on fewer or weaker assumptions than do structural approaches. The real distinction is that, in a structural approach, one’s a priori assumptions about behavior must be laid out explicitly, while in an experimentalist approach, key assumptions are left implicit …
If one accepts that inferences drawn from experimentalist work are just as contingent on a priori assumptions as those from structural work, the key presumed advantage of the experimentalist approach disappears. One is forced to accept that all empirical work in economics, whether “experimentalist” or “structural”, relies critically on a priori theoretical assumptions.
In econometrics, it is often said that the error term in the regression model used represents the effect of the variables that are omitted from the model. The error term is somehow thought to be a ‘cover-all’ term representing omitted content in the model and necessary to include to ‘save’ the assumed deterministic relation between the other random variables included in the model. Error terms are usually assumed to be orthogonal (uncorrelated) to the explanatory variables. But since they are unobservable, they are also impossible to empirically test. And without justification of the orthogonality assumption, there is as a rule nothing to ensure identifiability. To me, this only highlights that the important lesson to draw from the debate between ‘structuralist’ and ‘experimentalist’ econometricians is that no matter what set of assumptions you choose to build your analysis on, you will never be able to empirically test them conclusively. Ultimately it always comes down to a question of faith.