by Noah Smith has a fantastic essay on the problem with economics papers. Some excerpts: …in practice, I think what often happens in econ is more like the following: 1. Some papers make structural models, observe that these models can fit (or sort-of fit) a couple of stylized facts, and call it a day. Economists who like these theories (based on intuition, plausibility, or the fact that their dissertation adviser made the model) then use them for policy predictions forever after, without ever checking them rigorously against empirical evidence. 2. Other papers do purely empirical work, using simple linear models. Economists then use these linear models to make policy predictions (“Minimum wages don’t have significant disemployment effects”). 3. A third group of papers do empirical work,
Topics:
Chris Blattman considers the following as important: Economics, Research
This could be interesting, too:
Lars Pålsson Syll writes Daniel Waldenströms rappakalja om ojämlikheten
Peter Radford writes AJR, Nobel, and prompt engineering
Lars Pålsson Syll writes MMT explained
Lars Pålsson Syll writes Statens finanser funkar inte som du tror
Noah Smith has a fantastic essay on the problem with economics papers. Some excerpts:
…in practice, I think what often happens in econ is more like the following:
1. Some papers make structural models, observe that these models can fit (or sort-of fit) a couple of stylized facts, and call it a day. Economists who like these theories (based on intuition, plausibility, or the fact that their dissertation adviser made the model) then use them for policy predictions forever after, without ever checking them rigorously against empirical evidence.
2. Other papers do purely empirical work, using simple linear models. Economists then use these linear models to make policy predictions (“Minimum wages don’t have significant disemployment effects”).
3. A third group of papers do empirical work, observe the results, and then make one structural model per paper to “explain” the empirical result they just found. These models are generally never used or seen again.
A lot of young, smart economists trying to make it in the academic world these days seem to write papers that fall into Group 3.
…It’s easy to see this pro-forma model-making as a sort of conformity signaling – young, empirically-minded economists going the extra mile to prove that they don’t think the work of the older “theory generation” (who are now their advisers, reviewers, editors and senior colleagues) was for naught.
…But what is the result of all this pro-forma model-making? To some degree it’s just a waste of time and effort, generating models that will never actually be used for anything.
…Paul Romer worries about this in his “mathiness” essay:
[T]he new equilibrium: empirical work is science; theory is entertainment. Presenting a model is like doing a card trick. Everybody knows that there will be some sleight of hand. There is no intent to deceive because no one takes it seriously.
Pure theory papers seem to rarely get checked against data, leading to an accumulation on the shelves of models that support any and every conclusion. Meanwhile, pure empirical papers don’t often get used as guides to finding good structural models, but are simply linearly extrapolated.
Noah flags a bigger problem, however:
In other words, econ seems too focused on “theory vs. evidence” instead of using the two in conjunction. And when they do get used in conjunction, it’s often in a tacked-on, pro-forma sort of way, without a real meaningful interplay between the two.
…I see very few economists explicitly calling for the kind of “combined approach” to modeling that exists in other sciences – i.e., using evidence to continuously restrict the set of usable models.
Another view, however, is that for many problems it’s too soon to build a model. We need more data points and stylized facts before our models will be any good. For a given question, it’s rare to have more than a couple good empirical studies, especially outside the US.
I don’t completely buy this view. Theories and models will be useful along the way to guide where we seek data or how we design experiments. And I personally have found simple general models help me in interpreting the results of my own experiments. So I like seeing them. But I think we are a long way from having enough evidence to restrict the set of useable models.