by From Lars Syll Although discounting empirical evidence cannot be the right way to solve economic issues, there are still, in my opinion, a couple of weighty reasons why we perhaps shouldn’t be too excited about the so-called ’empirical revolution’ in economics. Behavioural experiments and laboratory research face the same basic problem as theoretical models — they are built on often rather artificial conditions and have difficulties with the ‘trade-off’ between internal and external validity. The more artificial conditions, the more internal validity, but also less external validity. The more we rig experiments to avoid the ‘confounding factors’, the less the conditions are reminiscent of the real ‘target system.’ The nodal issue is how economists using different isolation strategies
Topics:
Lars Pålsson Syll considers the following as important: Uncategorized
This could be interesting, too:
Lars Pålsson Syll writes Busting the ‘natural rate of unemployment’ myth
Merijn T. Knibbe writes The political economy of estimating productivity.
Merijn T. Knibbe writes Peak babies has been. Young men are not expendable, anymore.
Lars Pålsson Syll writes NAIRU — a harmful fairy tale
from Lars Syll
Although discounting empirical evidence cannot be the right way to solve economic issues, there are still, in my opinion, a couple of weighty reasons why we perhaps shouldn’t be too excited about the so-called ’empirical revolution’ in economics.
Behavioural experiments and laboratory research face the same basic problem as theoretical models — they are built on often rather artificial conditions and have difficulties with the ‘trade-off’ between internal and external validity. The more artificial conditions, the more internal validity, but also less external validity. The more we rig experiments to avoid the ‘confounding factors’, the less the conditions are reminiscent of the real ‘target system.’ The nodal issue is how economists using different isolation strategies in different ‘nomological machines’ attempt to learn about causal relationships. One may have justified doubts on the generalizability of this research strategy since the probability is high that causal mechanisms are different in different contexts and that lack of homogeneity and invariance doesn’t give us warranted export licenses to the ‘real’ societies or economies.
If we see experiments or laboratory research as theory tests or models that ultimately aspire to say something about the real ‘target system,’ then the problem of external validity is central (and was for a long time also a key reason why behavioural economists had trouble getting their research results published).
A standard procedure in behavioural economics — think of e.g. dictator or ultimatum games — is to set up a situation where one induce people to act according to the standard microeconomic — homo oeconomicus — benchmark model. In most cases, the results show that people do not behave as one would have predicted from the benchmark model, in spite of the setup almost invariably being ‘loaded’ for that purpose. [And in those cases where the result is consistent with the benchmark model, one, of course, have to remember that this in no way proves the benchmark model to be right or ‘true,’ since there, as a rule, may be many outcomes that are consistent with that model.]
For most heterodox economists this is just one more reason for giving up on the standard model. But not so for mainstreamers and many behaviouralists. To them, the empirical results are not reasons for giving up on their preferred hardcore axioms. So they set out to ‘save’ or ‘repair’ their model and try to ‘integrate’ the empirical results into mainstream economics. Instead of accepting that the homo oeconomicus model has zero explanatory real-world value, one puts lipstick on the pig and hope to go on with business as usual. Why we should keep on using that model as a benchmark when everyone knows it is false is something we are never told. Instead of using behavioural economics and its results as building blocks for a progressive alternative research program, the ‘save and repair’ strategy immunizes a hopelessly false and irrelevant model.
By this, I do not mean to say that empirical methods per se are so problematic that they can never be used. On the contrary, I am basically — though not without reservations — in favour of the increased use of behavioural experiments and laboratory research within economics. Not least as an alternative to completely barren ‘bridge-less’ axiomatic-deductive theory models. My criticism is more about aspiration levels and what we believe that we can achieve with our mediational epistemological tools and methods in the social sciences.
The increasing use of natural and quasi-natural experiments in economics during the last couple of decades has led several prominent economists to triumphantly declare it as a major step on a recent path toward empirics, where instead of being a deductive philosophy, economics is now increasingly becoming an inductive science.
Limiting model assumptions in economic science always have to be closely examined since if we are going to be able to show that the mechanisms or causes that we isolate and handle in our models are stable in the sense that they do not change when we ‘export’ them to our ‘target systems,’ we have to be able to show that they do not only hold under ceteris paribusconditions and a fortiori only are of limited value to our understanding, explanations or predictions of real economic systems.
‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. ‘It works there’ is no evidence for ‘it will work here.’ Causes deduced in an experimental setting still have to show that they come with an export-warrant to the target system. The causal background assumptions made have to be justified, and without licenses to export, the value of ‘rigorous’ and ‘precise’ methods is despairingly small.
Taking assumptions like utility maximization or market equilibrium as a matter of course leads to the ‘standing presumption in economics that, if an empirical statement is deduced from standard assumptions then that statement is reliable’ …
The ongoing importance of these assumptions is especially evident in those areas of economic research, where empirical results are challenging standard views on economic behaviour like experimental economics or behavioural finance … From the perspective of Model-Platonism, these research-areas are still framed by the ‘superior insights’ associated with early 20th century concepts, essentially because almost all of their results are framed in terms of rational individuals, who engage in optimizing behaviour and, thereby, attain equilibrium. For instance, the attitude to explain cooperation or fair behaviour in experiments by assuming an ‘inequality aversion’ integrated into (a fraction of) the subjects’ preferences is strictly in accordance with the assumption of rational individuals, a feature which the authors are keen to report …
While the mere emergence of research areas like experimental economics is sometimes deemed a clear sign for the advent of a new era … a closer look at these fields allows us to illustrate the enduring relevance of the Model-Platonism-topos and, thereby, shows the pervasion of these fields with a traditional neoclassical style of thought.
So — although it is good that people like Kahneman and Thaler are rewarded ‘Nobel prizes’ and that much of their research has vastly undermined the lure of axiomatic-deductive mainstream economics, there is still a long way to go before economics has become a truly empirical science.
Added : David Ruccio has a post up now on the Real-World Economics Blog that basically argues (I think) in the same spirit as yours truly here.