by Mainstream broken pieces models In economic modeling, people often just assume an objective function for one agent or another, throw that into a larger model, and then look only at some subset of the model’s overall implications. But that’s throwing away data … And in doing so, it dramatically lowers the empirical bar that a model has to clear. You’re essentially tossing a ton of broken, wrong structural assumptions into a model and then calibrating (or estimating) the parameters to match a fairly small set of things, then declaring victory. But because you’ve got the structure wrong, the model will fail and fail and fail as soon as you take it out of sample, or as soon as you apply it to any data other than the few things it was calibrated to match. Use broken pieces, and you get a broken machine … Dani Rodrik, when he talks about these issues, says that unrealistic assumptions are only bad if they’re ‘critical’ assumptions – that is, if changing them would change the model substantially. It’s OK to have non-critical assumptions that are unrealistic, just like a car will still run fine even if the cup-holder is cracked. That sounds good. In principle I agree.
Topics:
Lars Pålsson Syll considers the following as important: Economics
This could be interesting, too:
Lars Pålsson Syll writes Daniel Waldenströms rappakalja om ojämlikheten
Peter Radford writes AJR, Nobel, and prompt engineering
Lars Pålsson Syll writes MMT explained
Lars Pålsson Syll writes Statens finanser funkar inte som du tror
Mainstream broken pieces models
In economic modeling, people often just assume an objective function for one agent or another, throw that into a larger model, and then look only at some subset of the model’s overall implications. But that’s throwing away data …
And in doing so, it dramatically lowers the empirical bar that a model has to clear. You’re essentially tossing a ton of broken, wrong structural assumptions into a model and then calibrating (or estimating) the parameters to match a fairly small set of things, then declaring victory. But because you’ve got the structure wrong, the model will fail and fail and fail as soon as you take it out of sample, or as soon as you apply it to any data other than the few things it was calibrated to match.
Use broken pieces, and you get a broken machine …
Dani Rodrik, when he talks about these issues, says that unrealistic assumptions are only bad if they’re ‘critical’ assumptions – that is, if changing them would change the model substantially. It’s OK to have non-critical assumptions that are unrealistic, just like a car will still run fine even if the cup-holder is cracked. That sounds good. In principle I agree. But in practice, how the heck do you know in advance which assumptions are critical? You’d have to go check them all, by introducing alternatives for each and every one (actually every combination of assumptions, since model features tend to interact). No one is actually going to do that. It’s a non-starter.
The real solution, as I see it, is not to put any confidence in models with broken pieces.
No indeed, there’s no reason whatsoever to trust those models and their defenders.
Dani Rodrik’s Economics Rules describes economics as a more or less problem-free smorgasbord collection of models. Economics is portrayed as advancing through a judicious selection from a continually expanding library of models, models that are presented as ‘partial maps’ or ‘simplifications designed to show how specific mechanisms work.’
But one of the things that’s missing in Rodrik’s view of economic models is the all-important distinction between core and auxiliary assumptions. Although Rodrik repeatedly speaks of ‘unrealistic’ or ‘critical’ assumptions, he basically — as Noah Smith rightly remarks — just lumps them all together without differentiating between different types of assumptions, axioms or theorems.
Modern mainstream (neoclassical) economists ground their models on a set of core assumptions (CA) — basically describing the agents as ‘rational’ actors — and a set of auxiliary assumptions (AA). Together CA and AA make up what I will call the ur-model (M) of all mainstream neoclassical economic models. Based on these two sets of assumptions, they try to explain and predict both individual (micro) and — most importantly — social phenomena (macro).
The core assumptions typically consist of:
CA1 Completeness — rational actors are able to compare different alternatives and decide which one(s) he prefers
CA2 Transitivity — if the actor prefers A to B, and B to C, he must also prefer A to C.
CA3 Non-satiation — more is preferred to less.
CA4 Maximizing expected utility — in choice situations under risk (calculable uncertainty) the actor maximizes expected utility.
CA4 Consistent efficiency equilibria — the actions of different individuals are consistent, and the interaction between them result in an equilibrium.
When describing the actors as rational in these models, the concept of rationality used is instrumental rationality – choosing consistently the preferred alternative, which is judged to have the best consequences for the actor given his in the model exogenously given wishes/interests/ goals. How these preferences/wishes/interests/goals are formed is typically not considered to be within the realm of rationality, and a fortiori not constituting part of economics proper.
The picture given by this set of core assumptions (rational choice) is a rational agent with strong cognitive capacity that knows what alternatives he is facing, evaluates them carefully, calculates the consequences and chooses the one — given his preferences — that he believes has the best consequences according to him.
Weighing the different alternatives against each other, the actor makes a consistent optimizing (typically described as maximizing some kind of utility function) choice, and acts accordingly.
Beside the core assumptions (CA) the model also typically has a set of auxiliary assumptions (AA) spatio-temporally specifying the kind of social interaction between ‘rational actors’ that take place in the model. These assumptions can be seen as giving answers to questions such as
AA1 who are the actors and where and when do they act
AA2 which specific goals do they have
AA3 what are their interests
AA4 what kind of expectations do they have
AA5 what are their feasible actions
AA6 what kind of agreements (contracts) can they enter into
AA7 how much and what kind of information do they possess
AA8 how do the actions of the different individuals/agents interact with each other.
So, the ur-model of all economic models basically consist of a general specification of what (axiomatically) constitutes optimizing rational agents and a more specific description of the kind of situations in which these rational actors act (making AA serve as a kind of specification/restriction of the intended domain of application for CA and its deductively derived theorems). The list of assumptions can never be complete, since there will always unspecified background assumptions and some (often) silent omissions (like closure, transaction costs, etc., regularly based on some negligibility and applicability considerations). The hope, however, is that the ‘thin’ list of assumptions shall be sufficient to explain and predict ‘thick’ phenomena in the real, complex, world.
But in Rodrik’s model depiction we are essentially given the following structure,
A1, A2, … An
———————-
Theorem,
where a set of undifferentiated assumptions are used to infer a theorem.
This is, however, to vague and imprecise to be helpful, and does not give a true picture of the usual mainstream modeling strategy, where there’s a differentiation between a set of law-like hypotheses (CA) and a set of auxiliary assumptions (AA), giving the more adequate structure
CA1, CA2, … CAn & AA1, AA2, … AAn
———————————————–
Theorem
or,
CA1, CA2, … CAn
———————-
(AA1, AA2, … AAn) → Theorem,
more clearly underlining the function of AA as a set of (empirical, spatio-temporal) restrictions on the applicability of the deduced theorems.
This underlines the fact that specification of AA restricts the range of applicability of the deduced theorem. In the extreme cases we get
CA1, CA2, … CAn
———————
Theorem,
where the deduced theorems are analytical entities with universal and totally unrestricted applicability, or
AA1, AA2, … AAn
———————-
Theorem,
where the deduced theorem is transformed into an untestable tautological thought-experiment without any empirical commitment whatsoever beyond telling a coherent fictitious as-if story.
Not clearly differentiating between CA and AA means that Rodrik can’t make this all-important interpretative distinction, and so opens up for unwarrantedly ‘saving’ or ‘immunizing’ models from almost any kind of critique by simple equivocation between interpreting models as empirically empty and purely deductive-axiomatic analytical systems, or, respectively, as models with explicit empirical aspirations. Flexibility is usually something people deem positive, but in this methodological context it’s more troublesome than a sign of real strength. Models that are compatible with everything, or come with unspecified domains of application, are worthless from a scientific point of view.
Mainstream macro models are nothing but broken pieces models — and as Noah Smith puts it — there is no reason for us ‘to put any confidence in models with broken pieces.’