How to ensure that models serve society

by
Lars Pålsson Syll
My articles My siteAbout meMy booksMy videos
Follow on:Twitter

How to ensure that models serve society

• Mind the assumptions — assess uncertainty and sensitivity.

• Mind the hubris — complexity can be the enemy of relevance.

• Mind the framing — match purpose and context.

• Mind the consequences — quantification may backfire.

• Mind the unknowns — acknowledge ignorance.

Andrea Saltelli, John Kay, Deborah Mayo, Philip B. Stark, et al.

Five principles I think modern times “the model is the message” economists would benefit much from pondering. And especially when it comes to the last principles, they would benefit enormously from reading.

More than a hundred years after John Maynard Keynes wrote his seminal A Treatise on Probability (1921), it is still very difficult to find economics and statistics textbooks that seriously try to incorporate his far-reaching and incisive analysis of induction and evidential weight.

The standard view in statistics and economics — and the axiomatic probability theory underlying it — is to a large extent based on the rather simplistic idea that “more is better.” But as Keynes argues — “more of the same” is not what is essential when making inductive inferences. It’s rather a question of “more but different.”

Variation, not replication, is at the core of induction. Finding that p(x|y) = p(x|y & w) doesn’t make w “irrelevant.” Knowing that the probability is unchanged when w is present gives p(x|y & w) another evidential weight (“weight of argument”).

According to Keynes we live in a world permeated by unmeasurable uncertainty — not quantifiable stochastic risk — which often forces us to make decisions based on anything but “rational expectations.” Keynes rather thinks that we base our expectations on the confidence or “weight” we put on different events and alternatives. To Keynes, expectations are a question of weighing probabilities by “degrees of belief,” beliefs that often have preciously little to do with the kind of stochastic probabilistic calculations made by the rational agents as modelled by “modern” social sciences. And often we “simply do not know.” 

Science according to Keynes should help us penetrate “the true process of causation lying behind current events” and disclose “the causal forces behind the apparent facts.” Models can never be more than a starting point in that endeavour. He further argued that it was inadmissible to project history on the future. Consequently, we cannot presuppose that what has worked before, will continue to do so in the future. Getting hold of correlations between different “variables” is not enough. If our models cannot get at the causal structure that generated the data, they are not really “identified.”

How strange that writers of economics and statistics textbooks as a rule do not even touch upon these aspects of scientific methodology that seem so fundamental and important for anyone trying to understand how we learn and orient ourselves in an uncertain world. An educated guess on why this is a fact would be that Keynes’s concepts are not possible to squeeze into a single calculable numerical “probability.” In the quest for quantities, one puts a blind eye to qualities and looks the other way – but Keynes’s ideas keep creeping out from under the carpet.

It’s high time that economics and statistics textbooks give Keynes his due.