by From Lars Syll One way that macroeconomics stands out from other fields in economics is in how often it produces forecasts. The vast majority of empirical models in economics can be very successful at identifying causal relations or at fitting explaining behavior, but they are never used to provide unconditional forecasts, nor do people expect them to. Macroeconomists, instead, are asked to routinely produce forecasts to guide fiscal and monetary policy, and are perhaps too eager to comply … Forecasting is hard. Forecasting what people will do when their behavior is affected by many interrelated personal, local, and national variables is even harder. Forecasting when the forecasts cause changes in policy, which make people change their choices, which in turn make it required to revise the forecasts, is iteratively hard. Forecasting when economic agents themselves are forecasting your forecast to anticipate the policies that will be adopted involves strategic thinking and game theory that goes well beyond the standard statistical toolbox … To conclude with the most important message, yes, economics models do a poor job forecasting macroeconomic variables. This deserves to be exposed, discussed, and even sometimes ridiculed. Critics like Haldane (2016) are surely right, and the alternatives that they propose for improvement are definitely worth exploring.
Topics:
Lars Pålsson Syll considers the following as important: Uncategorized
This could be interesting, too:
Dean Baker writes Health insurance killing: Economics does have something to say
Lars Pålsson Syll writes Debunking mathematical economics
John Quiggin writes RBA policy is putting all our futures at risk
Merijn T. Knibbe writes ´Extra Unordinarily Persistent Large Otput Gaps´ (EU-PLOGs)
from Lars Syll
One way that macroeconomics stands out from other fields in economics is in how often it produces forecasts. The vast majority of empirical models in economics can be very successful at identifying causal relations or at fitting explaining behavior, but they are never used to provide unconditional forecasts, nor do people expect them to. Macroeconomists, instead, are asked to routinely produce forecasts to guide fiscal and monetary policy, and are perhaps too eager to comply …
Forecasting is hard. Forecasting what people will do when their behavior is affected by many interrelated personal, local, and national variables is even harder. Forecasting when the forecasts cause changes in policy, which make people change their choices, which in turn make it required to revise the forecasts, is iteratively hard. Forecasting when economic agents themselves are forecasting your forecast to anticipate the policies that will be adopted involves strategic thinking and game theory that goes well beyond the standard statistical toolbox …
To conclude with the most important message, yes, economics models do a poor job forecasting macroeconomic variables. This deserves to be exposed, discussed, and even sometimes ridiculed. Critics like Haldane (2016) are surely right, and the alternatives that they propose for improvement are definitely worth exploring. If nothing else, this may help the media and the public to start reporting and reading forecasts as probabilistic statements where the confidence bands or fan charts are as or more important than the point forecasts. But, before jumping to the conclusion that this is a damning critique of the state of macroeconomics, this section asked for an evaluation of forecasting performance in relative terms. Relative to other conditional predictions on the effectiveness of policies, relative to other forecasts for large diverse populations also made many years out, and relative to their accuracy per dollar of funding. From these perspectives, I am less convinced that economics forecasting is all that far behind other scientific fields.
Yes, indeed, forecasting sure is difficult. On that we all agree. But having said that, it’s pretty perplexing that a mainstream macroeconomist who has spent a large part of his career developing macro models, with all of the more or less standard assumptions on rational representative agents making only stochastic mistakes confronting intermittent ‘shocks’ in DSGE models with risk reduced uncertainty, admits that the models perform poorly when it comes to forecasting. Isn’t that kind of admitting that there is a monumental gap between model and reality? And isn’t that what critics — who Reis spend a lot of the article criticising — have pointed at for decades now?
Reis is not the only one trying to defend this rather unproductive forecasting activity so many modern macroeconomists spend their working days with. Not that long ago, Reis’ colleague Simon Wren-Lewis had a post up on forecasting activities, also defending an activity that admittedly has little value and most often come up with results no better than “intelligent guessing.”
So why do companies, governments, academic researchers, and central banks, continue with this more or less expensive, but obviously worthless, activity?
A couple of years ago yours truly was interviewed by a public radio journalist working on a series on Great Economic Thinkers. We were discussing the monumental failures of the predictions-and-forecasts-business. But — the journalist asked — if these cocksure economists with their “rigorous” and “precise” mathematical-statistical-econometric models are so wrong again and again — why do they persist wasting time on it?
In a discussion on uncertainty and the hopelessness of accurately modeling what will happen in the real world — in M. Szenberg’s Eminent Economists: Their Life Philosophies — Kenneth Arrow came up with what is probably the most plausible reason:
It is my view that most individuals underestimate the uncertainty of the world. This is almost as true of economists and other specialists as it is of the lay public. To me our knowledge of the way things work, in society or in nature, comes trailing clouds of vagueness … Experience during World War II as a weather forecaster added the news that the natural world as also unpredictable.
An incident illustrates both uncer-tainty and the unwilling-ness to entertain it. Some of my colleagues had the responsi-bility of preparing long-range weather forecasts, i.e., for the following month. The statisticians among us subjected these forecasts to verification and found they differed in no way from chance. The forecasters themselves were convinced and requested that the forecasts be discontinued. The reply read approximately like this: ‘The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.’
To this one might also add some concerns about ideology and apologetics. Although Reis is — as most other mainstream economists — only reluctantly prepared to discuss ideology, it’s undeniable that forecasting certainly is a non-negligible part of the labour market for (mainstream) economists, and so, of course, those in the business do not want to admit that they are occupied with worthless things (not to mention how hard it would be to sell the product with that kind of frank truthfulness …). Governments, the finance sector and (central) banks also want to give the impression to customers and voters that they, so to say, have the situation under control (telling people that next years X will be 3.048 % makes wonders in that respect). Why else would anyone want to pay them or vote for them? These are sure not glamorous aspects of economics as a science, but as a scientist it would be unforgivably dishonest to pretend that economics doesn’t also perform an ideological function in society.
And ultimately — who supplies these banks and companies with the basic models for the forecasting? People like Reis. I guess we have to allocate a part of the unforgivable dishonesty also to the academics that come up with blueprints for expensive and worthless activities.