Friday , March 29 2024
Home / Mike Norman Economics / Empirical Testing Of Macro Models — Brian Romanchuk

Empirical Testing Of Macro Models — Brian Romanchuk

Summary:
From a bond market practitioner's perspective, model testing is straightforward: does it make money? The ability to make money after model implementation (and not just "out of sample") is a simple quantitative metric -- although one might need to wait for a large enough sample to test this. Not everyone is a market participant, and they want to evaluate models on other metrics (e.g., does it help guide policy decisions?). However, the key insight of the "does is make money?" metric is that it is related to the more vague: "does it offer useful information about the future?" It is entirely possible for a model to have some statistical properties that are seen as "good" -- yet offer no useful information about the future. Usefulness in Forecasting and PricingAlthough one might be able to

Topics:
Mike Norman considers the following as important:

This could be interesting, too:

Mike Norman writes Rinse and repeat–Truss chaos–the new benchmark — Bill Mitchell

Lars Pålsson Syll writes The man who never wavered — Alan Bates

Joel Eissenberg writes You can’t fool Mother Nature

Bill Haskell writes Grades and learning

From a bond market practitioner's perspective, model testing is straightforward: does it make money? The ability to make money after model implementation (and not just "out of sample") is a simple quantitative metric -- although one might need to wait for a large enough sample to test this. Not everyone is a market participant, and they want to evaluate models on other metrics (e.g., does it help guide policy decisions?). However, the key insight of the "does is make money?" metric is that it is related to the more vague: "does it offer useful information about the future?" It is entirely possible for a model to have some statistical properties that are seen as "good" -- yet offer no useful information about the future.
Usefulness in Forecasting and Pricing
Although one might be able to make money from a mathematical model any number of ways, I am considering two types of financial models that are of interest.
  • Forecasting models that generate buy/sell signals.
  • Pricing models. Although this sounds unusual in a macroeconomics context, this is related to DSGE models, given their similarity to arbitrage-free pricing models (link to previous discussion).
Forecasting models are probably what most people would think of, the structure of DSGE models implies a need to worrying about pricing concepts. The key observation one can make about financial forecasting models is that they are not evaluated based on statistical tests (r-squared, whatever), rather the profits they generate after model creation. That is, passing statistical tests does not translate into a useful model.

Since there is less to say about pricing models, I will discuss them first....

Bond Economics
Empirical Testing Of Macro Models
Brian Romanchuk
Mike Norman
Mike Norman is an economist and veteran trader whose career has spanned over 30 years on Wall Street. He is a former member and trader on the CME, NYMEX, COMEX and NYFE and he managed money for one of the largest hedge funds and ran a prop trading desk for Credit Suisse.

Leave a Reply

Your email address will not be published. Required fields are marked *