Tag Archives: Statistics & Econometrics

.[embedded content]

The inherent epistemological limitation of econometric testing To understand the relationship between economic data and economic phenomena, it is helpful first to be clear about what we mean by each of these terms. Following Jim Woodward (1989), we can characterize “phenomena” as features of our experience that we take to be “relatively stable” and “which are potential objects of explanation and prediction by general theory.” The phenomena themselves are in...

The difference between rate and probability (wonkish) Suppose there is a series of Bernoulli trials, that each trial has the same probability p of success, and that the trials are independent—like the standard model of coin tossing, treating ‘heads’ as ‘success.’ Then the Law of Large Numbers guarantees that the rate of successes converges (in probability) to the probability of success. If a sequence of trials is random and the chance of success is the same...

What is this thing called probability? Fitting a model that has a parameter called ‘probability’ to data does not mean that the estimated value of that parameter estimates the probability of anything in the real world. Just as the map is not the territory, the model is not the phenomenon, and calling something ‘probability’ does not make it a probability, any more than drawing a mountain on a map creates a real mountain … In summary, the word ‘probability’...

Beware of Monte Carlo simulations In some fields—physics, geophysics, climate science, sensitivity analysis, and uncertainty quantification in particular—there is a popular impression that probabilities can be estimated in a ‘neutral’ or ‘automatic’ way by doing Monte Carlo simulations: just let the computer reveal the distribution … Setting aside other issues in numerical modeling, Monte Carlo simulation is a way to substitute computing for hand...

How to do econometrics properly Always, but always, plot your data. Remember that data quality is at least as important as data quantity. Always ask yourself, “Do these results make economic/common sense”? Check whether your “statistically significant” results are also “numerically/economically significant”. Be sure that you know exactly what assumptions are used/needed to obtain the results relating to the properties of any estimator or test that you...

While appeal to R squared is a common rhetorical device, it is a very tenuous connection to any plausible explanatory virtues for many reasons. Either it is meant to be merely a measure of predictability in a given data set or it is a measure of causal influence. In either case it does not tell us much about explanatory power. Taken as a measure of predictive power, it is limited in that it predicts variances only. But what we mostly want to predict is levels, about which it...

Adjusting for confounding (student stuff) .[embedded content] Simpson’s paradox is an interesting paradox in itself, but it also highlights a deficiency in the traditional econometric approach towards causality. Say you have 1000 observations on men and an equal amount of observations on women applying for admission to university studies, and that 70% of men are admitted, but only 30% of women. Running a logistic regression to find out the odds ratios (and...

What RCTs can and cannot tell us Unfortunately, social sciences’ hope that we can control simultaneously for a range of factors like education, labor force attachment, discrimination, and others is simply more wishful thinking. The problem is that the causal relations underlying such associations are so complex and so irregular that the mechanical process of regression analysis has no hope of unpacking them. One hope for quantitative researchers who...

The Deadly Sin of Statistical Reification People sometimes speak as if random variables “behave” in a certain way, as if they have a life of their own. Thus “X is normally distributed”, “W follows a gamma”, “The underlying distribution behind y is binomial”, and so on. To behave is to act, to be caused, to react. Somehow, it is thought, these distributions are causes. This is the Deadly Sin of Reification, perhaps caused by the beauty of the mathematics...