Tag Archives: Statistics & Econometrics

While appeal to R squared is a common rhetorical device, it is a very tenuous connection to any plausible explanatory virtues for many reasons. Either it is meant to be merely a measure of predictability in a given data set or it is a measure of causal influence. In either case it does not tell us much about explanatory power. Taken as a measure of predictive power, it is limited in that it predicts variances only. But what we mostly want to predict is levels, about which it...

Adjusting for confounding (student stuff) .[embedded content] Simpson’s paradox is an interesting paradox in itself, but it also highlights a deficiency in the traditional econometric approach towards causality. Say you have 1000 observations on men and an equal amount of observations on women applying for admission to university studies, and that 70% of men are admitted, but only 30% of women. Running a logistic regression to find out the odds ratios (and...

What RCTs can and cannot tell us Unfortunately, social sciences’ hope that we can control simultaneously for a range of factors like education, labor force attachment, discrimination, and others is simply more wishful thinking. The problem is that the causal relations underlying such associations are so complex and so irregular that the mechanical process of regression analysis has no hope of unpacking them. One hope for quantitative researchers who...

The Deadly Sin of Statistical Reification People sometimes speak as if random variables “behave” in a certain way, as if they have a life of their own. Thus “X is normally distributed”, “W follows a gamma”, “The underlying distribution behind y is binomial”, and so on. To behave is to act, to be caused, to react. Somehow, it is thought, these distributions are causes. This is the Deadly Sin of Reification, perhaps caused by the beauty of the mathematics...

Subjective probability — answering questions nobody asked Solve for x — give a single, unique number — in the following equation: x + y = 3. Of course, it cannot be done: under no rules of mathematics can a unique x be discovered; there are one too many unknowns. Nevertheless, someone holding to the subjective interpretation of probability could tell us, say, “1 feel x = 7.” Or he might say, “The following is my distribution for the possible values of x.”...

Trygve Haavelmo — with the completion (in 1958) of the twenty-fifth volume of Econometrica — assessed the role of econometrics in the advancement of economics, and although mainly positive of the “repair work” and “clearing-up work” done, he also found some grounds for despair: We have found certain general principles which would seem to make good sense. Essentially, these principles are based on the reasonable idea that, if an economic model is in fact “correct” or “true,” we...

Causal assumptions in need of careful justification As is brilliantly attested by the work of Pearl, an extensive and fruitful theory of causality can be erected upon the foundation of a Pearlian DAG. So, when we can assume that a certain DAG is indeed a Pearlian DAG representation of a system, we can apply that theory to further our causal understanding of the system. But this leaves entirely untouched the vital questions: when is a Pearlian DAG...

The importance of not equating science with statistical calculations All science entails human judgment, and using statistical models doesn’t relieve us of that necessity. Working with misspecified models, the scientific value of statistics is actually zero — even though you’re making valid statistical inferences! Statistical models are no substitutes for doing real science. Or as a famous German philosopher famously wrote 150 years ago: There is no royal...

Some pitfalls of using randomization and instrumental variables .[embedded content] Great presentation, but I think Angrist should have also mentioned that although ‘ideally controlled experiments’ may tell us with certainty what causes what effects, this is so only when given the right ‘closures.’ Making appropriate extrapolations from — ideal, accidental, natural or quasi — experiments to different settings, populations or target systems, is not easy. “It...

Bayes theorem — what’s the big deal? There’s nothing magical about Bayes’ theorem. It boils down to the truism that your belief is only as valid as its evidence. If you have good evidence, Bayes’ theorem can yield good results. If your evidence is flimsy, Bayes’ theorem won’t be of much use. Garbage in, garbage out. The potential for Bayes abuse begins with your initial estimate of the probability of your belief, often called the “prior” … In many cases,...