Tag Archives: Statistics & Econometrics

The Deadly Sin of Statistical Reification People sometimes speak as if random variables “behave” in a certain way, as if they have a life of their own. Thus “X is normally distributed”, “W follows a gamma”, “The underlying distribution behind y is binomial”, and so on. To behave is to act, to be caused, to react. Somehow, it is thought, these distributions are causes. This is the Deadly Sin of Reification, perhaps caused by the beauty of the mathematics...

Subjective probability — answering questions nobody asked Solve for x — give a single, unique number — in the following equation: x + y = 3. Of course, it cannot be done: under no rules of mathematics can a unique x be discovered; there are one too many unknowns. Nevertheless, someone holding to the subjective interpretation of probability could tell us, say, “1 feel x = 7.” Or he might say, “The following is my distribution for the possible values of x.”...

Trygve Haavelmo — with the completion (in 1958) of the twenty-fifth volume of Econometrica — assessed the role of econometrics in the advancement of economics, and although mainly positive of the “repair work” and “clearing-up work” done, he also found some grounds for despair: We have found certain general principles which would seem to make good sense. Essentially, these principles are based on the reasonable idea that, if an economic model is in fact “correct” or “true,” we...

Causal assumptions in need of careful justification As is brilliantly attested by the work of Pearl, an extensive and fruitful theory of causality can be erected upon the foundation of a Pearlian DAG. So, when we can assume that a certain DAG is indeed a Pearlian DAG representation of a system, we can apply that theory to further our causal understanding of the system. But this leaves entirely untouched the vital questions: when is a Pearlian DAG...

The importance of not equating science with statistical calculations All science entails human judgment, and using statistical models doesn’t relieve us of that necessity. Working with misspecified models, the scientific value of statistics is actually zero — even though you’re making valid statistical inferences! Statistical models are no substitutes for doing real science. Or as a famous German philosopher famously wrote 150 years ago: There is no royal...

Some pitfalls of using randomization and instrumental variables .[embedded content] Great presentation, but I think Angrist should have also mentioned that although ‘ideally controlled experiments’ may tell us with certainty what causes what effects, this is so only when given the right ‘closures.’ Making appropriate extrapolations from — ideal, accidental, natural or quasi — experiments to different settings, populations or target systems, is not easy. “It...

Bayes theorem — what’s the big deal? There’s nothing magical about Bayes’ theorem. It boils down to the truism that your belief is only as valid as its evidence. If you have good evidence, Bayes’ theorem can yield good results. If your evidence is flimsy, Bayes’ theorem won’t be of much use. Garbage in, garbage out. The potential for Bayes abuse begins with your initial estimate of the probability of your belief, often called the “prior” … In many cases,...

‘Severe tests’ of causal claims For many questions in the social sciences, a research design guaranteeing the validity of causal inferences is difficult to obtain. When this is the case, researchers can attempt to defend hypothesized causal relationships by seeking data that subjects their theory to repeated falsification. Karl Popper famously argued that the degree to which we have confidence in a hypothesis is not necessarily a function of the number of...

An ongoing concern is that excessive focus on formal modeling and statistics can lead to neglect of practical issues and to overconfidence in formal results … Analysis interpretation depends on contextual judgments about how reality is to be mapped onto the model, and how the formal analysis results are to be mapped back into reality. But overconfidence in formal outputs is only to be expected when much labor has gone into deductive reasoning. First, there is a need to feel...

On the limited value of randomization In Social Science and Medicine (December 2017), Angus Deaton & Nancy Cartwright argue that Randomized Controlled Trials (RCTs) do not have any warranted special status. They are, simply, far from being the ‘gold standard’ they are usually portrayed as: Contrary to frequent claims in the applied literature, randomization does not equalize everything other than the treatment in the treatment and control groups, it...