Tag Archives: Statistics & Econometrics

What statistics teachers get wrong! .[embedded content] This insightful video confirms what I always like to emphasize to my doctoral students: Statistics is no substitute for thinking. A non-trivial part of teaching statistics is made up of learning students to perform significance testing. A problem I have noticed repeatedly over the years, however, is that no matter how careful you try to be in explicating what the probabilities generated by these...

Biomedical, psychological, and social sciences are “soft” insofar as they focus on phenomena whose regularities are amorphous and situational (in contrast to the universal, exact laws which dominate physical sciences). In doing so, they must confront another major source of research uncertainty: Living organisms are characterized by natural variation and complex feedback within and across organisms, which introduces sampling variation or “noise” that we model as statistical...

The dangers of using pernicious fictions in statistics In much of science and medicine, the assumptions behind standard teaching, terminology, and interpretations of statistics are usually false, and hence the answers they provide to real-world questions are misleading … In light of this harsh reality, we should ask what meaning (if any) can we assign to the P-values, “statistical significance” declarations, “confidence” intervals, and posterior...

.[embedded content] For more on the pitfalls and intricacies of statistical tests and confidence intervals, have a look at this article by Sander Greenland et al.

What kind of ‘rigour’ do RCTs provide? The bad news is, first, that there is no reason in general to suppose that an ATE [Average Treatment Effect] observed in one population will hold in others. That is what the slogan widespread now in education and elsewhere registers: “Context matters”.  The issue in this paper is not though about when we can expect a study result to hold elsewhere but rather when we can have EBPP-style [Evidence-Based Policy and...

Is the p-value dead? .[embedded content] All science entails human judgement, and using statistical models doesn’t relieve us of that necessity. Working with misspecified models, the scientific value of significance testing is actually zero —  even though you’re making valid statistical inferences! Statistical models and concomitant significance tests are no substitutes for doing real science. In its standard form, a significance test is not the kind of...

It is instructive to consider cases in which most people readily accept causal claims in the absence of randomized experiments. Nowadays, few people doubt the effects of tobacco smoking on lung cancer. But in the 1950s, tobacco lobbyists embraced the idea that a genetic predisposition caused both a tendency to smoke and lung cancer … In other words, they claimed that there was an unblocked backdoor path. This idea was dispelled not by randomized, controlled experiments in...

My critique of RCTs Randomized Controlled Trials (RCTs), while useful in some contexts, are often overvalued in economics and social sciences. My critique centres on the following key points: 1. Lack of External Validity RCTs often suffer from problems of external validity, meaning that their results cannot easily be generalized beyond the specific experimental conditions. In the controlled environment of an RCT, many real-world factors are ignored or...

The history of econometrics There have been over four decades of econometric research on business cycles … But the significance of the formalization becomes more difficult to identify when it is assessed from the applied perspective … The wide conviction of the superiority of the methods of the science has converted the econometric community largely to a group of fundamentalist guards of mathematical rigour … So much so that the relevance of the research to...

As social scientists — and economists — we have to confront the all-important question of how to handle uncertainty and randomness. Should we define randomness with probability? If we do, we have to accept that to speak of randomness we also have to presuppose the existence of nomological probability machines, since probabilities cannot be spoken of — and actually, to be strict, do not at all exist — without specifying such system-contexts. Accepting Haavelmo’s domain of...