Tag Archives: Statistics & Econometrics

Slim by chocolate — a severe case of goofed p-hacking Frank randomly assigned the subjects to one of three diet groups. One group followed a low-carbohydrate diet. Another followed the same low-carb diet plus a daily 1.5 oz. bar of dark chocolate. And the rest, a control group, were instructed to make no changes to their current diet. They weighed themselves each morning for 21 days, and the study finished with a final round of questionnaires and blood...

Econometrics — science built on beliefs and untestable assumptions What is distinctive about structural models, in contrast to forecasting models, is that they are supposed to be – when successfully supported by observation – informative about the impact of interventions in the economy. As such, they carry causal content about the structure of the economy. Therefore, structural models do not model mere functional relations supported by correlations, their...

Top 10 critiques of econometrics •Achen, Christopher (1982). Interpreting and using regression. SAGE •Berk, Richard (2004). Regression Analysis: A Constructive Critique. SAGE •Freedman, David (1991). ‘Statistical Models and Shoe Leather’. Sociological Methodology •Kennedy, Peter (2002).  ‘Sinning in the Basement: What are the Rules? The Ten Commandments of Applied Econometrics’. Journal of Economic Surveys •Keynes, John Maynard (1939). ‘Professor...

Econometric inferences — idiosyncratic, unstable and inconsistent The impossibility of proper specification is true generally in regression analyses across the social sciences, whether we are looking at the factors affecting occupational status, voting behavior, etc. The problem is that as implied by the three conditions for regression analyses to yield accurate, unbiased estimates, you need to investigate a phenomenon that has underlying mathematical...

NHST — a case of statistical pseudoscience NHST is an incompatible amalgamation of the theories of Fisher and of Neyman and Pearson (Gigerenzer, 2004). Curiously, it is an amalgamation that is technically reassuring despite it being, philosophically, pseudoscience. More interestingly, the numerous critiques raised against it for the past 80 years have not only failed to debunk NHST from the researcher’s statistical toolbox, they have also failed to be...

Modern econometrics is fundamentally based on assuming — usually without any explicit justification — that we can gain causal knowledge by considering independent variables that may have an impact on the variation of a dependent variable. This is however, far from self-evident. Often the fundamental causes are constant forces that are not amenable to the kind of analysis econometrics supplies us with. As Stanley Lieberson has it in his modern classic Making It Count: One can...

Regression analysis and randomisation distract us from the real scientific issues  In my view, regression models are not a particularly good way of doing empirical work in the social sciences today, because the technique depends on knowledge that we do not have. Investigators who use the technique are not paying adequate attention to the connection – if any – between the models and the phenomena they are studying. Their conclusions may be valid for the...

A meta-analysis identifies a set of studies, each of which provides one or more estimates of the effect of some intervention. For example, one might be interested in the impact of job training programs on prisoner behavior after release. For some studies, the outcome of interest might be earnings; do inmates who participate in job training programs have higher earnings after release than those who do not? For other studies, the outcome might be the number of weeks employed...

The significance of insignificance (wonkish) Skewness, outliers, multiple hypothesis testing, and reliance on asymptotics often — as is well-known among most statisticians — give rise to spurious findings. But the problem may actually be even worse than we have thought. In an interesting new paper, Alwyn Young, after having looked at over 2000 regressions reported in American Economic Association journals, summarizes his findings: Armed with an idea and...

Econometrics and the rabbits principle In econometrics one often gets the feeling that many of its practitioners think of it as a kind of automatic inferential machine: input data and out comes causal knowledge. This is like pulling a rabbit from a hat. Great — but first you have to put the rabbit in the hat. And this is where assumptions come in to the picture. The assumption of imaginary ‘superpopulations’ is one of the many dubious assumptions used in...