Tag Archives: Statistics & Econometrics

Deborah Mayo on statistical significance testing [embedded content] Although yours truly appreciate much of Mayo’s philosophical-statistical work, it is essential to remember that her qualified use of ‘severe testing’ actually is pretty far from​ the usual day to day practice of significance testing among applied social scientists today. If, however, statistics​ users stuck to Mayo’s ‘severe testing,’ there would be less reason to criticize the modern...

Big data — Poor science Almost everything we do these days leaves some kind of data trace in some computer system somewhere. When such data is aggregated into huge databases it is called “Big Data”. It is claimed social science will be transformed by the application of computer processing and Big Data. The argument is that social science has, historically, been “theory rich” and “data poor” and now we will be able to apply the methods of “real science” to...

Why statistical significance is worthless in science There are at least around 20 or so common misunderstandings and abuses of p-values and NHST [Null Hypothesis Significance Testing]. Most of them are related to the definition of p-value … Other misunderstandings are about the implications of statistical significance. Statistical significance does not mean substantive significance: just because an observation (or a more extreme observation) was unlikely...

The most dangerous equation in the world Failure to take sample size into account and inferring causality from outliers can lead to incorrect policy actions. For this reason, Howard Wainer refers to the formula for the standard​ deviation of the mean the “most dangerous equation​ in the world.” For example, in the 1990s the Gates Foundation and other nonprofits advocated breaking up schools based on evidence that the best schools were small. To see the...

Econometrics — analysis with incredible​ certitude​ There have been over four decades of econometric research on business cycles … But the significance of the formalization becomes more difficult to identify when it is assessed from the applied perspective … The wide conviction of the superiority of the methods of the science has converted the econometric community largely to a group of fundamentalist guards of mathematical rigour … So much so that the...

 [embedded content] Advertisements

What RCTs can and cannot tell us We seek to promote an approach to RCTs that is tentative in its claims and that avoids simplistic generalisations about causality and replaces these with more nuanced and grounded accounts that acknowledge uncertainty, plausibility and statistical probability … Whilst promoting the use of RCTs in education we also need to be acutely aware of their limitations … Whilst the strength of an RCT rests on strong internal validity,...

‘Controlling for’ — a methodological urban legend Trying to reduce the risk of having established only ‘spurious relations’ when dealing with observational data, statisticians and econometricians standardly add control variables. The hope is that one thereby will be able to make more reliable causal inferences. But — as Keynes showed already back in the 1930s when criticizing statistical-econometric applications of regression analysis — if you do not manage...

Scott Page’s new book is a great introduction on how to use and evaluate different kinds of mathematical models in the social sciences. Yours truly will be back soon for a lengthy review, but let me just notice that — as I have over and over again emphasized on this blog — Page underscores that if we want to explain social phenomena, relations and structures, we have to go beyond data. Data do not speak for themselves. Without theory and a search for mechanisms and deep...

Unbiased estimates? Forget it! In realistic settings, unbiased estimates simply don’t exist. In the real world we have nonrandom samples, measurement error, nonadditivity, nonlinearity, etc etc etc. So forget about it. We’re living in the real world … It’s my impression that many practitioners in applied econometrics and statistics think of their estimation choice kinda like this: 1. The unbiased estimate. It’s the safe choice, maybe a bit boring and maybe...