Tag Archives: Statistics & Econometrics

Confusing statistics and research Coupled with downright incompetence in statistics, we often find the syndrome that I have come to call statisticism: the notion that computing is synonymous with doing research, the naïve faith that statistics is a complete or sufficient basis for scientific methodology, the superstition that statistical formulas exist for evaluating such things as the relative merits of different substantive theories or the “importance”...

The pitfalls of econometrics Ed Leamer’s Tantalus on the Road to Asymptopia is one of my favourite critiques of econometrics, and for the benefit of those who are not versed in the econometric jargon, this handy summary gives the gist of it in plain English: Most work in econometrics and regression analysis is made on the assumption that the researcher has a theoretical model that is ‘true.’ Based on this belief of having a correct specification for an...

Econometrics — the danger of calling your pet cat a dog Since econometrics doesn’t content itself with only making optimal predictions, but also aspires to explain things in terms of causes and effects, econometricians need loads of assumptions — most important of these are additivity and linearity. Important, simply because if they are not true, your model is invalid and descriptively incorrect.  And when the model is wrong — well, then it’s wrong. The...

Econometrics — junk science with no relevance whatsoever to real-world economics Do you believe that 10 to 20% of the decline in crime in the 1990s was caused by an increase in abortions in the 1970s? Or that the murder rate would have increased by 250% since 1974 if the United States had not built so many new prisons? Did you believe predictions that the welfare reform of the 1990s wouldforce 1,100,000 children into poverty? If you were misled by any of...

‘Goodness of fit’ is not what social science is about Which independent variables should be included in the equation? The goal is a “good fit” … How can a good fit be recognized? A popular measure for the satisfactoriness of a regression is the coefficient of determination, R2. If this number is large, it is said, the regression gives a good fit … Nothing about R2 supports these claims. This statistic is best regarded as characterizing the geometric shape...

The lack of positive results in econometrics For the sake of balancing the overly rosy picture of econometric achievements given in the usual econometrics textbooks today, it may be interesting to see how Trygve Haavelmo — with the completion (in 1958) of the twenty-fifth volume of Econometrica — assessed the role of econometrics in the advancement of economics.  We have found certain general principles which would seem to make good sense. Essentially,...

It’s not just p = 0.048 vs. p = 0.052 “[G]iven the realities of real-world research, it seems goofy to say that a result with, say, only a 4.8% probability of happening by chance is “significant,” while if the result had a 5.2% probability of happening by chance it is “not significant.” Uncertainty is a continuum, not a black-and-white difference” … My problem with the 0.048 vs. 0.052 thing is that it way, way, way understates the problem. Yes, there’s no...

When I present this argument … one or more scholars say, “But shouldn’t I control for everything I can in my regressions? If not, aren’t my coefficients biased due to excluded variables?” This argument is not as persuasive as it may seem initially. First of all, if what you are doing is misspecified already, then adding or excluding other variables has no tendency to make things consistently better or worse … The excluded variable argument only works if you are sure your...

A guide to econometrics 1. Thou shalt use common sense and economic theory. 2. Thou shalt ask the right question. 3. Thou shalt know the context. 4. Thou shalt inspect the data. 5. Thou shalt not worship complexity. 6. Thou shalt look long and hard at thy results. 7. Thou shalt beware the costs of data mining. 8. Thou shalt be willing to compromise. 9. Thou shalt not confuse statistical significance with substance. 10. Thou shalt confess in the presence of...

It is clearly the case that experienced modellers could easily come up with significantly different models based on the same set of data thus undermining claims to researcher-independent objectivity. This has been demonstrated empirically by Magnus and Morgan (1999) who conducted an experiment in which an apprentice had to try to replicate the analysis of a dataset that might have been carried out by three different experts (Leamer, Sims, and Hendry) following their published...