Tag Archives: Statistics & Econometrics

Econometric modeling and inference The impossibility of proper specification is true generally in regression analyses across the social sciences, whether we are looking at the factors affecting occupational status, voting behavior, etc. The problem is that as implied by the three conditions for regression analyses to yield accurate, unbiased estimates, you need to investigate a phenomenon that has underlying mathematical regularities – and, moreover,...

Why quasi-experimental evaluations fail Evaluation research tends to be method-driven. Everything needs to be apportioned as an ‘input’ or ‘output’, so that the programme itself becomes a ‘variable’, and the chief research interest in it is to inspect the dosage in order to see that a good proper spoonful has been applied … The quasi-exprimental conception is again deficient. Communities clearly differ. They also have attributes that are not reducible to...

Design-based vs model-based inferences Following the introduction of the model-based inferential framework by Fisher and the introduction of the design-based inferential framework by Neyman [and Pearson], survey sampling statisticians began to identify their respective weaknesses. With regard to the model-based framework, sampling statisticians found that conditioning on all stratification and selection/recruitment variables, and allowing for their...

Are RCTs — really — the best way to establish causality? The best method is always the one that yields the most convincing and relevant answers in the context at hand. We all have our preferred methods that we think are underused. My own personal favorites are cross-tabulations and graphs that stay close to the data; the hard work lies in deciding what to put into them and how to process the data to learn something that we did not know before, or that...

Statistical assumptions and racial bias Our analysis indicates that existing empirical work in this area is producing a misleading portrait of evidence as to the severity of racial bias in police behavior. Replicating and extending the study of police behavior in New York in Fryer (2019), we show that the consequences of ignoring the selective process that generates police data are severe, leading analysts to dramatically underestimate or conceal entirely...

The poverty of fictional storytelling in statistics and econometrics  The most expedient population and data generation model to adopt is one in which the population is regarded as a realization of an infinite super population. This setup is the standard perspective in mathematical statistics, in which random variables are assumed to exist with fixed moments for an uncountable and unspecified universe of events … This perspective is tantamount to assuming a...

Decision making — trustworthiness vs relevance The random assignment plus masking are supposed to make it likely that the two groups have the same distribution of causal factors. It is controversial how confident these measures should make us that they do this. This issue bears on the trustworthiness of causal claims backed by RCTs. As we noted, trustworthiness is the central topic of many other guides. But we aim to move beyond that; we concentrate on...

‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right closures. Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. ‘It works there’ is no evidence for ‘it will work here.’ Causes deduced in an experimental setting still have to show that they come with a transportability warrant to the target population. The causal...

Modularity — a questionable assumption Modularity is the mark of a type of independence from context. The same functional relationship between variables will hold in a given component of the contributing mechanisms whether or not there is a change in a different component. The total effect may change when different components contribute, but the operation of the modular mechanism will not be changed nor change them. In situations where the presence or...

In defense of DAGs .[embedded content]