Summary:
What kind of ‘rigour’ do RCTs provide? The bad news is, first, that there is no reason in general to suppose that an ATE [Average Treatment Effect] observed in one population will hold in others. That is what the slogan widespread now in education and elsewhere registers: “Context matters”. The issue in this paper is not though about when we can expect a study result to hold elsewhere but rather when we can have EBPP-style [Evidence-Based Policy and Practice] “rigorous” evidence about any of the kinds of claims needed in practice. Here too, the news is bad: There are no good explicit methods with detailed content for inferring either general causal claims or causal predictions about what will happen in a specific case that look anything like rigorous
Topics:
Lars Pålsson Syll considers the following as important: Statistics & Econometrics
This could be interesting, too:
What kind of ‘rigour’ do RCTs provide? The bad news is, first, that there is no reason in general to suppose that an ATE [Average Treatment Effect] observed in one population will hold in others. That is what the slogan widespread now in education and elsewhere registers: “Context matters”. The issue in this paper is not though about when we can expect a study result to hold elsewhere but rather when we can have EBPP-style [Evidence-Based Policy and Practice] “rigorous” evidence about any of the kinds of claims needed in practice. Here too, the news is bad: There are no good explicit methods with detailed content for inferring either general causal claims or causal predictions about what will happen in a specific case that look anything like rigorous
Topics:
Lars Pålsson Syll considers the following as important: Statistics & Econometrics
This could be interesting, too:
Lars Pålsson Syll writes Is the p-value dead?
Lars Pålsson Syll writes Alternatives to RCTs
Lars Pålsson Syll writes My critique of RCTs
Lars Pålsson Syll writes The history of econometrics