by Why quasi-experimental evaluations fail Evaluation research tends to be method-driven. Everything needs to be apportioned as an ‘input’ or ‘output’, so that the programme itself becomes a ‘variable’, and the chief research interest in it is to inspect the dosage in order to see that a good proper spoonful has been applied … The quasi-exprimental conception is again deficient. Communities clearly differ. They also have attributes that are not reducible to those of the individual members … A particular programme will only ‘work’ if the contextual conditions into which it is inserted are conducive to its operation, as it is implemented. Quasi-experimentation’s method of random allocation, or efforts to mimic it as closely as possible, represent an
Topics:
Lars Pålsson Syll considers the following as important: Statistics & Econometrics
This could be interesting, too:
Lars Pålsson Syll writes What statistics teachers get wrong!
Lars Pålsson Syll writes Statistical uncertainty
Lars Pålsson Syll writes The dangers of using pernicious fictions in statistics
Lars Pålsson Syll writes Interpreting confidence intervals
Why quasi-experimental evaluations fail
Evaluation research tends to be method-driven. Everything needs to be apportioned as an ‘input’ or ‘output’, so that the programme itself becomes a ‘variable’, and the chief research interest in it is to inspect the dosage in order to see that a good proper spoonful has been applied …
The quasi-exprimental conception is again deficient. Communities clearly differ. They also have attributes that are not reducible to those of the individual members … A particular programme will only ‘work’ if the contextual conditions into which it is inserted are conducive to its operation, as it is implemented. Quasi-experimentation’s method of random allocation, or efforts to mimic it as closely as possible, represent an endeavour to cancel out differences, to find out whether a programme will work without the added advantage of special conditions liable to enable it to do so. This is absurd. It is an effort to write out what is essential to a programme—social conditions favourable to its success. These are of critical importance to sensible evaluation, and the policy maker needs to know about them. Making no attempt to identify especially conducive conditions and in effect ensuring that the general, and therefore the unconducive, are fully written into the programme almost guarantees the mixed results we characteristically find.