by The most dangerous equation in the world Failure to take sample size into account and inferring causality from outliers can lead to incorrect policy actions. For this reason, Howard Wainer refers to the formula for the standard deviation of the mean the “most dangerous equation in the world.” For example, in the 1990s the Gates Foundation and other nonprofits advocated breaking up schools based on evidence that the best schools were small. To see the flawed reasoning, imagine that schools come in two sizes — small schools with 100 students and large schools with 1600 students — and that students scores at both types of schools are drawn from the same distribution with a mean score of 100 and a standard deviation of 80. At small schools, the standard
Topics:
Lars Pålsson Syll considers the following as important: Statistics & Econometrics
This could be interesting, too:
Lars Pålsson Syll writes The ‘Just One More’ Paradox (student stuff)
Lars Pålsson Syll writes Monte Carlo simulation explained (student stuff)
Lars Pålsson Syll writes The importance of ‘causal spread’
Lars Pålsson Syll writes Applied econometrics — a messy business
The most dangerous equation in the world
Failure to take sample size into account and inferring causality from outliers can lead to incorrect policy actions. For this reason, Howard Wainer refers to the formula for the standard deviation of the mean the “most dangerous equation in the world.” For example, in the 1990s the Gates Foundation and other nonprofits advocated breaking up schools based on evidence that the best schools were small. To see the flawed reasoning, imagine that schools come in two sizes — small schools with 100 students and large schools with 1600 students — and that students scores at both types of schools are drawn from the same distribution with a mean score of 100 and a standard deviation of 80. At small schools, the standard deviation of the mean equals 8. At large schools, the standard deviation of the mean equals 2.
If we assign the label ‘high-performing’ to schools with means above 110 and the label ‘exceptional’ to schools with means above 120, then only small schools will meet either threshold. For the small schools, an average score of 110 is 1.25 standard deviations above the mean; such events occur about 10% of the time. A mean score of 120 is 2.5 standard deviations above the mean … When we do these same calculations for large schools, we find that the ‘high-performing’ threshold lies 5 standard deviations above the mean and the ‘exceptional’ threshold lies 10 standard deviations above the mean. Such events would, in practice, never occur. Thus, the fact that the very best schools are smaller is not evidence that smaller schools perform better.