Friday , December 13 2024
Home / Mike Norman Economics / Andrew Gelman — Yes, you can do statistical inference from nonrandom samples. Which is a good thing, considering that nonrandom samples are pretty much all we’ve got.

Andrew Gelman — Yes, you can do statistical inference from nonrandom samples. Which is a good thing, considering that nonrandom samples are pretty much all we’ve got.

Summary:
To put it another way: Sure, it’s fine to say that you “cannot reach external validity” from your sample alone. But in the meantime you still need to make decisions. We don’t throw away the entire polling industry just cos their response rates are below 10%; we work on doing better. Our samples are never perfect but we can make them closer to the population. Remember the Chestertonian principle that extreme skepticism is a form of credulity. Making assumptions is necessary. However, it is also necessary to recognize and acknowledge limitations. Formal modeling is never more accurate for the math than the assumptions permit.Reasoning is a tool of intelligence. It is not a magic wand. Taking reasoning for a magic wand because it is highly formalized is magical thinking.It is important to

Topics:
Mike Norman considers the following as important: ,

This could be interesting, too:

Joel Eissenberg writes Trusting statistics

Jeff Mosenkis (IPA) writes IPA’s weekly links

James Kwak writes COVID-19: The Statistics of Social Distancing

James Kwak writes COVID-19: The Statistics of Social Distancing

To put it another way: Sure, it’s fine to say that you “cannot reach external validity” from your sample alone. But in the meantime you still need to make decisions. We don’t throw away the entire polling industry just cos their response rates are below 10%; we work on doing better. Our samples are never perfect but we can make them closer to the population.
Remember the Chestertonian principle that extreme skepticism is a form of credulity.
Making assumptions is necessary. However, it is also necessary to recognize and acknowledge limitations. Formal modeling is never more accurate for the math than the assumptions permit.

Reasoning is a tool of intelligence. It is not a magic wand. Taking reasoning for a magic wand because it is highly formalized is magical thinking.

It is important to distinguish necessity from contingency. Necessity is based on logic necessity (tautology) and logical impossibility (contradiction). These are purely syntactical, that is, based on applying rules to signs. Logical necessity is probability one; contradiction is probability zero. All description is contingent on observation.

Statistics is a reasoning tool for dealing with contingency. The formal aspect of the tool does not vary, but its application is dependent on assumption and measurement. Thinking that the results will be the same owing to the invariant formal aspect is a mistake. Results can never be more precise than measurements or more accurate than assumptions permit, no matter how rigorous the formal methods applied.

Statistical Modeling, Causal Inference, and Social Science
Yes, you can do statistical inference from nonrandom samples. Which is a good thing, considering that nonrandom samples are pretty much all we’ve got.
Andrew Gelman | Professor of Statistics and Political Science and Director of the Applied Statistics Center, Columbia University

Mike Norman
Mike Norman is an economist and veteran trader whose career has spanned over 30 years on Wall Street. He is a former member and trader on the CME, NYMEX, COMEX and NYFE and he managed money for one of the largest hedge funds and ran a prop trading desk for Credit Suisse.

Leave a Reply

Your email address will not be published. Required fields are marked *