Sunday , December 22 2024
Home / Lars P. Syll / Texas sharpshooter fallacy

Texas sharpshooter fallacy

Summary:
Data clusters are everywhere, even in random data. Someone who looks for an explanation will inevitably find one, but a theory that fits a data cluster is not persuasive evidence. The found explanation needs to make sense, and it needs to be tested with uncontaminated data. Similarly, someone who fires enough bullets at enough targets is bound to hit​ one. A researcher who tests hundreds of theories is bound to find evidence supporting at least one. Such evidence is unconvincing unless the theory is sensible and confirmed with fresh data. When you hear that the data support a theory, don’t be persuaded until you’ve answered two questions. First, does the theory make sense? If it doesn’t, don’t be easily persuaded that nonsense is sensible. Second, is there​ a Texas

Topics:
Lars Pålsson Syll considers the following as important:

This could be interesting, too:

Lars Pålsson Syll writes The history of econometrics

Lars Pålsson Syll writes What statistics teachers get wrong!

Lars Pålsson Syll writes Statistical uncertainty

Lars Pålsson Syll writes The dangers of using pernicious fictions in statistics

Data clusters are everywhere, even in random data. Someone who looks for an explanation will inevitably find one, but a theory that fits a data cluster is not persuasive evidence. The found explanation needs to make sense, and it needs to be tested with uncontaminated data.

Texas sharpshooter fallacySimilarly, someone who fires enough bullets at enough targets is bound to hit​ one. A researcher who tests hundreds of theories is bound to find evidence supporting at least one. Such evidence is unconvincing unless the theory is sensible and confirmed with fresh data.

When you hear that the data support a theory, don’t be persuaded until you’ve answered two questions. First, does the theory make sense? If it doesn’t, don’t be easily persuaded that nonsense is sensible. Second, is there​ a Texas sharpshooter​ in the house? Did the person promoting the theory look at the data before coming up with​ the theory​? Were hundreds of theories tested before settling on the theory being promoted? If there is a smoking gun, withhold judgment on the theory until it has been tested with different data.

Advertisements
Lars Pålsson Syll
Professor at Malmö University. Primary research interest - the philosophy, history and methodology of economics.

Leave a Reply

Your email address will not be published. Required fields are marked *