by Steve Levitt, in addition to gaining fame (at least at an economist level, not a Justin Bieber level) for writing Freakonomics, has made a career teasing cause and effect out of (largely) observational data. (By “observational data,” I mean that he doesn’t explicitly run controlled experiments in a lot of cases and just looks at the world as it transpired naturally instead.) Observational data presents an interesting challenge because people usually make choices in life rather than being guided by randomness. As a result, we often end up with selection bias that makes causal interpretations difficult- for example, we can look at people with and without pets and observe that the people who have pets are happier. (This is hypothetical, but it is in keeping with everything I would like to
Topics:
Jodi Beggs considers the following as important: Causality, cause and effect, Economics, happiness, instrumental variables, statistics, steve levitt
This could be interesting, too:
Lars Pålsson Syll writes The non-existence of economic laws
Lars Pålsson Syll writes Cutting-edge macroeconomics …
Lars Pålsson Syll writes Tourism — a critical perspective
Lars Pålsson Syll writes Eurofanatiker försöker köra över folket igen
Steve Levitt, in addition to gaining fame (at least at an economist level, not a Justin Bieber level) for writing Freakonomics, has made a career teasing cause and effect out of (largely) observational data. (By “observational data,” I mean that he doesn’t explicitly run controlled experiments in a lot of cases and just looks at the world as it transpired naturally instead.) Observational data presents an interesting challenge because people usually make choices in life rather than being guided by randomness. As a result, we often end up with selection bias that makes causal interpretations difficult- for example, we can look at people with and without pets and observe that the people who have pets are happier. (This is hypothetical, but it is in keeping with everything I would like to believe about the world.) This doesn’t mean that pets make people happier, since it could just be the case that people who are already happier also tend to adopt pets. It would be better from a data analysis standpoint if people were just randomly endowed with pets (like when a cat showed up on my doorstep when I was little I suppose), but unfortunately for science purposes we live in a society where people choose whether or not to have pets, and this choice aspect kind of messes things up.
To try to overcome this issue, economists tend to look really hard for sources of randomization- technically known as instrumental variables. In the pet example, whether a stray animal showed up on the doorstep might make a good instrumental variable, at least if the showing up was fairly random and people tended to keep the animals once they showed up. Using some statistical fanciness, we could compare the group of people who had animals show up versus those that didn’t and get a reasonable estimate of the causal effect of pets on happiness.
I know what you’re thinking- this is all nice in theory, but it’s not like we keep good records on stray animal on doorstep prevalence. This is true, and wouldn’t it be nice if we could actively create an instrumental variable- perhaps let a bunch of stray cats loose in a neighborhood and record what happens? (I was going to add a disclaimer to not try this, but it could actually be pretty interesting for research purposes.) How about if we could introduce a source of randomness in the easiest way possible, by flipping a coin?
Turns out that Levitt actually implemented the coin flip as instrumental variable to assess the causal effect of change on happiness:
Little is known about whether people make good choices when facing important decisions. This paper reports on a large-scale randomized field experiment in which research subjects having difficulty making a decision flipped a coin to help determine their choice. For important decisions (e.g. quitting a job or ending a relationship), those who make a change (regardless of the outcome of the coin toss) report being substantially happier two months and six months later. This correlation, however, need not reflect a causal impact. To assess causality, I use the outcome of a coin toss. Individuals who are told by the coin toss to make a change are much more likely to make a change and are happier six months later than those who were told by the coin to maintain the status quo. The results of this paper suggest that people may be excessively cautious when facing life-changing choices.
(Note that you should be able to access the article with most university email addresses, and even some alumni email addresses. Worth trying, at least.)
So let’s think this through…the outcome of a coin flip is random, so people were essentially randomized into “change” and “don’t change” groups. This randomization implies that the two groups are (at least approximately) comparable along other dimensions, leaving the change directive as the only systematic difference between the groups (and therefore the only plausible cause of any observed differences in outcomes). If everyone who was told by the coin to make a change actually did so (and vice versa), Levitt wouldn’t have even had to do anything statistically fancy and could have just compared the average levels of happiness of the two groups. Because not everyone listened to the coin (which I guess is sort of a good thing for the world more generally), he had to do the more fancy version of the math but is still able to find a statistically significant causal effect of change on happiness. Cool, huh? Now go make a change- apparently it’s good for you.