Brendan Nyhan

The coming experimental revolution

Back in 2004, I wrote a post titled “Politics goes Moneyball” about the increasing use of experimentation to measure the effectiveness of campaign tactics. Since then, progress — which has been led by Yale’s Alan Gerber and Donald Green — has been relatively slow but steady. Here’s the latest sign that people are finally catching on —
the founding of a new organization called the Analyst Institute that practices “Moneyball for progressive politics”:

A Job Posting from the Analyst Institute:

The Analyst Institute is hiring for several positions. We are looking for people who are analytical, quantitatively-minded, and comfortable with statistical analysis software. The more campaign experience the better, and the more experience with experimentation the better.

The Analyst Institute is a new organization that does cutting edge analytics and evaluation of voter registration, persuasion, and mobilization. We work in close collaboration with major progressive organizations around the country. Some people have described what we are trying to do as “Moneyball for progressive politics.”

The executive director, Todd Rogers, is finishing a PhD at Harvard and has worked with Gerber on a turnout experiment (PDF).

The larger question is why everyone in politics, business, nonprofits, and government isn’t constantly doing experiments to see what works and what doesn’t. Yale’s Ian Ayres recently asked this question about sports teams, which could use randomization to evaluate the effectiveness of various strategies (say, a zone defense versus man-to-man in basketball), but the principle applies far more broadly. Despite all the folk wisdom out there about what works or what doesn’t in politics or business, even the experts actually know very little — it is very difficult or impossible to determine causality from observational data. By contrast, experiments are simple, cheap, and easy to evaluate. Academics in the social sciences are doing more of them, especially outside the lab (i.e. “field experiments”), but the real revolution will happen when white collar professionals start to catch on.