While I obviously support more polling on misinformation, I don’t agree with Ezra Klein’s suggestion that it’s the best way to determine whether there is “epistemic closure” among ideological or partisan groups:
The question is how do you measure epistemic closure?
The easy answer is you test for its product: Misinformation. What you’d want to do, I guess, is continuously poll a standard set of questions based on empirical facts. “Has GDP grown since President X’s inauguration?” “Have global temperatures been rising or falling in recent decades?” “Does the United States have longer life expectancy than other developed nations?” “Do a majority of Americans approve of the president’s job performance?” That sort of thing. Have representatives of both parties decide the questions and then see whether respondents from one party or the other get more questions right.
The problem is that misperceptions are not necessarily the result of a closed information loop. Someone with a relatively balanced media diet can still end up with false beliefs — it all depends on how they interpret the news that they receive (i.e., the extent to which they are willing to accept information that is inconsistent with their preferences).
A better approach would be to measure (a) to what extent ideological elites on the left and right are failing to engage with outside sources of information and (b) to what extent their adherents are consuming political news largely or entirely from like-minded elites.
For an example of (a), see Hargittai, Gallo, and Kane on the insularity of linking patterns among liberal and conservative blogs in 2004 and 2005, a paper which includes this table comparing linkage patterns (more negative values indicate more insularity):
For an example of (b), see this new Gentzkow and Shapiro paper on ideological segregation online, which includes a table comparing the audiences of the online sources visited by liberals and conservatives:
Update 4/27 9:04 AM: See also this Baum and Groeling paper which finds that “Daily Kos on the left and Free Republic and Fox News on the right demonstrate clear and strong preferences for news stories that benefit the party most closely associated with their own ideological orientations”:
Update 4/27 10:57 AM: I neglected to mention another relevant paper by Lawrence, Sides, and Farrell showing the ideological skew of readers of liberal and conservative blogs:
Henry Farrell, one of the authors of that piece, objects to part of my argument above at The Monkey Cage:
Julian Sanchez’s argument – which started this debate – seemed to me to be making a somewhat different point. While he focuses on the impact of an alternative sphere of media, his concern is with the consequences…
These consequences could plausibly manifest themselves either if conservatives (or liberals; or whoever) only consume conservative media or if they consume both conservative and non-conservative media, but tend to weight the arguments of the former much more heavily than the latter. And we simply cannot figure this out from data on media consumption patterns (or, for that matter, linkage patterns) alone…
Data on divergent patterns of media and information consumption is valuable in figuring out what people think. But people interested in this question aren’t so much worried about the actual patterns of consumption as about its putative consequences for political beliefs. So I think that first cut research to identify whether epistemic closure is a problem should focus on consequences, contra Brendan, looking at the extent to which individuals with different ideologies tend towards closure across a variety of politically salient issues. But it would be nice to see a second wave of research, extending the stuff that Brendan talks about to look at how variation in patterns of media consumption intersected with false political beliefs. And a third body of research could do some experimental work to figure out more precisely the underlying causal mechanisms…
I agree that the studies above don’t quite get at the core of Sanchez’s claim regarding the unwillingness of elites on the right to acknowledge outside sources of information as valid. Focusing on the putative consequences of closure, however, still strikes me as far too indirect. Instead, why not try to measure Sanchez’s claim directly? For instance, one could code the sources referenced in the National Review and Weekly Standard and compare them to those referenced in The Nation and American Prospect. If Sanchez is correct, the sources cited by conservatives in a non-disparaging way are more likely to be fellow travelers in the movement than those cited by liberals.
(Note: See the papers linked above for more details on their data and the definitions of the measures in the tables.)