Meta-Analysis

Abstract

Do informational interventions shape electoral choices and thereby promote political accountability? The chapters in Part II of this book provided answers to this question in particular contexts. The studies individually provide rich insights not only into the impact of interventions that were common to all studies, but also on the effects of alternative interventions that were specific to each one. In this chapter, we assess the larger lessons that we can glean from our coordinated studies. As outlined in Chapter 3, all studies seek to test common hypotheses about the impact of harmonized informational interventions, using consistent measurements of outcome variables. Our preregistered analysis allows us to evaluate whether, pooling data from the set of studies in the initiative, information about politician performance led voters to alter their electoral behavior. It also informs a discussion about the conditions under which they did or did not do so. We find that the overall effect of information is quite precisely estimated and not statistically distinguishable from zero. The analysis shows modest impacts of information on voters' knowledge of the information provided to them. However, the interventions did not appear to shape voters’ evaluations of candidates, and, in particular, they did not discernibly influence vote choice. This slate of null results obtains in nearly all analyses for the individual country studies too. Nor is there strong evidence of impact on voter turnout, though under some specifications we find suggestive evidence that bad news may boost voter mobilization. Our results are robust to different analytic strategies and across a variety of modeling and dataset construction choices. The findings also suggest that the estimated effect in our missing study would have needed to be extremely large to alter our broader conclusions. The size of our meta-analysis reduces the chances that null estimated effects stem from low statistical power, and the fact that our results are so consistent across the individual studies limits the possibility that our mostly null effects are due to idiosyncrasies in implementation or study design. In the rest of this chapter, we first describe the prespecified approach that we use to analyze the pooled dataset. We then report our main findings, point out the consistency of results across studies, and report robustness checks. Next we consider several possible reasons for our null findings by testing the prespecified hypotheses. The most plausible reason for the null effects stems from the failure of the interventions to shape voters’ perceptions of politicians; we do not find evidence, however, that this is due to partisan or ethnic attachments or other heuristic substitutes for information. It is critical to underscore the similarities of these interventions to previous treatments in the experimental research literature and to interventions for which donor organizations in the transparency space routinely advocate. Indeed, our interventions were crafted by researchers with substantial country-specific expertise, usually in collaboration with local NGOs. Our null results across wide array of contexts therefore provide an important baseline of evidence against which future studies can be weighed. This chapter could be profitably read in conjunction with Chapter 3, which discusses the common interventions and our measurement of key variables, but it can be read as a standalone chapter as well.