emphasis on its own signature programs, even if there are robust IE findings that confirm their effectiveness (box 5.8). Similarly, IE users accustomed to the culture of evidence-based policy making are more likely to be receptive to IEs and use them to make policy decisions. For instance, the report “Making Results Count” on M&E for India (Fiszbein and Shah 2008) shows that in Implementation Status Reports for 11 projects, the sector manager made at least one comment related to data analysis in 3 cases (for example, “What did the data say?”) and on data quality in none of the 11 cases. The Country Management Unit made no comments in the Implementation Status Reports for these projects on (i) progress toward Project Development Objectives that made reference to actual data; (ii) implementation of M&E activities; (iii) data analysis; and (iv) data quality. This is consistent with reported uses of IE findings in India: Of the 8 completed World Bank IEs for which survey responses were available (of a total of 11 completed IEs), the use of IE findings to validate project impacts or inform the design of a follow-on or new project was reported for around one-third of IEs or less, even though the
Box 5.8
Political Context for Using Impact Evaluations: The Case of Atención a Crisis in Nicaragua
The CCT component of the Atención a Crisis pilot, modeled on an existing CCT in Nicaragua—the Red de Protección Social—was implemented for one year in 2005–06. The goal was to learn the impacts of a CCT scheme with some enhanced features, including skill promotion and productive investment. Hence, the pilot was accompanied by a rigorous IE program, initiated mainly by the World Bank, with some involvement of government counterparts, to inform the social protection strategies of the country. This IE effort resulted in a number of papers showing that the pilot had significant positive effects on consumption, utilization of health care, and early childhood cognitive outcomes and helped protect vulnerable households from the adverse effects of negative shocks. In addition, the program had significant social spillover effects and improved households’ attitudes about— and hence investments in—the future. The ICR stated that the IEs were being used extensively. However, the findings of these IEs actually were not used for policy making. This was mainly because of the change of government in 2008, when the Sandinista National Liberation Front came into power. This led to a massive change in public officials across ministries and agencies involved in Atención a Crisis. Despite the evaluation results, the new government associated social protection programs such as Atención a Crisis with the previous regime and thought of them as typical cash handouts that perpetuate passive aid receiving and populist agendas. In undermining the reputation of Atención a Crisis, the new government also argued that these programs increased domestic violence against women and failed to help empower them. As a result, they switched the focus of social policies away from CCT schemes to others types of interventions. Sources: IEG country case studies.
Use and Influence of World Bank Group Impact Evaluations
111