of sound data collection plans, comparable and well-measured indicators, analytical rigor, and capacity into local evaluation systems. Yet identifying this type of contribution is difficult because IE is not the only determining factor. Examples of projects with this influence, such as the Southwest Poverty Reduction Project in China and the Life Enrichment Project in India, are illustrative. In the former case, the outcomes of the IE included a significant increase in the government’s poverty monitoring and analysis capability and an unusually good database for detailed project monitoring and evaluation. In the latter, the IE helped the client start to understand the benefits of such evaluation, because it was the first of its type on the topic of HIV/AIDS prevention and other development programs for migrant workers. There are at least two other cases where IEs contributed substantially to the development and institutionalization of results-oriented evaluation systems for policy making. One is the IE of Progresa/Oportunidades in Mexico; this was argued to be a major catalyst to the redesign and renewed focus on results of the M&E system of the Ministry of Social Development that housed the program (Franco and Fernandez-Ordonez 2010). The other example is the positive influence that a set of IE activities, including some supported by the World Bank, had on SINERGIA, Colombia’s national results-based management and evaluation system (Castro 2008).
Factors Affecting Use and Influence of Impact Evaluations The effectiveness of IEs for enhancing development practice depends on many factors inherent in their production that vary by how they are perceived. IEG has found many cases where IE use and influence depend on the active engagement of clients, the relevance of the evaluation, the timeliness of its results, and its perceived quality. Use also often depends on dissemination, the type of policy the evaluation is useful for, the people who commission the study, and the policy environment in which the IE is conducted. IEs are more likely to be used in development policy if they focus on the issues that are of interest to future users. The low usage rate of IEs for project assessment, as pointed out in the previous section, can be partly explained by the lack of alignment between the outcomes assessed in the IE and the project’s results framework. At the World Bank, 59 percent of IEs aligned with the results framework merited a substantive discussion in the ICR, compared with 20 percent of IEs not aligned. Similarly for IFC, around 80 percent of IEs were adequately mentioned in the PCR, compared with 17 percent of nonaligned IEs.22 There are some encouraging signs that World Bank IEs are starting to be more closely linked with operations. DIME is promoting a results-based model for the Bank’s operations, which includes collaborating with Bank operations to
106
World Bank Group Impact Evaluations: Relevance and Effectiveness