Independent Magazine - Issue n.4, 2022

THE IFAD EVALUATION ARCHITECTURE

Spotlight on quality enhancement processes, procedures and tools

I ndependent evaluation and self-evaluation are the two pillars of the IFAD evaluation architecture. IOE is responsible for the former, while Management oversees the latter. Before delving into the details of the different approaches quality enhanced processes (QE), it is paramount to start by recognizing that the entire evaluation function at IFAD follows internationally recognized good standards and practices.

Pursuant to the publishing of the 2022 edition of the IFAD Evaluation Manual [here], common quality standards now apply to both self- and independent evaluation. It is this first time in the history of the Fund that methodological guidance and standards for evaluations are applied across the whole organization. This achievement enhances consistency between the two evaluative functions, ultimately fostering a stronger results and evaluation culture at IFAD.

Both independent and self-evaluation functions work at project, country or regional, and corporate or thematic levels. Furthermore, the majority of evaluations are based on contribution analysis, which aims to measure interventions’ contribution to the overall change. To complement these analyses, impact assessments or evaluations, conducted by RIA and IOE, respectively, and corporate impact assessment reporting, conducted by RIA, are based on attribution analysis.

With these overarching quality standards and levels of analysis in place, differences nonetheless remain between the two pillars in terms of quality enhancement processes, procedures and tools. IOE is structurally, functionally and behaviourally independent of Management. The organizational and functional independence of IOE is essential for ensuring sound, credible and transparent evidence consistent with international norms and standards and the principles set forth in the Revised IFAD Evaluation Policy [here], including data privacy and confidentiality.

The range of evaluation products is tailored to the different needs and potential uses to support accountability and learning within IFAD. They are interlinked and the delivery of one product contributes to others. The composition and design of IOE products address three distinct priorities: (i) providing the necessary evidence for accountability at the project, country and corporate levels; (ii) contributing to learning by providing sufficient knowledge to IFAD Management and relevant stakeholders to strengthen organizational and development effectiveness; and (iii) strengthening the credibility and quality of self-assessments and independent evaluation products.

All evaluations undergo a thorough peer review process with the aim of enhancing the quality of the products. Different mechanisms of IOE peer review are needed for evaluations at corporate level, country level, and project level, as reflected throughout the four stages involved in all evaluations outlined below. Not all of the following steps are sequential, and some are overlapping or iterative; nonetheless, the broad stages provide the overall flow of tasks throughout the evaluation process.

In the context of stage four of its QE process, and in-line with its Multi-Year Evaluation Strategy [here], IOE is increasingly building evaluation dialogues to support evaluation capacity and use within and outside IFAD. These efforts are ensuring that IOE retains and deepens its position as an internationally recognized leader in the evaluation of rural development programmes, policies and strategies. Moreover, IOE is an active member of the United Nations Evaluation Group (UNEG) [here], the Evaluation Cooperation Group (ECG) [here], the Global Evaluation Initiative (GEI) [here], and the International Development Evaluation Association (IDEAS) [here].

In addition, our staff members engage prominently in multiple networks, associations and knowledge coalitions, including the International Research Group for Policy and Programme Evaluation (IntEval), the European Evaluation Society [here], the African Evaluation Association [here], the Asia Pacific Evaluation Association [here], the Wilton Park dialogue series [here], and the gLOCAL Evaluation Week [here].

With regard to the self-evaluation system, this is the responsibility of Management and is overseen by the Operational Policy and Results Division (OPR) and the Research and Impact Assessment Division (RIA). At the core of self-evaluation is the Development Effectiveness Framework (DEF). Introduced in 2016, the DEF was developed to ensure that evidence is collected from projects and systematically used and to create the necessary structure to facilitate the collection and use of evidence in decisions on project design and implementation. Self-evaluation projects are designed to achieve the expected results of the DEF – namely, to strengthen accountability, enhance learning and ultimately, ensure that the decision-making process is based entirely on reliable evidence. An updated version of the DEF will be introduced in 2022.

Self-evaluation products are developed at three main levels: country, project and corporate. At the country strategic opportunities programme (COSOP) level, self-evaluation begins at design, when the results framework for the country strategy is reviewed by IFAD’s Quality Assurance Group, OPR and other members of the Operational Strategy and Policy Guidance Committee, utilizing the development effectiveness matrix for COSOPs. At completion, COSOPs undergo a completion review – i.e. a self-evaluation of their strategic objectives and IFAD’s performance in achieving them.

At the project level, self-evaluation is fully integrated into the operation life cycle. At design, the development effectiveness matrix is used to review and enhance evaluability. During implementation, project teams prepare the annual supervision report, describing the progress made and identifying the main challenges encountered during execution. At midterm, project teams conduct a full review of the progress made and report it in the midterm review. At the end of the operation execution period, the relevant regional division prepares a project completion report (PCR). In addition to these common self-evaluation practices, which are applied to all projects, RIA conducts rigorous impact assessments of a representative sample of approximately 15 per cent of the projects closing in each replenishment period.

At the corporate level, the Report on IFAD’s Development Effectiveness presents the Fund’s annual operational and organizational performance by reporting on a set of 79 Results Management Framework (RMF) indicators agreed upon with Member States. IFAD also conducts thematic or cluster reviews on areas of specific interest to the Fund.

Looking ahead, IOE and Management will continue to maximise synergies and complementarities between independent and self-evaluations in two broad action pathways. First, collaboration between IOE and Management is pursued throughout the evaluation process, consistent with the independence of IOE. This includes the processes for selecting, planning and designing evaluation products, conducting evaluations and ensuring the identification and sharing of lessons learned and recommendations. Second, both IOE and Management work to ensure that the findings and lessons from each evaluation are communicated, absorbed and applied across institutions and shared with development partners and end clients in rural areas.