African Development Bank Evaluation Profile - Evaluation Systems Review 2016

II. AFRICAN DEVELOPMENT BANK GROUP

African Development Bank Group (AfDB) Independent Development Evaluation (IDEV) Evaluation Mandate The African Development Bank’s (AfDB) independent evaluations are managed by the Independent Evaluation Department (IDEV), whose mission is to enhance the development effectiveness of the AfDB through evaluations of strategies, policies, operations, processes and procedures. IDEV’s mandate covers: •

independent and instrumental evaluations

•

oversight of self-evaluation processes and products

•

proactive engagement in evaluation partnerships and knowledge-sharing activities.

IDEV’s core objectives are to: •

contribute to enhanced learning

•

provide a basis for accountability

•

promote an evaluation culture within the AfDB and regional member countries (IDEV 2016).

The current evaluation policy was approved in 2007 (OPEV 2007). In 2012 IDEV commissioned its first self-assessment, which highlighted how the department should consider reprioritising and rebalancing its evaluation products, while improving the quality and impact of its work. This self-assessment became a driver for the formulation of new policies; a revised policy introduced significant changes as compared to the 2007 policy and is under consideration by the AfDB Board of Directors. In parallel, an Independent Evaluation Strategy (2013-2017) was developed and implementation began in 2013 (IDEV 2013).

The strategy is being realised through the implementation of three-year rolling work programmes, the most recent of which was approved by the Board in December 2015 and covers the period 2016-2018. The following priorities were embedded in the programme: •

A shift in emphasis from stand-alone project evaluations to country and regional strategy evaluations, as well as thematic/sectoral and corporate evaluations.

•

A strong knowledge management, dissemination and outreach function to share the knowledge generated by evaluations and ensure that evaluation findings are used in operations, strategies and policies of the AfDB. The aim is to strengthen IDEV’s role as a knowledge broker to support the AfDB in managing its operations efficiently.

•

Strengthening evaluation systems, capacity and evaluation culture in the AfDB and within the region. Through this and work carried out to improve self-evaluation within the AfDB, IDEV will contribute to managing for development results (IDEV 2015).

Organisational Structure and Reporting Lines IDEV is a separate entity reporting directly to the AfDB’s Board of Directors through its Committee on Operations and Development Effectiveness (CODE). There is no direct involvement in operational and policy work, but the Evaluator-General attends senior management operations and policy committee meetings to keep close collaboration with management and feed lessons from evaluation as needed in the discussions. The threeyear rolling work plans are drafted in consultation with operations units and governing authorities (OECD 2010, IDEV 2016).

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

77

II. AFRICAN DEVELOPMENT BANK GROUP

Central/main evaluation units

Programme/operational units Reporting line

High level policy groups or ministries

Lines of communication

Evaluator-General's Office

Division 1: Infrastructure (transport, energy, water, ICT) Private Sector

Division 2: Agriculture Social Sectors Governance Corporate Country & Regional

Division 3: Knowledge Management, Evaluation Culture and Capacity

Cross divisional team brought together to work on a specific cross cutting issues, e.g. an evaluation on inclusive growth may include expertise on transport, private and social sectors

The Independent Evaluation Strategy in 2013 described the new internal organisational structure of IDEV, introducing three core divisions as opposed to the previous two divisions. Two divisions are in charge of specific evaluations, divided by thematic focus areas. The third division specifically addresses knowledge management, supporting self-evaluation and is leading activities for strengthening evaluation capacity in regional member countries (IDEV 2013).

Types of Evaluation IDEV adheres to international standards for the quality of its evaluation, in particular the OECD DAC Quality Standards for Development Evaluation and the ECG “Big Book on Good Practice Standards”.

• Project completion report and supervision report validations and validation

syntheses • Project cluster evaluations • Impact evaluations • Country/regional/sector-wide evaluations • Thematic/corporate evaluations • Evaluation syntheses

78

II. AFRICAN DEVELOPMENT BANK GROUP

Resources Snapshot of evaluation resources AfDB

IDEV has a total of 30 staff that includes support staff. Additionally, the unit has 11 long-term, fulltime consultants who contribute to evaluations, knowledge management and evaluation capacity development as the need arises.

The budget for centralised evaluations is independently managed and amounts to EUR 9.4 million for the year 2016, which is an increase from 2015.

Head / Director / Assistant Director

Professional evaluation staff

Administrative / Support staff

EUR 9 000 000 Over the previous work programme period 2% of the AfDB administrative budget (2013-2015), IDEV delivered 78 Project Completion Report (PCR) and Extended Supervision Report 12 (XSR) validations, 3 PCR/XSR validation syntheses, 5 project cluster evaluations, 25 project results Average evaluations produced per year (2015) assessments, 1 impact evaluation, 18 country strategy evaluations, 12 sector and thematic evaluations, 6 corporate evaluations, and 35 knowledge management and learning events (IDEV 2016, IDEV 2015).

Principles of Evaluation Independence The Evaluator-General, head of IDEV, reports directly to the AfDB’s Board of Directors through its Committee on Operations and Development Effectiveness (CODE). CODE maintains oversight of IDEV’s work, endorses and recommends the IDEV’s work programme and associated budget to the Board of Directors for approval. CODE also ensures that the department’s budget is ring-fenced and free from management influence. Through its operational setup, IDEV is independent from the AfDB’s operational and policy work (IDEV 2016).

Competence and capacity building While there is no specific AfDB policy on evaluation capacity development (ECD), the evaluation department is involved in ECD activities to support external evaluation associations and networks, including support for training activities. The 2013 reorganisation underlined a growing commitment to capacity development, especially supporting ECD in regional member countries. The strategy mentions the option of an evaluation accreditation scheme for new and current staff at IDEV. Internal skills constraints are also addressed through the provision of training and hiring of external consultants when necessary.

Transparency and participation The Independent Evaluation Strategy underlines the importance of transparency.

Formal management responses are provided by senior management and published along with evaluation reports.

The 2013 strategy outlined the plans for establishing a Management Action Reporting Mechanism, which is intended to ensure clarity over which recommendations management accepts, how it intends to respond to them, action deadlines, and whether these are met

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

79

II. AFRICAN DEVELOPMENT BANK GROUP

in practice. All evaluation reports on country, sectoral or thematic topics are discussed by CODE. Evaluation reports are published and results are communicated to AfDB staff and to external stakeholders in full and summary form and through feedback workshops (IDEV 2013, OECD 2010).

IDEV publishes all evaluation publications, including management responses to evaluations. The website provides public access to IDEV’s key information and publications.

The internal and external circulation of reports to development partners occurs systematically, and social media is increasingly being made use of. The unit focuses on creating specific outreach products to identified target audiences.

Knowledge management From the 2013 reorganisation of IDEV, a new focus on knowledge management has been outlined. IDEV put resources towards regional learning events, evaluation community of practice events, a wide dissemination of evaluation reports, and the publishing a quarterly knowledge publication. A new website for IDEV was also launched in 2013 and revamped in 2015, creating better access to knowledge products (IDEV 2015).

Co-ordination with donors and country recipients Collaboration with other donors is sought for country assistance evaluations, thematic or sector reviews, and in the areas of knowledge management and evaluation capacity development. Partner countries are consulted at the start-up of the evaluation, during the evaluation and at the submission of the reports.

Quality assurance To ensure the highest quality, IDEV has implemented since 2013 a dual system of review covering both design documents and final reports. The individual yearly work programme of each IDEV staff includes the peer review of two to three evaluations conducted by colleagues. In addition, the main reports produced for each evaluation are reviewed by at least one external expert reviewer using standardised review guidelines and templates.

IDEV has established a sitting expert panel for the comprehensive evaluation of the AfDB’s development results, which is the department’s major undertaking for 2014-2016.

For decentralised (self-) assessments and evaluations commissioned by AfDB management, quality assurance can be provided on request by the Quality Assurance and Results Department. The unit can provide advice, review Terms of Reference, review draft reports, etc., but there is no explicit mandate to provide support to assessments conducted by the operational management.

Note to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles” provides explanatory notes on the profiles.

80