Netherlands Evaluation Profile - Evaluation Systems Review 2016

II. NETHERLANDS

Netherlands Policy and Operations Evaluation Department (IOB), Ministry of Foreign Affairs (MFA) Evaluation Mandate The Policy and Operations Evaluation Department (IOB) is an independent unit responsible for evaluation activities for the Ministry of Foreign Affairs (MFA). Given the mandate and independence of IOB, the directorates general remain responsible for the evaluation of foreign policy, foreign trade and development co-operation. In addition to evaluations led by IOB, departments and embassies commission decentralised evaluations.

The Evaluation Policy (2009) and guidelines for evaluation define the objective of IOB as: “to increase insight into the implementation and effects of the Dutch foreign policy realised by the MFA and its operations. Better knowledge of the outcomes of development cooperation allows policy makers to devise measures that are more effective and focused.”

IOB’s scope of work includes: •

conducting evaluations for learning and accountability purposes

•

programming evaluations in the field of development co-operation, foreign trade and foreign policy

•

improving the quality of evaluations by developing quality requirements and evaluation guidelines, advising on evaluation design and implementation, and systematic assessment of evaluations

•

strengthening evaluation capacity in partner countries

• communicating evaluation results.

IOB conducts policy reviews, impact evaluations, process and other evaluations and studies. The policy reviews give an assessment of a specific policy area of the Dutch government with a comprehensive analysis of a general or operational policy objective.

Ministries are obliged to conduct these reviews every five to seven years. The policy reviews rely on information gathered in separate evaluations. Ex-post impact evaluations look into the net effects of the policy. IOB also carries out short evaluations, synthesis studies and ad-hoc research. Operations departments and embassies undertake evaluations of projects and programmes.

In line with policy developments in the Netherlands, the IOB’s work has shifted from evaluation of individual projects towards evaluation of foreign trade and foreign policy.

Responsibility and scope of activities In general, IOB conducts larger programme evaluations, while departments, embassies and implementing agencies are responsible for decentralised evaluation, such as the evaluation of individual projects, subsidies and instruments. IOB is responsible for managing the evaluations and the dissemination of the results. The IOB evaluators conduct individual evaluations and write policy reviews and synthesis reports. Part of the evaluation may be contracted to external consultants with specialised knowledge of the particular topic, particularly if the evaluation is commissioned jointly with partner institutions. Typically the IOB’s internal evaluators write the main reports, rather than externally recruited consultants. External consultants are often recruited for case studies. The Director of IOB is responsible for the contents of the final reports (including evaluations, reviews and studies).

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

167

II. NETHERLANDS

Organisational Structure and Reporting Lines IOB is placed under the authority of the Deputy Secretary-General of the MFA. The Head of IOB reports to the Deputy Secretary- General of the MFA. Central/main evaluation units

Programme/operational units Reporting line

High level policy groups or ministries

Lines of communication

Parliament

Minister of Foreign Affairs / Minister for Foreign Trade and Development Cooperation

Policy and Operations Evaluation Departement (IOB)

Externally Recruited Experts (internal evaluators write the main reports, rather than externally hired consultants)

The reporting line of the Head of IOB varies depending on the type of evaluation. In case of policy reviews, the report goes to the Parliament with a policy reaction from the Minister of Foreign Affairs and/or the Minister for Foreign Trade and Development Cooperation, after discussion by the Council of Ministers. In general, individual evaluations that will be part of a policy review will be published without a policy reaction from a Minister, unless it is agreed with one of the Ministers to send the report to Parliament. Studies may be published directly, though the timing and modality will be discussed with the management of the Ministry. The IOB Director approves the individual evaluation reports.

Until recently, an Advisory Panel advised on the usability of evaluations. The Panel consisted of the personnel with technical skills and experience of evaluation and development co-operation. Currently IOB is working on the development of a new panel or advisory board. This panel or board should include the three policy areas of Development Cooperation, Foreign Trade and Foreign Affairs.

Proposals for the Evaluation Programme are discussed inter alia with the Financial and Economic Affairs Department and the Directors-General. The Director of IOB, and subsequently the Directors-General for Foreign Policy and Foreign Trade provide the final approval. Approval of the Director-General for International Cooperation (DGIS) is not required, though IOB aims at coordinating the programming with the DGIS.

168

II. NETHERLANDS

Types of Evaluation • Thematic and cross-cutting evaluations • Sector-wide evaluations • Programme evaluations • Impact evaluations • Policy/strategy evaluations • Project/activity evaluations • Systematic reviews

IOB uses the OECD DAC criteria as well as other criteria such as additionality, coherence, and cross-cutting issues (gender).

Resources IOB has 26 researchers and evaluators working on a full-time basis.

A separate component of the annual budget is allocated to centralised evaluations. The budget for the evaluation programme is discussed in the Audit Committee and the management of the MFA. Decentralised evaluations conducted by departments and embassies are financed from the programme budgets. This is defined during the project design phase.

Snapshot of evaluation resources Netherlands

Head / Director / Assistant Director

Professional evaluation staff

Administrative / Support staff EUR 2 500 000 0.06% of the ODA budget 10-15

Principles of Evaluation Independence

Produced 10-15 evaluations per year

IOB is institutionally separated from the operations departments and embassies. The Head of IOB reports to the Deputy Secretary-General of the MFA. The Director of IOB is recruited externally with a seven-year fixed contract.

Competence and capacity building The skills of the evaluation staff are maintained by internal and external training.

When needed, consultants or researchers are hired.

IOB itself has a help desk function and it advises the staff outside the evaluation unit upon request. Support is typically provided on methodological issues, Terms of References (ToR), assessment of bids, participation in reference groups and feedback on draft reports.

No particular financial resources are allocated on delivery of training, but informal contacts between evaluators and policy officers are maintained.

Transparency and participation Most evaluations are made public. The only exceptions are specific requests from departments to analyse or assess specific internal topics.

EVALUATION SYSTEMS IN DEVELOPMENT CO-OPERATION: 2016 REVIEW © OECD 2016

169

II. NETHERLANDS

Knowledge management and use IOB is responsible for communicating the evaluation results and ensuring with the Policy Department that they are fed into policy. All the IOB’s evaluations, research and annual reports can be accessed on the website. IOB also disseminates the evaluation reports via other media, lectures, interviews, articles, conferences and seminars.

Co-ordination with donors and partner countries For the past five years, about nine joint evaluations have been carried out with bilateral agencies and partner governments, such as the European Commission Evaluation Unit, line ministries in Mozambique, Nigeria, Tanzania and Zambia and the University of Ghana.

Working with other donor countries is recognised as being useful, especially when more than one donor supports the same programme.

Quality assurance In order to ensure high quality reports, there is a quality control system in place with several instruments. Firstly, the IOB Director appoints internal peer reviewers to each evaluation. The reviewers provide advice during the entire process to the evaluator on evaluability and evaluation approach, the ToR, steps in the evaluation process as well as to the draft final report before it is submitted to external reference group. Secondly, an external reference group is established, consisting of internal staff and external experts, and meets at key moments during the process (e.g. to discuss the draft ToR, partial reports and the draft final report). Thirdly, the draft of the ToR is discussed within IOB. Fourthly, the evaluator him/herself conducts a self-evaluation of the evaluation in a systematic manner and checks its responsiveness against the Terms of Reference (MFA 2009). Finally, the IOB Director approves the report when all steps have been taken and if he/she agrees that the report complies with all quality standards.

Note to reader: The section at the beginning of Part II entitled “Introduction and key for the member profiles� provides explanatory notes on the profiles.

170