Dr Indran A. Naidoo Director Independent Office of Evaluation
It gives me great pleasure to introduce the 9th edition of Independent Magazine. The feedback we have received on the magazine, which consolidates news from a suite of evaluation products and explicitly seeks to stimulate global debate on our profession, has been very encouraging. The magazine has accelerated in scale, reaching all countries across the world and bringing in the full spectrum of development practitioners, policy experts, civil society, academia and the media, who engage with this platform that allows access to the complete spectrum of IOE resources.
This edition brings another layer in the explanation of how evaluation works internationally. Of great importance are the words of the Secretary-General of the United Nations, Mr António Guterres, who has called for evaluation to inform the work of the UN.
Echoing these words, we have gathered insights on a set of questions at the heart of evaluation from the heads of three of the most influential networks that inform global practice. Dr Véronique Salze-Lozac’h, Chair of the Evaluation Cooperation Group (ECG) brings the perspective of the evaluation offices of the multilateral development banks and international financial institutions, while Mr Eddie Yee Woo Guo, Chair of the United Nations Evaluation Group (UNEG), represents the UN, and Dr Jörg Faust, Chair of the OECD/DAC network and Director of Deval, adds insights from a bilateral standpoint. IOE regularly interacts with these networks and benefits from the support and perspectives they bring.
In the context of these networks, in this edition we elaborate on IOE’s participation in international events such as the UNEG annual meeting, where we reported on our successes in the application of neuroscience, and reflected on the importance of combining theory of change and systems thinking in complex strategic and policy evaluations. We also shed light on IOE’s active engagement in the ECG spring meeting, in the first Evaluation Week of the European Bank for Reconstruction and Development (EBRD), in the ‘4th International Conference on Evaluating Environment and Development’, and in the ‘10th EvalMENA Annual Meeting’, as well as on ongoing discussions surrounding the Global Evaluation Initiative (GEI), of which we are funding members.
Profile
Discussions emanating from these events show that the global evaluation architecture is mature and committed to bringing the highest quality thinking and knowledge to what is today a profession that has come of age. We must approach our profession in a principle-oriented manner, keeping to our established norms and standards as consistency of approach provides us with credibility. At the same time, we need to be aware that results occur within climatic and geo-political contexts and this has to be factored into our assessment of development results.
The stability of IOE within IFAD, borne from a long history of practice, reflects the centrality of the evaluation function within the Fund and the strong support of both the Executive Board and Management. As readers who follow the Independent Magazine series have seen in previous editions, IOE seeks to give voice to a diversity of perspectives on evaluation results and to operationalize evaluation in real development contexts.
Organizational responsiveness continues to be demonstrated through IOE’s learning events, which allow deeper conversations about findings and, more importantly, give agency to management to explain their particular contexts of operation, and how they factor recommendations into decision-making. The recent review carried out by the Evaluation Advisory Panel, in 2023, noted the very high uptake on IOE’s recommendations by IFAD Management. This is in line with the words of appreciation of Board Members, and with the President’s Report on the Implementation Status of Evaluation Recommendations and Management Actions (PRISMA), as well as with the very positive affirmation by the recent MOPAN review.
I wish to take this opportunity to thank the outgoing Chair of the Evaluation Committee, H.E. Miguel Garcia Winder, Ambassador and Permanent Representative from Mexico to the UN Agencies Based in Rome, who steered the Committee successfully from September 2022. We appreciate his support to our mandate and work, and found his insightful comments an important part of our learning. He will be missed for his wit and candour.
I wish to welcome the new Chair of Evaluation Committee, H.E. Dr Yaya Olaniran, Permanent Representative from Nigeria to the UN Agencies Based in Rome, who we look forward to working with in the coming years. I also take the opportunity of wishing IOE Deputy Director, Mr Fabrizio Felloni, all the best in his new role at the Independent Evaluation Office of the Global Environmental Facility (GEF) in Washington, DC. His exemplary service and contributions to IFAD and to IOE are acknowledged with gratitude. He shall be missed.
I hope you find this issue enriching.
Dr Indran A. Naidoo Director
Independent Office of Evaluation of IFAD
INDEPENDENT OFFICE OF
IOE PILLARS
MISSION
Our mission is to promote accountability and learning through independent, credible and useful evaluations of IFAD's work.
GUIDING PRINCIPLES
Accountability, learning, independence and partnership are the principles that guide our work
HISTORY
The IFAD’s evaluation function has evolved over more than 40 years of existence, from being discharged by a unit internal to the IFAD management into a fully-fledged independent evaluation outfit.
QUALITY ASSURANCE
We have supported two external peer reviews of IFAD’s evaluation function conducted by independent professionals in the field of evaluation (2009 and 2019).
Fabrizio Felloni Acting Director | 2020-2021
Director Luciano Lavizzari Kees Tuinenburg Officer in Charge| 2013-2014 Ashwani Muthoo Acting Director | 2012-2013
Indran A. Naidoo Director
Contribute to forging IFAD’s corporate culture as a transparent, learningoriented and accountable organization.
Improve evaluation coverage and promote transformative evaluations that reflect the scale and scope of IFAD operations.
Engage with Management, Member States and external partners to support evaluation capacity and use within and outside IFAD. BUILD EVALUATION DIALOGUES
Retain and deepen IOE’s position as an internationally recognized leader in the evaluation of rural development programmes, policies and strategies.
EVALUATION OF IFAD (IOE)
IOE EVALUATION PRODUCTS
[click on icons to access database]
Aggregation level
Sub-regional Evaluations
Primary objective
Assess strategy, common intervention approaches and IFAD organizational set up in a set of countries that share salient characteristics.
Main users
Regional and country director(s), technical advisors, operational staff, and government counterparts.
Aggregation level
Project Completion Report
Validations
Primary objective
Validate the project completion reports prepared by IFAD Management.
Main users
IOE and IFAD Management for reporting (ARIE and RIDE) and feedback.
Evaluation Synthesis Thematic Evaluations
Primary objective
Contribute to knowledge generation by consolidating findings from past evaluations.
Main users
Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.
Primary objective
Provide evidence of development effectiveness, performance and results of operations in a thematic topic.
Main users
Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.
Corporate Level Evaluations
Primary objective
Assess the organizational performance and institutional effectiveness of IFAD.
Main users
Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.
Aggregation level
Annual Report of Independent Evaluation
Primary objective
Report all of IOE’s evaluation activities in a given year, and presents a synthesis of IFAD’s performance, lessons and challenges.
Main users
Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.
Project Performance Evaluation
Primary objective
Assess the performance and results of projectlevel operations funded by IFAD.
Main users
Regional and country director(s), technical advisors, operational staff, and government counterparts.
Guarantees the avoidance of conflicts of interest by upholding the provisions that safeguard the behavioral, organizational and structural independence of the Office.
Impact Evaluation
Primary objective
Provide a rigorous quantitative assessment of the impact on rural poverty of selected IFAD’s operations.
Main users
Regional and country director(s), technical advisors, operational staff, and government counterparts.
Project Cluster Evaluations
Primary objective
Assess the experience of several projects that have a common theme or common major component.
Main users
Regional and country director(s), technical advisors, operational staff, and government counterparts.
LEARNING & ACCOUNTABILITY
Reflects IFAD’s increasing focus on embracing a culture of evidence-based management to maximize development effectiveness, in which evaluation has a critical role to play
Clarifies that accountability and learning are objectives of evaluation, and emphasizes the effective use of evaluation products to this end
Aggregation level
Country Strategy and Programme Evaluation
Primary objective
Assess performance and results of country strategy and operations and provide lessons and recommendations to guide preparation of next country strategy
Main users
Divisional and country director, country team, and government.
EVALUATION COMMITTEE
Presents the roles and responsibilities of the Evaluation Committee and, of its Chair, in the evaluation function on behalf of the Executive Board
Editorial Board
Proofreading
Revisioning
Independent Magazine brings to the forefront of the global development dialogue the major efforts undertaken by the Independent Office of Evaluation of IFAD, while seeking to advance the organization’s vision of vibrant, inclusive and sustainable rural economies, where people live free from poverty and hunger. To present the richness of rural life, and detail facets of local community lifestyle, Independent Magazine also zooms in on cultural activities and landmark occurrences in countries featured by IOE’s evaluations.
From evaluation phobia to evaluation
and Türkiye to engage on policy matters
Operational paradigm of fragility in West African Sahel
Access to credit leads to higher agriculture yields
IFAD strategy contributes to resilience in Burundi
Principle-based evaluation needed for new world order
Enhancing international partnerships for evaluation
Evaluations need to analyse the machinery of an organization
from evaluation must take place at country level
New KM toolbox reveals power of informal conversations
FROM EVALUATION PHOBIA TO EVALUATION EMBRACING:
STORY OF A MINDSET SHIFT
Recognizing the traditional tension between evaluators and evaluands, steeped in stereotypes, IOE has put in place a drive to perfect its communication craft, including by horning-in neuroscience-based principles. The result has been a mindset shift among stakeholders and evaluands alike. Co-creation and enhanced engagement are now a daily reality in the work of IOE, which maintains its full independence whilst thriving on milestone publications. The roots of these results run deep and date back to the beginning of 2021, when IOE embarked on its ‘revolutionary’ journey.
This approach has been one of the keys in the rise of appreciation towards IOE’s work, which has ensured a rapid surge in uptake on evaluation recommendations, as confirmed by the recent MOPAN Review. This trend echoes the words of the UN Secretary-General, António Guterres, who emphasized the importance of using high quality evaluative evidence to improve efforts on the ground in his message to the UNEG EvalWeek 2024.
The UNEG EvalWeek 2024 took place from 29 January to 2 February 2024, in Málaga. Hosted by the World Intellectual Property Organization (WIPO) and the United Nations Office on Drugs and Crime (UNODC), in collaboration with CIFAL Málaga. On 30 January, Dr Naidoo and Dr Alexander Voccia, Coordinator of the Evaluation Communication Unit, delivered a joint presentation in the seminar titled ‘What does it
“Evaluation is critical to ensure that our work is making an impact on the ground and a difference in the lives of people. […]. We need credible, high-quality evaluation to cover more programme areas and ensure that evidence, results, transparency, and continuous learning and improvement guide our work, every step of the way.”
- António Guterres
United Nations Secretary-General
“We need to move from a culture of evaluation phobia to one of evaluation embracing. To do so, we must create psychologically safe environments where people feel free to express themselves, and in which we are empathetic listeners.”
- Dr Indran A. Naidoo Director Independent Office of Evaluation IFAD
“IOE’s communication offerings present users with an immersive experience, spanning the full spectrum of the evaluation function. Opportunities for real, tangible change are fast emerging, as the suite of products facilitate greater uptake of IOE’s work”.
- Dr Alexander Voccia Senior Evaluation Communication & KM Specialist Coordinator, Evaluation Communication Unit IOE, IFAD
Profile
Profile
take to build a credible, independent evaluation function?’, which was chaired by Judit Jankovic, Senior Evaluation Specialist at the International Criminal Court.
Drawing from his over thirty-year long career, Dr Naidoo explained that there tends to be an assumption that evaluators can affect change and enhance programme quality. Unfortunately, that would appear to not be always the case. Cognizant of this, IOE’s new approach builds on a dynamic and engaging process of triangulation that is based on independence, credibility and utility. This has resulted in enhanced opportunities for dialogue and mutual learning throughout the evaluation process.
the production of a pioneering online training, and a communication module within the IFAD Evaluation Manual, in addition to a video series knows as the ‘evaluation pills’, among other outputs. This undertaking is bearing notable fruits, as IOE has refined its interactions with Management, whilst continuing to engage in global dialogues and networks.
Dr Voccia explained that a large suite of innovative communication products provides enhanced visibility to these efforts and achievements. These include a state-of-the art fully independent website, and a first-of-a-kind online magazine, which have already reached over a hounded thousand users across all countries in the world, since their launch.
“Evidence, on its own, does not drive change. As a matter of fact, 70 per cent of initiatives that attempt to trigger change fail. Instead, organizations that develop initiatives geared towards changing mindsets are two times more likely to succeed in achieving the progress desired. We must be empathetic listeners. People must look forward to an evaluator coming, because we all share the common goal of making programmes better. Disagreement is also fine, it’s part of a healthy process of interaction”, clarified the IOE Director.
To spearhead this effort, IOE adopted a ground-breaking neuroscience-based approach to its communication efforts which resulted in
In addition to contributing experiences vis-àvis its state-of-the-art approach to communication, during EvalWeek, IOE also stimulated thought-provoking conversations on some of the core methodological issues pertaining to evaluations. On 30 January, Fabrizio Felloni, IOE Deputy Director, Dr Kouessi Maximin Kodjo, Lead Evaluation Officer, and Massiel Jiménez, Evaluation Research Analyst, hosted a session titled ‘The crisis of linear thinking - Combining theory of change and systems thinking in complex strategic and policy evaluations’, in which they also delivered a joint presentation.
Mr Felloni, Dr Kodjo and Ms Jiménez explained
Profile
that a theory of change (ToC) does not necessarily have to be linear, despite the linear thinking that it normally entails. A linear ToC risks over-simplifying reality. As a result, it is inadequate for humanitarian programmes, due to the dynamic and unpredictable natures of crises, as well as for complex development programmes.
unpack complexity.
Mr Felloni provided further points of reflection in his presentation during the session titled ‘Evaluating Policy Influence’. In particular, the IOE Deputy Director used the example of IFAD’s support to the Central America Strategy for Territorial
These considerations notwithstanding, a ToC has the merit of helping to understand intended results, causal links and main assumptions. Therefore, ToCs should not be abandoned. Instead, they should be enriched by incorporating additional frameworks, such as a system thinking approach that can help better capture and
Development, approved in 2010 by the Summit of the Central America Integration System. In his presentation, Mr Felloni showed several steps of policy development that can be evaluated with limited resources through desk reviews, virtual interviews and brief in-country work. Instead, an assessment of socio-economic effects of a
policy typically requires primary data collection, including surveys.
Following the closing of the sessions, Dr Naidoo and Mr Felloni attended the UNEG annual general meeting. During the meeting, the UNEG working groups provided an updated on past and ongoing activities.
UNEG is an interagency professional network bringing together the evaluation units of the UN system, including UN departments, specialized agencies, funds and programmes, and affiliated organizations. UNEG’s current operational strategies are outlined in the UNEG Principles of Working Together 2022 and UNEG Strategy 2020-2024.
IFAD and the strategic
There is potential for IFAD and the Government of Türkiye to further leverage their strategic partnership in order to promote engagement on policy matters and effective knowledge management for greater scaling up of results favourable to smallholder farming. Dr Kouessi Maximin Kodjo, Lead Evaluation Officer at the Independent Office of Evaluation of IFAD (IOE) presented this recommendation during the national workshop organized to discuss the Türkiye Country Strategy and Programme
Dr Kodjo & Ms Jiménez joint presentation
Dr Naidoo & Dr Voccia joint presentation
the Government of Türkiye to leverage partnership to foster engagement on policy matters
Evaluation (CSPE) undertaken in 2023. The event took place in Ankara, on 16 April 2024. Dr Kodjo explained that it is necessary that IFAD identifies the right entry points to engage in policy debates aligned with the country context of well-developed commercial agriculture, and noted that key strategic partners at the central and provincial levels should widen the space for IFAD to do this.
“IFAD should strengthen the country programme knowledge management framework for improved performance in
generating relevant knowledge and lessons, with the active involvement of government stakeholders. Organizing debates and discussions at strategic and operational levels on knowledge will be critical for the identification of options for scaling up positive results, as well as their incorporation in policy and strategic decisions”, highlighted IOE’s Lead Evaluation Officer.
This recommendation was put forth in response to the fact that the CSPE found no evidence of concrete policy
results or changes due to IFAD-supported operations over the evaluated period. In addition, there was limited evidence of the extent to which the effort to deliver knowledge products through documentation and dissemination of best practices of IFAD-supported projects was converted into lessons learned and used for informed decision-making.
“Together, we need to identify concrete entry points and opportunities for even better policy engagement going forward. We also need to ac -
tively convert the knowledge gained through ongoing investments into lessons used for informed decision-making,” stated Dr Donal Brown, Associate Vice-President of IFAD’s Programme Management Department.
Co-organized by the Government of Türkiye and IOE, in collaboration with IFAD’s Near East, North Africa and Europe (NEN) Division, the online virtual workshop brought together a wealth of high-level attendees, including Dr Ahmet BAĞCI Deputy Minister, Ministry in charge of Agriculture, and Mr. Osman YILDIZ, Director General of Agricultural Reform. Over 60 participants joined the event, representing the Government of Türkiye from the Ministry of Agriculture and Forestry; the Ministry of Treasury and Finance; and the Ministry of Environment,
Urbanization and Climate Change. A host of international development partners also attended, along with civil society organizations, private sector partners and representatives of UN organizations and agencies. Dr Brown and Dina Saleh, Regional Director of NEN, represented the Fund’s Senior Management. Dr Indran A. Naidoo, IOE Director, represented IOE alongside Dr Kodjo.
The CSPE covered the 2016 IFAD country strategic opportunities programme and four projects implemented between 2015 and 2022. The total estimated cost of the projects covered by the CSPE amounts to US$233.2 million, including US$136.6 million financed by IFAD, and US$96.6 million from the Government of Türkiye, domestic bank co-financiers, and beneficiaries.
During the workshop, participants discussed the main findings, recommendations and issues emerging from the CSPE. The meeting also provided an opportunity to focus on the opportunities and challenges of the partnership between IFAD and the Government in the near future, and discuss strategic priorities for IFAD’s upcoming programme.
“Overall, the programme supported by IFAD contributed to enhancing the resilience of smallholders’ social livelihoods and ecosystems, in remote mountainous areas, thanks to interventions that aimed to increase the productivity and production of crops and livestock, diversify the income sources, and strengthen sustainable management of natural production assets”, affirmed Dr Naidoo.
The IFAD-supported programme contributed to increased agricultural productivity and production in both crop and animal production systems, as well as more resilient agricultural ecosystems in upland areas. Support for economic diversification and livestock production helped improve smallholder incomes. IFAD’s efforts also contributed to improving the resilience of beneficiary households to climate shocks, by strengthening their absorptive and adaptive capacities and enhancing sustainable natural resource management. Numerous technologies, practices, and pro -
cesses were introduced and promoted by the programme, which were new to the projects’ beneficiaries. These include improved fodder crops, shepherd shelters, juice extractors, dairy cattle milking machines and seed drillers. With regard to scaling-up of results, evaluative evidence suggests several positive achievements through governmental institutions at the provincial level.
These successes notwithstanding, areas for improvement remain. Value chain development activities for processing and marketing of agricultural products, and
for access of poor farmers to markets led to modest results, because numerous actions reached better-off farmers, and effective partnerships with strong private actors for the access to markets of smallholder farmers have not yet occurred. Furthermore, the programme made little effort to strengthen grassroots organizations and enable them to take on responsibilities for the management of rehabilitated rangelands. This was because the development of social bonding and bridging capital was not embedded explicitly within the programme strategy, and this gap can be attributable to the lack of a re-
silience framework.
Looking ahead, the CSPE recommends further prioritizing the resilience of rural livelihoods in the mountain areas of Türkiye in an integrated manner, by deploying innovative approaches that build on the existing country potentials in value-chain segments. It also suggests improving the inclusiveness of the country programme towards poor and vulnerable rural women, as well as young men and young women, and strengthening the programmatic approach in the delivery of IFAD’s support, and fostering the learning culture. Türkiye has a population of 84.78 million people, and currently hosts an estimated
number of 3.5 million refugees as of February 2023. It is an upper-middle-income country and has the 19th largest economy in the world. As of 2022, it has the largest agricultural economy in Europe, according to OECD data. The country faces agricultural challenges, among which are land fragmentation, a lack of contemporary technologies and decision-making tools for efficient cropping patterns in remote areas, and low animal feeding levels resulting in non-optimal productivity due to expensive feed inputs.
Dr Indran A. Naidoo opening statement
Dr Indran A. Naidoo closing statement
Dr Kouessi Maximin Kodjo presentation
Türkiye CSPE infographic
Cappadox Festival: Where art and music meet in Cappadocia*
Now in its fifth edition, the Cappadox Festival consistently brings new perspectives and energy to Cappadocia. In a region renowned for its boutique cave hotels, magnificent scenery, balloon rides, underground rock cities, rustic-chic pottery and outdoor experiences, contemporary art, gastronomy and famous-name concerts are still somewhat rare.
Known for its surreal landscapes and historical significance, Cappadocia provides a magical setting that enhances the festival’s eclectic offerings. This year’s festival embraced the theme of “Changing Skies” and featured music, contemporary art, gastronomy and various outdoor activities.
The program will welcome visitors until June 13. Named after Erdemci’s 2001 exhibition at Karşı Sanat titled “Regrets, Reveries, and Changing Skies,” Cappadox calls upon participants to collectively remember the shared experiences formed gradually and the encounters made possible by the multilayered historical, cultural and physical geography of Cappadocia.
Furthermore, Cappadox’s music program offeres participants a magical and unforgettable experience that touched all senses, blending the unique beauty of nature with captivating performances.
IFAD to elevate knowledge on rural transformation to global level
IFAD should ascertain its role in elevating the operational knowledge it generates to a global level in order to inform the rural transformation debate. The forthcoming corporate-level evaluation (CLE) on knowledge management (KM) carried out by the Independent Office of Evaluation of IFAD (IOE) recommends that the Fund take tangible steps in this direction, in response to the challenges that the report uncovered. These shortcomings and their respective remedial actions were addressed during a learning seminar on 15 May 2024.
“Knowledge will be key for IFAD to raise its relevance as a development player. Knowledge is vital matter to ensure organizations exist at the forefront of rural transformation. IFAD can play a major role in translating knowledge from operations and contributing to the global understanding of how rural transformation can happen”, affirmed Dr Indran A. Naidoo, Director of IOE.
The seminar discussed how the IFAD has leveraged its knowledge base to underpin rural transformation, in particular at country level, and reviewed to what extent its existing strategies, structures
and functions have supported its ambition to contribute to transformative change in partner countries. The period 2016 to mid-2023, which the evaluation focused on, saw major strategic developments and organizational reforms. The extent of these changes, and their disjointed nature, made it challenging for IFAD to discharge a forceful organization-wide KM initiative in line with the ambitions of its 2019 KM strategy.
“KM is definitively central to IFAD and to IFAD13’s business model. Knowledge generation, dissemination and use allows IFAD to adaptively manage its operations portfolio as well as respond to implementation challenges and steer project
performance. Secondly, the generation and use of knowledge is also central to drive policy engagement and ensure long-term sustainability of IFAD’s interventions on the ground. Thirdly, KM is the basis for collecting, curating and applying lessons learned to enhance future development impact”, said Dr Donal Brown, Associate Vice President, Programme Management Department.
Organized by IOE, the online virtual seminar brought together members of IFAD’s Senior Management, including Dr Brown and Dr Jyotsna Puri, Associate Vice President, Strategy and Knowledge Department. Over 100 participants joined the event, from many IFAD departments across headquarters and field-based duty stations. The seminar included presentations, and afforded IFAD Management an opportunity to present their response and follow-up actions to the evaluation’s recommendations. The seminar also featured a live Q&A session, which drew inspiration from KM practices that worked in regional divisions. These included knowledge packs in East and Southern Africa, knowledge clinics in Asia and the Pacific, sub-regional knowledge sharing in
Dr Johanna Pennarz Lead author, CLE report on KM
Profile
West and Central Africa, and integrating South-South and Triangular Cooperation and KM in Latin America. Zenda Ofir, IOE Senior Advisor, provided comments and further insights.
Discussions highlighted that KM practices were often effective in their contribution to country programmes. Clear KM frameworks and strategies aligned with needs on the ground, and country strategic opportunities programme priorities enabled success. Moreover, evidence from the country case studies demonstrates that IFAD has the potential and knowledge to deliver highly transformative KM practices at country level, which can support rural transformation, and which in some cases has been delivered.
“The CLE case studies found positive cases where KM practices have shown results in countries. But overall, they require more support. IFAD’s internal structures and mechanisms have not been conducive to effective and efficient KM practices throughout the organization. The absence of evidence on KM effectiveness was a particular concern for this evaluation, and this was related to insufficient KM ca-
pacity and senior guidance”, explained Dr Johanna Pennarz, IOE Lead Evaluation Officer, and lead author of the CLE report.
Unfortunately, there is no consistent body of experiences being leveraged in any country to build the foundational capacity for KM during design or implementation. Approaches are typically one-off or ad hoc, and they are to a large extent related to individual capacity and motivation. CLE findings presented during the seminar point to the fact that IFAD’s institutional set-up for KM is not adequate to position itself as a knowledge player on rural transformation within the global landscape. The formal KM architecture remains centralized and does not support the organization-wide nature of KM and the diversity of KM roles in IFAD.
Dr
Puri AVP, SKD, IFAD
“I really want to acknowledge the challenges in implementing the KM Strategy 2019 which, as the report says, in its overall framing and assumptions is very strong. However, it is very important to understand the extent to which we have the financial and staff resources for translating this strategy into practice. Within the CLE findings, I really want
to acknowledge the fact that organizational reforms within IFAD have challenged the implementation of effective, organization-wide KM initiatives”, noted Dr Puri.
The 2019 KM strategy has been a state-of-the art document at the time it has been draft. During the following period, the organisational changes and ongoing decentralization reforms have clearly limited its effectiveness as a corporate document. Limitations include unclear KM roles across organizational levels, lack of attention to indigenous knowledge, and assumptions on resourcing and incentives at decentralized levels that did not hold. The assumption that there would be sufficient human and financial resources for KM did not hold. High workloads, understaffed country offices, vacant positions and the knowledge drain resulting from reassignment and turnover of staff undermine sustained KM initiatives. In the same vein, the evaluation noted the absence of systematic plans for capturing and transferring tacit knowledge before staff departure in many countries.
The KM architecture defined in the strategy was lagging behind the unfolding of IFAD’s decentralization process, which has seen the Fund making substantive progress in enhancing its field presence since 2019.
There is no consistent approach for budgeting on KM and the cost-effectiveness of KM practices and products is not monitored. IFAD’s
Dr Donal Brown AVP, PMD, IFAD
Jyotsna
knowledge products are biased towards research and are available only in few languages, limiting their utility for country stakeholders. Technical knowledge makes up the largest share of IFAD’s knowledge products, while experiential and policy knowledge from country programmes remains minimal. The country case studies only found very few cases where the systematic generation and sharing of knowledge has enabled effective policy engagement.
In the same context, while IFAD collaborates in some networks, it does not fully leverage platforms discussing contemporary KM concepts like local knowledge and decolonization. The Fund has also has not sufficiently addressed demands for knowledge from the Global South.
Seminar participants discussed recommendations that the CLE put forth. Firstly, IFAD Management should initiate a decisive shift away from its centralized KM architecture and allocate human and financial resources across decentralized levels. Secondly, IFAD Management should monitor KM effectiveness, and focus on KM practices that provide the best value for money at global and operational levels. Thirdly, IFAD’s next Strategic Framework should define how knowledge would enable IFAD to position itself as driver of rural transformation.
Value chain approaches poor
“It is possible to reach poor and very poor households and groups through value chain approaches. However, this requires attention and consistent focus, including in terms of the entry barriers for poorer producers ”, explained Fabrizio Felloni, Deputy Director of the Independent Office of Evaluation of IFAD (IOE), in a presentation during the first Evaluation Week of the European Bank for Reconstruction and Development (EBRD), on 11 March 2024.
Dr Johanna Pennarz presentation
approaches can be used to reach poor and very poor households
The Evaluation Week took place at the EBRD headquarters, in London, from the 11th to the 15th of March 2024. Organized by EBRD’s Independent Evaluation Department, the event brought together high-level experts from evaluation, results management and research centres to share knowledge on the importance of collaboration among multilateral development banks (MDBs), including on key topics such as resilience, climate
change and agribusiness. Participants shed light on the role of independent evaluation and results management in supporting internal as well as cross-institutional collaboration for greater impact on the ground. Dr Indran A. Naidoo, IOE Director, and Mr Felloni represented the office throughout the week.
The proceedings took place under the theme ‘collaboration within and across international financial institutions’, which focused on how evaluation can contribute to the pledge made by the MDBs at the 2023 Annual Meeting of the Bretton Woods institutions ‘strengthen collaboration for greater impact’.
Mr Felloni delivered his presentation on day one of the Evaluation Week, in the context of the session titled ‘Pro Poor Value Chains and Agribusiness’, which included over 90 participants in presence and online. The Deputy Director drew from the findings of the IOE corporate level evaluation of IFAD’s engagement in pro-poor value chain development. While the evaluation was carried out in 2019, its findings remain very relevant.
“ The work being presented was carried out some time ago, but its impact continues to permeate the organization. In this regard, it is important to always keep in mind that an evaluation which may be initially contested, will likely get used and affect an organization to a great extent over time ”, said Dr Naidoo.
Mr Felloni highlighted that value chain development requires long-term engagement and multiple-phase support. Project designs should systematically assess the degree of preparedness for value chain support, considering the local context and previous experience of the government and other partners. Based on this, project designs should focus priorities and approaches for value chain strengthening. Equally important is to promote an inclusive value chain governance and an inclusive policy and regulatory environment, by establishing or strengthening multi-stakeholder platforms and interprofessional associations that provide small-scale producers and other value chain stakeholders. Furthermore, strong partnerships are
necessary to enhance market intelligence throughout the project cycle. This can be achieved by collaborating with organizations that have strong value chain expertise to ensure that projects are based on thorough analysis of commodity market structure, demand and supply, price level and volatility, and barriers facing small-scale producers.
The Evaluation Week also included two EvalTalks. The first was titled ‘Bigger Better Bolder: unleashing collaboration across MDBs for greater impact’, and drew on the experience of the MDBs’ impact results management and independent evaluation. The second was titled ‘Scaling up climate ambitions: opportunities and challenges of MDBs cooperation’, and featured panellists from EBRD, the African Development Bank, the Center for Global Development, and the Multilateral Organisation Performance Assessment Network (MOPAN).
Furthermore, on 14 March, the agenda featured a training on ‘assessing effects of investment and policy interventions’, by the
Independent Evaluation Group of the World Bank, and an impact seminar on ‘shaping resilient systems in a complex world’, by the EBRD impact team, with panellists from the World Bank Group and the Stockholm Resilience Center.
Sandwiched in between this rich agenda, the ECG spring meeting unfolded on the 12th and 13th of March. The four key themes of the session were: evaluation of private sector operations; advances of MDBs on their path to becoming key green financiers; methodology of evaluation synthesis; and evaluating capital increase and corporate strategies of parent institutions.
Fabrizio Felloni presentation
CLE on Pro-poor Value Chains
FEATURE STORY
ENHANCING PARTNERSHIPS EVALUATION
ENHANCING INTERNATIONAL PARTNERSHIPS FOR STRONGER
“No man is an island” affirmed John Donne in his famous poem, which explores the interconnectedness of humanity. No individual is isolated, but rather an integral part of the broader human collective. Well-aware of the importance of working together to achieve a brighter future for humankind, the United Nations established Sustainable Development Goal 17 as ‘partnerships for the Goals’. IOE has taken this principle to heart, and has enshrined the concept of partnerships within its multi-year evaluation strategy 2022-2027. The strategy calls for IOE to step up partnerships for evaluation knowledge and cooperation in evaluation capacity development, with a view to contributing to an evaluation culture within and outside IFAD, and re-establishing IOE as an internationally recognized leader in the evaluation of rural development.
Pursuant to these goals, IOE has formal membership of three global professional evaluation networks comprising the United Nations and international financial institutions. These are the UN Evaluation Group (UNEG), the Evaluation Cooperation Group (ECG), and the Global Evaluation Initiative (GEI). In addition, IOE contributes to the work of the International Research Group for Policy and Program Evaluation (INTEVAL), of which it hosted the 38th annual meeting in 2023, and engages in multiple regional networks, including the European Evaluation Society, the African Evaluation Association, the American Evaluation Association, and the Asia-Pacific Evaluation Association, among others.
Through these committed collaborative efforts, IOE is continuously enhancing collaboration with other organizations, think tanks and universities to promote innovative approaches, share good practices, boost the professionalization and growth of the evaluation function, and raise the bar of methodological quality and rigour.
In this spirit, Independent Magazine reached out to Dr Véronique Salze-Lozac’h, ECG Chair and Chief Evaluator at the European Bank for Reconstruction and Development; Eddie Yee Woo Guo, UNEG Chair and Director of the Inspection and Evaluation Division at the UN Office of Internal Oversight Services; and Dr Jörg Faust, Chair of the OECD/DAC evaluation network, and Director of the German Institute for Development Evaluation. Véronique, Eddie and Jörg provided invaluable insights and perspectives into the value, importance, opportunities, challenges and future scenarios of international collaboration for evaluation through joint partnerships and networks.
Good afternoon, esteemed colleagues.
Good afternoon, Alexander
What do you see as being the role of the network that you are currently chairing and of other partnerships in contributing to an enhanced global evaluation profession, based upon on commonly agreed standards, norms, principles and guidance?
Véronique
One of the first roles of evaluation networks, such as the ECG, EvalNet and UNEG is to promote and uphold the quality, credibility and usefulness of evaluation in the UN system and the multilateral development banks,
Dr Véronique Salze-Lozac’h, ECG Chair and Chief Evaluator, EBRD
for the benefit of the countries and clients whom we serve. To this end, firstly, networks help to develop and update common standards, norms, principles and guidance for evaluation practice and management, such as the UNEG Norms and Standards for Evaluation or the ECG Good Practice Standards. These help us to speak the same language. These norms and principles are not meant to be strictly defined as standards, but rather offer a solid reference framework with good practices.
“Networks contribute to an enhanced global evaluation profession that is based on commonly agreed standards, principles and norms, and that can deliver relevant, credible and useful evaluations for accountability, learning and decision-making.”
Secondly, the ECG fosters learning and dissemination of knowledge based on evidence from evaluations, including through events, webinars, publications and communities of practice. Primarily among members, but also to a wider audience. We learn a lot from each other’s and constantly challenge our practices to improve and adapt. It is also a good way to advocate for the value and the benefits of evaluation. I really believe in cross-fertilization and in the advocacy role of these networks. Knowing that we are backed by a network gives added credibility to and assurance of each independent evaluation department within its organization.
- Dr Véronique Salze-Lozac’h
Thirdly, it helps support professional evaluation capacity development of evaluators and evaluation units, through joint training, mentoring, secondments, and other mechanisms. This is an aspect that we need to work and collaborate on more, insofar as it can provide important opportunities for growth of the evaluative profession.
Fourthly, we use the network very much for peer reviewing our reports so that there is cross-fertilization. Finally, we can have coordinated evaluations on topics of mutual interest or concern, such as the SDGs, humanitarian response, gender equality or country-level evaluations.
By doing these activities, the networks contribute to an enhanced global evaluation profession that is based on commonly agreed standards, principles and norms, and that can deliver relevant, credible and useful evaluations for accountability, learning and decision-making.
Eddie
UNEG’s raison d’être is to support the strengthening and harmonization of evaluation practices within the United Nations system, with the ultimate goal of strengthening the work of the United Nations system in service to the peoples of the world. Towards this end, the establishment of UNEG norms and standards for evaluation in the UN system in 2006 was a pivotal moment in the evolution of UN evaluation; it set forth clear foundational principles and guidelines for the establishment, practice and conduct of evaluation for all UN evaluators throughout the UN system. Enlightened UN managers, UN evaluators and evaluation managers have taken the norms and standards and run well with it, as evidenced by the growth of UNEG from 43 to 61 entities since 2004, and the norms and standards have been updated in 2016.
While the UNEG agenda is strictly to promote evaluation within the UN system, the reality is that the UNEG norms and standards, along with a significant amount of normative guidance that it has developed and issued, are in effect global goods that evaluators all over the world can draw upon. I believe ECG occupies a similar, perhaps larger, space in this regard as well, and there are opportunities for greater exchange and collaboration between the two networks.
Aside from promoting standards in the practice and professionalization of evaluation, UNEG has a particularly important role in carrying forward global norms and conventions into evaluation practice, to serve as a constant reminder of, and advocate for, the international commitments that have been made as part of the United Nations; in terms of integrating and promoting global themes like human rights, gender equality and empowerment, environmental stewardship, and leaving no one behind (including disability inclusion) – important aspects that, as an organization dedicated to global sustainable development goals, has to be built into the manner in which all activities and outcomes must be evaluated, so as to ensure their due consideration.
Jörg
The core functions of evaluation consist in the creation of practice-relevant knowledge, in the strengthening of accountability and in fostering learning. In our policy field, evaluation has been well-institutionalized and has provided an important value-added to policy making along these functions. Even if evaluators sometimes rightfully grumble about an occasional lack of attention by policy makers to evidence provided by evaluation, development cooperation belongs to the few policy fields where systematic evaluation is firmly established and used.
In this regard, the work of international evaluation networks such as those of UN development organizations, international finance organizations and those of member countries from the OECD Development Assistance Committees (DAC) has been of indispensable value for strengthening the above-mentioned evaluation functions. More concretely, the work of these networks has consisted in developing standards and criteria across international boundaries, in creating peer exchange among evaluation units and in preparing and conducting joint evaluative work. Altogether, those networks have played a very important role in “lobbying” for evidence-based policy making: a contribution that is of great importance in a world that is ideologically increasingly polarized and where credible and reliable evidence is threatened by fake news and the manipulation of information. At the same time, however, there are serious challenges, which evaluators in general but particularly those networks need to address in the future.
Generally speaking, it is of great importance for making evaluations comparable and for creating a common ground for fruitful exchange that evaluators have a common understanding about the quality of an evaluation. In this regard, I consider the basic principles of evaluation formulated by the United Nations Evaluation Group as key: independence, credibility and utility. If we add the Do-No-Harm principle and the principle of partnership, which are both of particular relevance in the field of development cooperation, evaluators have a great set of guiding principles that are useful for elaborating more detailed standards and quality guidelines that result in improved quality of our work. Even if there is yet no overarching formal agreement on these five principles, they nevertheless are implicitly used by many if not most development organizations as guiding principles of their evaluative work.
Another example of successful norm-setting by international networks are of course the six evaluation criteria developed by the OECD-Evalnet. Measuring the success of development against those criteria – relevance, effectiveness, efficiency, impact, sustainability, coherence - has been a big achievement: for analysing development measures from different angles and for making evaluations comparable across boundaries, while at the same time not overloading them with normative content. Interestingly, a recent OECD recommendation proposes that the organization’s member countries use these criteria in all other policy fields. In my opinion this is a great example of norm diffusion. And instead of some critique raised against the criteria, they are nevertheless being used by an increasing number of evaluators and organizations outside the OECD, something that shows their universal nature and potential of becoming a global public good.
Eddie Yee Woo Guo, UNEG Chair and Director, Inspection
What is the added value of evaluation groups, networks and partnerships among multilateral agencies in helping to advance methodological thinking, exchange best practices and carry out joint work?
Véronique
“We
cannot continue to evaluate like historians, coming a year or two after the fact to tell decision-makers and people what they may be able to observe and learn directly through monitoring and social media.”
The starting point is very much linked to the adoption of a common language and mutual understanding of criteria and standards. Moreover, these networks help foster a common understanding of evaluation concepts, methods and standards across different contexts and sectors, and promote a culture of evidence-based decision-making. They can enhance the quality, credibility and usefulness of evaluations by ensuring coherence, complementarity and alignment of evaluation frameworks, criteria and indicators, and by reducing duplication and gaps in evaluation coverage.
- Eddie Yee Woo Guo
Obviously, these are ongoing dialogues, which are not set in stone. For example, there is an ECG working group that has helped develop a common understanding on the definitions and use of rapid assessments and real-time evaluations, which were recently introduced in our toolset. Similarly, OECD-DAC recently added ‘coherence’ within its list of criteria after a long discussion among all network participants.
These networks also have the role of facilitating the sharing of good practices, lessons learned and innovative approaches to address complex and emerging development challenges, such as climate change, inequality and fragility. When we develop new ideas within our institutions we share them with others, and in turn adopt each other’s good practices if they fit our own needs. We see this a lot. For example, during the COVID-19 crisis, one international financial institution (IFI) decided to carry out a real-time assessment of the support of its institution to address the crisis. Thereafter, other IFIs enquired about how this was going to be done. In the end, several of us delivered a real-time assessment or a rapid assessment.
Equally important is the ability of these networks to increase the influence and impact of evaluations by disseminating and communicating findings and recommendations more widely and effectively, and by engaging with relevant stakeholders and partners at global, regional and national levels. These efforts give more credibility and power to reports. In practical terms, at EBRD we always include findings and lessons from evaluations of other IFIs within our own reports and we know that our Board members are very interested in this knowledge.
In light of these benefits, more could be done in terms of leveraging the comparative advantages and synergies of different agencies, and mobilizing resources and expertise to conduct joint or coordinated evaluations on topics of strategic importance or mutual interest.
Eddie
For UNEG, the fragmentation of the United Nations system, while perhaps a necessary built-in redundancy to ensure survival and resilience of the organization since its founding, nevertheless constitutes a significant constraint on its overall effectiveness, efficiency and overall cohesion. Each UN agency in pursuit of its respective mandate can, and do, end up creating systemic level inefficiencies, overlaps, duplications and at times, cross-purposed actions. The efforts of the Secretary-General to bring the organization together as one at the country level through the Resident Coordinator system is laudable and necessary, but constantly challenged by the recalcitrance of agencies with differing mandates and governance.
Networks like UNEG, and similarly purposed professional groups and actors within the organization help to build bridges and foster more cohesive, strategic, joint-up actions in the system, at the same time providing key development actors, national governments, and the designated coordinators or coordinating bodies an efficient means of communicating and engaging with disparate evaluation actors. Also, whether at global, regional or country level, the need for more systemic approaches is recognized and networks can support collaborative and joint approaches to evaluating outcomes and impacts beyond the narrow focus of single agency programmes and projects. Networking allows for pooling of resources; opportunity to crowd source information and experience; identify new resources; introduction to people and institutions; collaborative work for collective good, as well as raise awareness and disseminate good practices.
Jörg
In the case of the Evalnet, there are many examples of how this network has provided these benefits to its members and partners. Regarding methods, there has been continuous exchange on topics such as rigorous impact evaluations or contribution analysis. Perhaps even more important, exchange and peer learning has taken place regarding challenging topics in development cooperation such as blended finance, interventions in fragile or conflict-affected settings, climate mitigation and adaptation or gender and human rights. Often, such work consisted in creating temporary working groups, which developed common lessons or orientations for the above-mentioned topics, which hereafter had a positive impact on the quality of evaluations and recommendations. Moreover, the network also provided transparency for members but also for the global development and evaluation communities by providing condensed information about the evaluation systems of DAC members, international financial organizations and UN development agencies. The network’s regular report on evaluation systems is an important element of transparency and peer learning. Finally, Evalnet like other evaluation networks, provides an important informal space for mutual exchange among evaluators and evaluation managers.
Beyond peer learning and methodological as well as thematic work, Evalnet has also worked as a catalyst for joint or well-coordinated evaluations in the past. While I personally think that there have been far too few joint evaluations in our field during the last decade, without Evalnet there would have been even less. A recent example of such joint work that has been initiated by the network has been the Covid-19 Global Evaluation Coalition, that encompassed not only bilateral and multilateral development actors but also many countries from the Global South. The work of the coalition not only helped to decrease transaction costs and to increase insights in evaluation units’ endeavours across the globe to provide policy-relevant evidence on how to cope with a global pandemic, but also provided different perspectives, including those
Many countries have incorporated the SDGs into their own development strategies, and need support to track progress. In the face of the current permacrisis, how important is it to step up partnerships for evaluation capacity development for achievement of broad national development objectives, social inclusiveness and equity, including by influencing policy-making and operational work?
In the face of the current global challenges, which pose unprecedented challenges and risks to human development and well-being, fostering partnerships and collaboration, building capacity, and helping to create new networks at the country level is more important than ever. It is paramount to step up partnerships for evaluation capacity development, to ensure that countries have the necessary knowledge and capacities to adapt and respond effectively, and to advance towards broad national
development objectives, social inclusiveness and equity. Evaluation can also contribute to building resilience and preparedness for future shocks and crises, by identifying and addressing the root causes and drivers of vulnerability and fragility.
“Our networks can play an important role to shield evaluation from attempts to reduce its independence, a threat that should not be underestimated in a world where democratic accountability and transparency is under serious attack.”
Dr Jörg Faust
Furthermore, let us not forget that evaluation capacity development is not only about technical skills and tools, but also about creating an enabling environment and a culture of evaluation that values and uses evidence for decision-making and action. This requires strong leadership and ownership at the national level, as well as effective partnerships and coordination among different stakeholders, including government, civil society, academia, the private sector, donors and multilateral agencies. Inter-agency collaboration and coordination can enhance evaluation capacity development by providing more coherent and harmonized support to countries, and by leveraging the comparative advantages and synergies of different agencies. For example, inter-agency evaluations can generate more comprehensive and credible insights on cross-cutting issues, and can increase the influence and impact of evaluation findings and recommendations.
However, multilateral development banks (MDBs) and IFIs must realize that while they are important drivers of these processes, they are not the key actors. Countries are the key actors, given that it is at the national level where everything happens. For this reason, more attention must be put into the development of evaluation capacity at country level. This includes NGOs, civil society, national evaluation societies and the private sector. These stakeholders must be empowered and have ownership of evaluation tools.
For example, the Asian evaluation week, that the independent evaluation department of the Asian Development Bank organizes every year, has been promoting evaluation at the country level. This is very visible in the case of the Democratic People’s Republic of China, where bringing evaluators from many different networks together has helped national organizations to learn, to adopt good practices and be motivated to carry out evaluative work. Showing the benefits of evaluation at the country level, for both the public and non-public sector, is something that we need to do more.
Eddie
Interesting question, but if I may, I would like to separate the question of response to permacrisis from that of national evaluation capacity development. The latter is a need in and of itself, even if there is no ‘permacrisis’.
When it comes to national evaluation capacity development (NECD) in support of SDGs, with very limited resources available, sharing of training resources is efficient and much needed. New technology allows for larger scale training events that can benefit more evaluators. While the evaluation community can and should play a significant role in providing methodological guidance and supporting actual training with their expertise, I actually believe that NECD is primarily the responsibility of our programme colleagues, and it should be part of national development programmes; that is, an integral part of the countries’ overall SDG strategy. Non-national evaluators have roles as advocates, demonstrators, resource persons and even as trainers. But the job of designing a developmental programme that effectively informs and provides an evidence-base in support of decision-making to achieve the SDGs has to be that of the national governments and the development agencies and partners that seek to support them. Funding for evaluation and NECD should be included in, and streamed through, national institutional development components of the
enterprise.
With regard to the ‘permacrisis’, I agree this is a remarkably fraught time of multiple, and complex, crises. I remain optimistic and do not think them anymore permanent than the last World War. The World Bank seminar on the Future of Evaluation touched on this well. The current multiple crises, and the advent of technological advances, require us to change the way we do evaluation, and the times do call for evaluators to take on a more activist role, in order to be more timely, useful and relevant. We cannot continue to evaluate like historians, coming a year or two after the fact to tell decision-makers and people what they may be able to observe and learn directly through monitoring and social media. More and more, evaluation information has to be closer to real time in order to be of real use. AI, while one of the potential sources of the so-called permacrisis, also opens up opportunities for evaluation; to mine, recover and synthesize old data for real new use, as well as opportunities to speed up and expand our data-collection methods. AI and IT innovations create opportunities for evaluators to engage in new ways with stakeholders and programme beneficiaries. The important thing here is for networks to do this in a systemic manner. At UNEG, we are appreciative of UNDP for taking the lead in the development of the AIDA initiative, and with UNICEF, on the Global Evaluation Synthesis initiative.
Jörg
Strengthening evaluation capacity development in a globalized world is key. For too long, the evaluation of development cooperation has been mainly directed towards those who finance and implement projects and programmes and too little towards those who experience the impact of our policy field. Moreover, many countries in Africa, Asia and Latin America are also keen on evaluating their own policies and programmes. Partnering with them on an eye-to-eye-level in this endeavour by strengthening evaluation capacities in the respective countries is therefore an important measure to support accountability, bureaucratic quality and evidence-based policy-making beyond the narrow scope of development cooperation. Obviously, evaluation networks can play an important role in fostering such partnerships and global ECD initiatives as the Global Evaluation Initiative (GEI) and the National Evaluation Conferences (NEC).
Looking ahead, what do you see as being the biggest challenges and opportunities for renewed and enhanced inter-agency collaboration and coordination, including in light of the ever-growing proliferation of agency-specific mandates?
Véronique
We need to work more together, we need to collaborate, we need to coordinate. There is a call for our organizations, including our independent evaluation departments, to work more as a system to ensure that we can compare our findings, and build evaluative evidence together. There is also a need for healthy stimulation among each other’s organizations, to push each other to deliver better results and more effective outcomes.
This need for collaboration was the topic of a panel discussion during the first EBRD Evaluation Week, in March 2024. The event featured evaluators from the World Bank Group and EBRD, as well representatives from the Management side, including from the European Investment Bank and the International Finance Corporation. Discussions focused on the opportunities and challenges of collaboration between independent evaluation departments, as well as partnerships within our institutions, namely between evaluation and Management. It is important that we look at these kinds of collaborations, because nobody can face the challenges that confront us by working in isolation.
At the same time, we must recognize that there are also difficulties in working together. One of these is to balance the need for alignment and coherence among agencies with the respect for their diversity and autonomy. Different agencies have different mandates, priorities, approaches and methods for evaluation, which can create tensions and conflicts when working together. This notwithstanding, I believe that a possible opportunity is to develop common standards and principles for evaluation quality and ethics, as well
as joint frameworks and tools for planning, conducting and using evaluations to address big cross-cutting topics such as the SDGs, gender and climate change, to name a few.
Eddie
There are a couple of opportunities for our enhanced collaboration and coordination. First, there are opportunities in innovation. As discussed before, the current reality is of a world very much in crisis, and there is an urgent need for us to strengthen how evaluation can contribute best in such times, in the pivotal areas where the organization’s work addresses them head on – in peace operations, humanitarian work, climate change mitigation, and ensuring inclusion of all voices. The challenge is before us to innovate and find ways of doing our evaluations faster, and more accurately, to be able to support crisis decision-making in a timely manner. There is great opportunity in tapping developments in use of AI to support our work in these areas. But we are all chasing after these opportunities within our own bubbles. As professional networks, we need to have a clear approach on how to harness these collectively.
Second, there are opportunities to work as one. We have a Secretary-General that understands and supports evaluation, who has, analogously to his establishment of Resident Coordinators in support of the concept of One UN, established the System-Wide Evaluation Office, with which we can now concretely work and support urgently needed system-wide evaluation, as well as find ways to utilize the large evaluation data-sets available in the system. The system’s evaluation functions need more systemic and sustainable approaches – while engaging with local actors – to use long-term design (institutional and systemic and behaviour change) and innovation. With the Secretary-General’s management reforms, which included requiring and strengthening evaluation in all pillars of work, the large gap of evaluation coverage in the peace and security pillar can begin to be filled, and evaluative evidence in this important area of work brought into the SDG fold. Evaluation is still relatively nascent in the peace and security pillar, and there is good opportunity to establish and develop strong evaluation practice therein, but our colleagues in the peace and security pillar need our support, and we need to build stronger linkages between evaluation functions within the different pillars. We should strive to work as one, even if the rest of the system may not be all there yet.
Jörg
Evaluation networks are a cornerstone of the global evaluation system in our field. However, institutional fragmentation is severe and there is still a huge potential for increasing harmonization and joint work. Moreover, the emergence of evaluation standards and policies in the Global South, while in principle to be very much welcomed, has further added to an increasing complexity in the international evaluation system. Against this background, the global evaluation community faces a serious collective action problem consisting of three dimensions. First, there is a need to define universally accepted standards and guidelines, which provide a common ground for a joint understanding of what good and functional evaluation is but at the same time can be adopted to cultural and institutional peculiarities. Second, there is a need to invest more in joint evaluations, not only because of efficiency gains but also because silo-driven institutional perspectives often miss to carve out relevant evidence.
Finally, further investments need to be made in storing existing evaluations in such a way that the already existing evidence can be easily extracted and used by evaluators, implementers and policy-makers alike. Obviously, artificial intelligence could play a major role in fulfilling the latter task. Altogether, these challenges are not of minor size but at the same time, without international evaluation networks we will not be able to address them effectively. And finally, our networks can play an important role to shield evaluation from attempts to reduce its independence, a threat that should not be underestimated in a world where democratic accountability and transparency is under serious attack.
Thank you, esteemed colleagues. You are welcome, Alexander.
Conceptual change in thinking and evidence-based body of win-win solutions needed to integrate environmental and socio-economic domains
4th International Conference on Evaluating Environment and Development
“An evidence-based body of win-win solutions is needed, along with adequate donor investments in piloting such solutions, to convince governments of the viability and need to pursue the sustainable food security that results from agricultural interventions that strengthen climate, environmental and economic resilience together”, explained Dr Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD (IOE) during the 4th International Conference on Evaluating Environment and Development, organized by the Independent Evaluation Office of the Global Environmental Facility (GEF) from the 5th to the 7th of March 2024.
Held in Washington DC, the conference focused on how evaluation practice has increasingly integrated the cross-pollination of environmental, socioeconomic, and policy domains. Leading practitioners and thinkers on environmental evaluation, as well as young and emerging evaluators, came together to share concrete experiences with environmental interventions and evaluations, and to inform future evaluation practice. Panellists brought a wealth of knowledge on active efforts to develop evaluation approaches and methods that better reflect the varied and complex relationships of the natural world and the larger systems in which evaluands are measured.
“We, evaluators, have not challenged the global development indicators. Our measurement bias keeps us outside reality, at times, not allowing us to see that many development issues are historically and politically caused. When we plug environment into gross domestic products, the rating of countries changes. This points to where the weaknesses are”, Dr Naidoo affirmed.
On the first day of the event, Dr Naidoo moderated a panel discussion on Evaluations for Sustainable Food Security that provided a better understanding of the conditions needed in development efforts and evaluations to facilitate sustainable food security in the context of accelerating climate change. The panel included Dr S. Nanthikesan, IOE Lead Evaluation Officer, who conceptualized the session, and was accompanied by Michael Quinn Patton, Founder and Director of Utilization-Focused Evaluation; Katherine Dawes, Evaluation Officer of the US Environmental Protection Agency; Jane Davidson, Founder and Principal Consultant at Real Evaluation; and Andy Rowe, Founding Member of Footprint Evaluation.
Accelerating climate change poses a serious threat to the sustainability of food security. At the same time, food production in the form of agricultural activities could also harm the ecosystem, and further accelerate the climate change effects. Thus, global actors need to
collectively emphasize the urgency and need to understand, assess, and address the consequences to the ecosystem from human efforts to ensure food security for all.
“To have sustainable food security in the era of accelerating climate change, we need more than do-no-harm solutions. We need transformative solutions - solutions that promote climate, environmental and development resilience together are key. Evaluations can provide the necessary evidence. To do so, they need to be able to assess how development efforts impact ecosystem–environmental quality, biodiversity, natural resources replenishment, to name a few. Stronger multi-agency international collaboration is needed to provide the requisite initial investments, build necessary capacities and partnerships to advocate and strengthen climate friendly policies, and generate a user-friendly database of solutions”, Dr Nanthikesan explained.
Building on this premise, the panel discussions addressed first, the methods to assess the sustainability of food security in the context of a rapidly changing climate; second, the steps to scale up and refine these methods; third, how operational managers are taking up this approach; and fourth, how policy makers are taking up this approach at a national, regional and global level.
In this context, Dr Nanthikesan presented the pioneering method developed in the thematic evaluation of IFAD’s support to smallholder farmers’ adaptation to climate change to capture the environmental consequences of IFAD’s agricultural solutions to promote climate adaptation. The evaluation found that successful projects pursued integrated approaches with nature-based solutions, while failing to ‘do no harm’ is likely to lead to low sustainability of benefits of IFAD interventions. The event furthered this contribution to elaborate a methodology that, for the first time, helps evaluations address environmental and social-equity impact of development interventions together.
On the second day of the conference, Dr Naidoo acted as a panellist in the session titled ‘Evaluating Sustainability in Environment Funds’. In his remarks, the IOE Director stressed the critical importance of systematically integrating environmental and social impacts into evaluative practice, which is something that IFAD has done since 2010.
“ Michael Quinn Patton spoke about intercon -
nectivity over a decade ago, to no avail. Today, the post-COVID-19 reality and the climate have forced us to wake up and take note. Unfortunately, the horses have bolted in many cases. Being systemic and looking at the big picture calls for a fundamental change to how we view ourselves as evaluators. It calls for advocacy, pushing the boundaries, rethinking and reconceptualizing. It calls for changing the way we think about values ”, underscored the IOE Director.
On the final day of the event, Dr Naidoo delivered closing thoughts alongside Sabine Bernabè, Director General at the World Bank Independent Evaluation Group; Juha Ilari Uitto, Director of the GEF Independent Evaluation Office; Emmanuel Jimenez, Director General at the Independent Evaluation Department of the Asian Development Bank; and Rob D. van den Berg, Professor at King’s College, London. The session reflected on where we stand in mainstreaming environmental and ecosystem considerations in key development evaluations.
“ Evaluating the environment is very much a po -
litical question that has profound social ramifications. While at the country level results are more nuanced and differentiated, we can see that the Global South is at the receiving end of excesses that were caused elsewhere, which it has little or no power to change. For this reason, the deeper question is if we have efficient feedback loops towards decision-makers in order to affect change. At IFAD, efforts to mainstream environmental aspects have ensured that these are on the agenda. We have a methodology which privileges the marginalised ”, concluded the IOE Director.
Overall, the GEF conference themes addressed the urgency of tackling the environmental consequences of development, and focused on evaluation systems, evaluation practice, measuring sustainability, and thematic evaluations that looked at food production that is climate adaptative and provides mitigation while promoting social and economic equity as the foundation of food security. Conference outcomes will be disseminated through webinars and blogs via the Earth-Eval Community of Practice and a book will be published from key contributions to the conference.
Dr Indran A. Naidoo presentation
Technological, financial, social and institutional innovations address agricultural challenges in Rwanda
Various technological, financial, social and institutional innovations were introduced in IFAD’s portfolio in Rwanda, addressing key agricultural challenges. Steven Jonckheere, Senior Evaluation Officer at the Independent Office of Evaluation of IFAD (IOE), presented these results during the national workshop organized to discuss the findings of the Country Strategy and Programme Evaluation (CSPE) of the Republic of Rwanda, on 5 March 2024.
Innovations included intensive rice cultivation, improved animal breeds, a performance-based grant facility, community competitions for natural resource management and public-private-producer partnerships. These efforts contributed to enhancing productivity and, in some cases, to structural change.
“The evaluation found strong alignment of the country programme with government priorities. It built on IFAD’s comparative advantage in support of smallholders, to boost productivity and access to markets by focusing on investment in livestock, agricultural exports and irrigation”, noted Fabrizio Felloni, IOE Deputy Director, speaking on behalf of Dr Nai-
participants joined the event, representing the Government of Rwanda, including from the Ministry of Economic Planning and Finance; the Ministry of Agriculture and Animal Resources; the Rwanda Agriculture and Animal Resources Development Board; the National Agricultural Export Development Board; and the Single Project Implementation Unit. A host of development partners, civil society organizations, private sector partners, and IFAD senior management and staff also attended the workshop.
Co-organized by the Ministry of Agriculture and Animal Resources of the Republic of Rwanda and IOE, in collaboration with IFAD’s East and Southern Africa Division (ESA), the on line virtual workshop brought together a wealth of high-level attendeees, including the Honourable Dr Ildephonse Musafiri, Minister of Agriculture and Animal Resources; Sarah Nyiramutangwa, Head of the Single Project Implementation Unit, Ministry of Agriculture and Animal Resources; Donal Brown, Associate Vice-President, Programme Management Department, IFAD; and Sara Mbago-Bhunu, Regional Director, ESA, IFAD. Over 50
“I look forward to discussing the findings and reflecting on them, to make sure that implementation of future projects is even better. Most importantly, we need to ensure sustainability of interventions”, said the Honourable Dr Musafiri.
The CSPE was the third country-level evaluation in Rwanda, and covered the period 2013–2022. It focused on nine loan-funded projects and 24 IFAD-funded grants. The estimated cost of the nine investment projects covered by the CSPE amounts to US$509.6 million, of which US$280.1 million was financed by IFAD. The remaining funds came from the Government, other co-financiers and beneficiaries.
doo..
Steven Jonckheere Lead author, Rwanda CSPE
Profile
“Together, the Government of Rwanda and IFAD have achieved significant progress in promoting rural development, and I would like to strongly reiterate IFAD’s commitment to continue supporting the Government and the people of Rwanda to improve their livelihoods”, affirmed Dr Brown.
During the workshop, participants discussed the main findings, recommendations and issues emerging from the CSPE. The meeting also provided an opportunity to focus on the opportunities and challenges of the partnership between IFAD and the Government in the near future, and discuss strategic priorities for IFAD’s upcoming programme.
The country programme contributed to improving food security, mainly through increasing food availability and access, including by increasing the production of staple and horticultural crops, reducing post-harvest losses and introducing livestock. It also contributed to increasing incomes by improving production and productivity, reducing post-harvest losses, diversifying to higher value crops, and adopting group selling, price
mechanisms, and new sources of income.
Individual projects reached or exceeded their outreach appraisal targets in terms of persons receiving services promoted or supported by project interventions, and contributed to product, process and functional upgrading, whilst building the capacities of producers’ organizations. Strengthening technical skills, supplying agricultural inputs and supporting irrigated agriculture contributed to productivity increases.
and
Unfortunately, the project designs did not include nutrition-sensitive interventions in a strategic manner. Consequently, few activities addressed the main cause of malnutrition, namely, the limited consumption of nutritionally diverse foods. Projects often applied a blanket approach, without tailoring interventions to the actual needs of the target groups and context, while increases in a number of commodities failed to meet expectations and were not always sustainable.
Very limited results were achieved in fostering financial inclusion, with an overreliance
on the use of matching grants and insufficient outreach. The supported cooperatives remained generally inadequate, particularly in terms of leadership, income generation, governance and record-keeping. In addition, their business orientation and vertical linkages among stakeholders at different functional levels remained weak.
Seeking to further strengthen the impact of IFAD’s operations in Rwanda, it is recommended that in future programmes the Fund should focus on thematic areas in which it has demonstrated a comparative advantage, such as livestock, agricultural export and irrigation. IFAD should also refine targeting strategies to sharpen the poverty focus and increase attention to youth inclusion, and articulate a coherent action plan for non-lending activities to increase IFAD’s scale of impact and influence.
Rwanda has an estimated population of 13.2 million, more than 72 per cent of which lives in rural areas. It is a low-income country. Although the level of poverty has declined over the past decade, the pace of poverty reduction has slowed; almost one in three Rwandans lives in poverty.
Dr Donal Brown AVP, PMD, IFAD
Dr Ildephonse Musafiri, Minister of Agriculture
Animal Resources
Sara Mbago-Bhunu, Regional Director, ESA, IFAD
How a sculptor is taking charge in preserving the environment*
Analytical framework paradigm of fragility
11th International conference
Inspired by the rich cultural heritage of his homeland, Claude Nizeyimana, a 25-year-old Rwandan sculptor, last year initiated a project ‘Ingoma100,’ with an aim of intertwining the realms of art and nature. ‘Ingoma100,’ which translates to ‘100 Drums’ is a collaboration with diverse artists to plant trees across various regions of the country.
Grounded in his experience as a sculptor, Nizeyimana discovered the motivation for ‘Ingoma100’ during a period of reflection. Recognizing his reliance on trees as a fundamental aspect of his craft, he realised a disconnection between his artistic output and the natural lifecycle of trees. Acknowledging the indispensable role of trees in various aspects of human life, he adopted a practical yet proactive perspective.
Realizing the detrimental consequences of using trees without taking action, Nizeyimana resolved to make a meaningful difference. “I found out that using these trees and not doing anything afterwards is bad. So, I told myself that in my little capacity, I can start doing something; that maybe if I use 3 trees in a year, maybe I can plant 1,000 trees to make an impact.”
Nizeyimana dreams big for his project, driven by a deep-seated passion for tree planting. “I want to see my project go far,” he expressed. Beyond planting trees, he is committed to conducting research on their utility and potential benefits.
Applying an analytical framework complementary to the theory of change was an innovative and successful approach that the Independent Office of Evaluation of IFAD (IOE) adopted in designing the sub-regional evaluation of countries with fragile situations in West and Central Africa. Dr Kouessi Maximin Kodjo, Lead Evaluation Officer at IOE, presented the details of this approach, the main results and lessons during the 11th International conference of the African Evaluation Society (AfreEA), along with the salient results of the evaluation itself.
The 11th AfrEA Conference 2024 provided a platform for professionals, researchers, students and policymakers worldwide to share insights, best practices and innovations in the field of monitoring and evaluation, especially on topics related to African contexts. Held in Kigali, Rwanda, the conference covered a wide range of topics related to made-in-Africa evaluation, indigenous evaluation, youth-led evaluations, best practices, emerging trends, technological innovations, impact assessment and ca-
framework helps ascertain operational fragility in West African Sahel
conference of the African Evaluation Society
pacity-building in the field. The event featured two days of pre-conference workshops and training sessions, on the 18th and 19th of March, followed by the three-day main conference, from the 20th to the 22nd of March 2024.
Presenting during the strand on ‘evidence generation, use and application in African political governance system’ – one of the twelve of the conference’s extremely rich programme –, Dr Kodjo explained that applying an analytical framework complementary to the theory of change led to ascertaining the operational paradigm of fragility and resilience building in rural areas of the West African Sahel. Initial contextual analysis and later outcome assessments were
based on the fragility and resilience framework, which has been useful in generating the evaluative findings. The framework enabled to carry out assessments focusing on the three main resilience-building pillars, namely absorptive, adaptive and transformative capacities. This was useful to deal with the complexity of the evaluation, which followed a systemic approach.
The evaluation identified many lessons from rural development operations in fragile situations. These included the importance to build on the strengths of endogenous and customary arrangements for managing natural resources or land tenure, as well as of existing mechanisms such as credit and saving and economic interest groups, in order to develop absorptive and adaptive capacities. The evaluation also found that it is useful to invest in approaches for improved natural resources management, for instance by promoting small-scale irrigation schemes and climate-smart agricultural practices.
Dr Kodjo highlighted that it is equally important to support the diversification of econom-
ic opportunities, which are crucial for effective resilience building strategies. However, these may be challenged by insecurity that can hamper farmers’ access to markets. In those situations, it is imperative to invest sufficiently in strengthening human and social capital for the emergence of strong and effective grassroots organizations. In addition, in contexts of fragility, it is critical to develop a resilience framework based on a nexus approach of conflict prevention, resilience-building and development, which should guide operations.
Selected papers and presentations from the conference will be compiled into proceedings and may be published in special editions of the African Evaluation Journal and a Conference Proceedings Compendium, and shall be made available online, contributing to the broader knowledge base in the field of monitoring and evaluation.
Dr Kodjo’s presentation
Dr Kouessi Maximin Kodjo, Lead Evaluation Officer, IOE
PRINCIPLE-BASED EVALUATION
“We need a principle-based approach to evaluation policy and practice to ensure that M&E is optimally pitched to bring about the transformation that is necessary. This means moving away from the use of ahistorical country ratings and simplistic criteria that do not reflect the new world order. For this to happen, there must be a critical approach to setting up evaluation, merging best thinking, recognizing the politics of the craft and drawing-in stakeholders. Egypt Vision 2030 offers a shining example. The development agenda reforms planning and M&E by applying the methodology of programmes and performance-based budget plans”, affirmed Dr Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD (IOE), in his keynote address during the 10th EvalMENA Annual Meeting and First National Egyptian Association for Evaluation (EgyEval) Meeting, on 15 February 2024.
EgyEval hosted the 10th EvalMENA annual meeting in Cairo, from 12 to 15 February 2024. The conference was held in the Institute of National Planning of the Ministry of Planning and
IN NEW WORLD
Economic Development, under the auspices of Her Excellency the Minister of Planning and Economic Development, Dr. Hala El Said. The theme for this year was ‘Institutionalization of Evaluation in Arab Countries: An Approach to Realize SDGs Based on Evidence, and the Role of National Evaluation Associations’. The primary objective of the meeting was to share the experiences of Arab Evaluation Associations regarding the institutionalization of programme, projects, and policy evaluation for development. To this end, the event explored ways in which development partners support national efforts and generate recommendations at both regional and national levels.
The IOE Director opened the proceedings of the third day of the conference, addressing the importance of effective evaluation architecture for attainment of the SDGs in a session titled ‘Partnerships needed for Financing owned Evaluations’. The session was chaired by Dr Doha Mounir Abdelhamid, Senior Economics and Development Evaluation Expert, and also featured a presentation by Ian Goldman, President of the International Evaluation Academy. Dr Naidoo drew on lessons from the work of IOE and IFAD in Egypt, as well as on experiences from the National Evaluation Capacity (NEC) series, which was held in Hurghada, Egypt, in 2019, and which became the largest evaluation
POLICY AND PRACTICE REQUIRED
WORLD ORDER
event by government representation globally under Dr Naidoo’s six-year tenure as Director of the Independent Evaluation Office of UNDP.
“Parts of global oversight continue to rate countries as if there is no past. Remnants of classic thinking remain, plagued by simplistic assumptions and the conduct of evaluation in linear terms. This approach fails to recognize the complexity of the world we live in”, stated Dr Naidoo recalling that, in essence, evaluation should be used to reduce inequalities and promote sustainable development.
In response to these shortcomings, strong monitoring and evaluation systems need to be implemented at the national, local and provincial levels in order to ensure that limited resources are optimally utilized to achieve development objectives, explained the IOE Director. Evaluation must strive to help countries accelerate attainment of Agenda 2030. However, SDG progress has been derailed by COVID-19, and is further compounded by the global perma-crisis that we live in. The effects of COVID-19 will continue into the future because they have disrupted the way in which matters such as remittances, value chains and financial flows work.
Against this backdrop, Egypt produced a very good UN socio-economic response plan during
the pandemic, which led to a re-prioritization of resources in order to build back better. The development agenda of Egypt is very clear, as articulated by the national government in the ‘2030 Egypt Vision’. Egypt also plays a strong role internationally, including by engaging with IFAD and sitting on the Fund’s Evaluation Committee.
“The Government of Egypt shows the highest level of transparency and engagement in improving the development impact of the country. IFAD has continued to support the government and has benefitted from close engagement with H.E. Dr Rania Al Mashat, Egypt’s Minister of International Cooperation, who recently launched IFAD’s 13th replenishment meeting. Egypt is now also part of a broader coalition working within Africa and in liaison with international financial institutions and UN agencies”, underscored Dr Naidoo.
The four-day conference also looked at the existence of national evaluation policies through which the programmes implemented to achieve sustainable development within the framework of the state’s vision and SDGs are evaluated, and the institutions that do so are evaluated.
Dr Indran A. Naidoo presentation
Cairo named OIC capital of tourism for 2026
Cairo has been designated as the capital of tourism for member states of the Organization of Islamic Cooperation (OIC) for the year 2026. The prestigious title was revealed during the 12th session of the Conference of Tourism Ministers of OIC member states, which took place in Uzbekistan from May 31 to June 2, 2024.
The conference, themed “Developing the Tourism Industry in a Sustainable and Flexible Manner,” aimed to select tourist cities from the Islamic world for the years 2025 and 2026. Minister of Tourism and Antiquities Ahmed Issa expressed his delight at Cairo’s selection as the OIC capital of tourism for member states, highlighting the city’s diverse range of tourism offerings.
Adel El-Gendy, the Director General of the General Department of Strategic Management, highlighted that Cairo City Break showcases the ministry’s endeavours to enhance various historical, spiritual, and heritage attractions.
*www.egypttoday.com/
Evaluations need
Evaluations are required to analyse how far organizational processes, procedures and setups are fit for purpose, as this often lays of the foundation of performance and, ultimately, success. This menas the methodologies to conduct these evaluations are complex and challenging. To exchange experiences of different organizations that have conducted such evaluations, a webinar organized under the auspices of the Evaluation Cooperation Group (ECG) discussed the main conceptual and methodological challenges of corporate evaluations. Speaking at the event, Dr S. Nanthikesan, Lead Evaluation Officer at the Independent Office of Evaluation of IFAD (IOE), presented lessons from IOE’s 2023 Corporate-level Evaluation of IFAD’s Decentralization Experience.
Held on 18 March 2024, the ECG webinar was chaired by Emmanuel Pondard, Head of the Evaluation Division (EV) at the European Investment Bank (EIB). In addition to the insights delivered by IOE’s Lead Evaluation Officer, the event featured presentations by Mónica Lledó Moreno and Lena Zimmer, respectively Senior Evaluator and Evaluator at the EV. Over 50 participants joined the online session.
Discussions sought to shed light on a number of salient issues related to corporate evaluations, including how evaluation differs from organizational audit in this respect; the evaluation methods in the absence of a defined normative framework; and how to address the sensitivity of evaluated stakeholders who may
Dr S. Nanthikesan Lead Evaluation Officer, IOE Profile
need to analyse the machinery of an organization
feel directly judged vis-à-vis their roles and status within the organization.
Dr Nanthikesan touched upon a number of these issues in his presentation. He explained that the framework for evaluation was determined based on management proclamations that decentralization was pursued to enhance IFAD’s contribution to development effectiveness. The evaluation pursued a mix of quantitative and qualitative methods to assess the contribution of field presence to improving the performance of lending and non-lending activities and to assess organizational fit-for-purpose to deliver effective operations and country strategies. He also noted that given the sensitivity of the evaluation of IFAD’s decentralization, to improve buy-in, IOE undertook sustained engagement with Management throughout the evaluation process, from design to reporting. This resulted in an enthusiastic support from staff and middle management, while Senior Management took a different stance, upon the initial release of the report, in April 2023. The situation evolved
over the course of the following months, as the importance of the evaluation’s findings became increasingly appreciated. In November 2023, when IOE held an organization-wide learning event to discuss the evaluation, Management pledged to implement all recommendations and outlined the actions already underway.
In the same vein, the EV presentations highlighted the added value of the evaluation of EIB’s advisory activities in the EU. The report found that while the EIB has developed a wide range and breath of advisory activities, the advisory offer in its entirety is rather unknown to both internal and external audiences. For this reason, communication across directorates and mandates would need to be improved to increase synergies. In addition, clearer longterm strategic orientations could help to better position advisory in the EIB’s business model.
Dr S. Nanthikesan presentation
Learning from evaluation must take place at country level
“It is important to make sure that learning from evaluation efforts is taking place in the countries that we work in, and that we involve the voices of local people. The traditional model of ‘fly-in, get the data and feed it back to our institutions’ needs to change. We need to be much closer to the partners that we work with, ensure that learning is happening at the country level, and that the notion of accountability is also for the beneficiaries of the projects, not only for our donors”. Dr Sabine Bernabè, Vice Presi-
dent and Director-General of the Independent Evaluation Group (IEG) of the World Bank, explained the importance of embracing this approach during a briefing session with the staff of the Independent Office of Evaluation of IFAD (IOE), on 31 May 2024.
In her visit to IFAD headquarters in Rome, Dr Bernabè was joined by Mr Dugan Fraser, Program Manager of the Global Evaluation Initiative (GEI). Dr Bernabè and Mr Fraser made a presentation to IOE staff on the work of GEI to date, and delved into its plans for future collaboration. The valuable insights sparked an engaging exchange around the important opportunities that the growth and development of the GEI presents the international evaluation community. During their visit, Dr Bernabè and Mr Fraser also met with Dr Gerardine Mukeshimana, Vice-President of IFAD.
GEI is an alliance of bilateral and multilateral development agencies and centres for higher education and research. IOE is a significant contributor to the Initiative, which supports developing countries in strengthening their monitoring and evaluation (M&E) systems to help governments gather and use evidence that improves the lives of their citizens. IEG leads the Initiative, in collaboration with the Independent Evaluation Office of the United Nations Development Programme (IEO).
In his presentation, Mr Fraser noted that GEI uses an integrated systems approach to enhance the institutional capacity of developing countries to conduct and learn from evaluations. Lessons learned through practice allow continuous refinement of the approach to provide effective country-specific solutions. In particular, GEI provides training, technical
assistance and knowledge services to partner countries. Flagship programmes include the Centres for Learning on Evaluation and Results (CLEARs), the International Program for Development Evaluation Training (IPDET), the Better Evaluation platform, and the gLOCAL Evaluation Week.
GEI has achieved notable results over the past three years. These include having trained 15,000 people in M&E, strengthened 34 sub-national M&E systems, organized 1,147 M&E events that have been attended by 69,000 people, reached 1.2 million users via social media, and accrued over 2 million web page views.
Looking at the next five years, the Initiative is seeking to further enhance its offerings in a number of ways. Key priorities include achiev-
ing an even deeper engagement at country level for transformative impacts, selectively expanding its reach of regional partners to cover more countries, launching new and fully functional regional centres and partnerships, and increasing capacity development opportunities for a more diverse and inclusive cadre of M&E professionals.
Dr Indran A. Naidoo, IOE Director, was among the first the first to believe in the value of the GEI. In his previous role as Director of IEO, he signed a Memorandum of Understanding with Alison Evans, former World Bank Vice President and IEG Director General, in January 2020. This marked the first step towards establishing the global partnership to support evaluation capacity development.
NEW KNOWLEDGE MANAGEMENT TOOLBOX INFORMAL CONVERSATIONS
Open and informal conversations allow to capture the essence of individuals and truly hear their voices. Two remote participatory video evaluation pilots (PVEs) in Brazil and Peru revealed this finding, showcasing how the use of video interviews for communication materials hold exciting possibilities that add value to traditional evaluations. The Independent Office of Evaluation of IFAD (IOE) recently published the PVEs in the context of its new Toolbox to evaluate knowledge management (KM) practices, which was developed as part of the corporate-level evaluation on KM in IFAD.
IOE developed the PVE methodology as a way to find innovative approaches that enhance the impact of IFAD’s programmes, and include the voices of local and indigenous communities. The eight interviews with IFAD beneficiaries provided crucial feedback on the value add ed by the Fund’s projects, the challenges en countered, and potential ideas for improvement. By providing a genuine opportunity for gath er authentic feedback, the PVEs helped to address under-representation and remote access, gain a deeper understanding of beneficiary needs and successes, and ensure that local voices were captured.
In addition to the PVEs, IOE’s new Toolbox also features two other innovative resources, name ly the mapping of knowledge sharing and dis semination, and the tracking of KM practices across different transformative levels.
The mapping of selected signature solutions, championed by IFAD, that have improved the lives of rural people helps to understand how knowledge-specific approaches and innova tions have moved through IFAD’s work. In par
ticular, the resource identifies the mechanisms that enabled effective replication of signature solutions in countries across the global portfolio, shedding light on how IFAD shares and scales up its solutions.
The ten selected signature solutions are small scale irrigation, homestead gardens, the gender action learning system, learning routes, seeds certification, new ruralities, experience capitalization, farmer field school, leasehold forestry, and public-private producer partnerships.
TOOLBOX REVEALS THE POWER OF OPEN AND CONVERSATIONS
It emerges that IFAD has effectively championed these ten signature solutions, which have been promoted and developed by the organization. The knowledge generated has ensured tailored interventions that address specific needs of rural communities and maximize positive outcomes. The solutions have been applied transversally across various regions, allowing for cross-context learning and adaptation, showcasing their versatility and effectiveness in diverse rural landscapes. Scaling up is facilitated through active engagement of
stakeholders in knowledge-sharing activities such as interviews, talks, and community platforms, which amplify reach and replicability.
The third component of IOE’s KM Toolbox is the tracking of KM practices across different transformative levels. The rubric evaluates the immediacy, medium-term, and long-term outcomes of IFAD’s KM practices in terms of transformation on a country-by-country basis, through 20 case studies from across the world. The case studies recorded and assessed the factors which influenced the effectiveness and transformational nature of KM practices, including internal KM frameworks, staff capacities and incentives, IFAD’s country presence, partnerships, and government capacity.
The exercise reaffirmed that a coherent country-level KM strategy motivates implementing partners to concentrate more on institutional learning. It also highlighted that strategic selection and monitoring of KM products helps to ensure impact, replicability and innovation; that KM planning and professionalizing can enhance quality and effectiveness in documenting practices and lessons; and that recording knowledge and lessons allows for adaptive management and flexible implementation, as well as supporting future programme development. In line with the other resources of the Toolbox, the rubric also confirmed, once again, that the richness of local knowledge requires appropriate tools for cross-learning and knowledge-sharing mechanisms.
