Independent Magazine – Issue n.2, 2021

Independent

Learning. Transparency. Accountability.

MAGAZINE

LEARNING FROM EVIDENCE Not just another day in the office

GOOD IS NOT GOOD ENOUGH Climate responses to escalating threats

Issue n.2

E VA L U AT I O N C A PA C I T Y 2021

Making a difference in Egypt

1

In o ur ow n wo rds

W

elcome to the second edition of Independent Magazine. As the IOE magazine series evolves, we wish to use this medium to advance debate globally on how to best produce and use performance feedback and evidence to build organizations. Independent Magazine demonstrates our commitment to engaging with the broad spectrum of evaluative issues by inviting contributions from evaluators, evaluation stakeholders and the different users of evaluation, with a view to optimize the impact of evaluation.

In this edition, IOE is pleased to introduce the contributions of key management and leaders at IFAD as users of evaluation. The dynamic nature of IFAD as a UN specialized agency and International Financial Institution, which provides loans and advisory services to governments, means that there is a high demand for performance information, for fiscal and programmatic decisions that depend on credible information, of which independent evaluation is key. In the independent construct of evaluation with its normative definitions, standards and practices, there is discussion on the interplay between independence, credibility and utility. In the independent evaluation offices of the International Financial Institutions the evaluation policies are well-established, demonstrating a maturity of function and common understanding on that what needs to be done, for what purpose.

Evaluation supports accountability and learning to improve transparency and accountability. However, the area that can benefit from user perspectives is that of learning, which is defined herein as reflective opportunities. In practice, the construct of all evaluation needs to accept engagement as a part of process credibility, seen as necessary for generating ongoing reflection as part of the learning process to build understanding. This implies moving beyond a transactional approach to evaluation between evaluator and evaluand. It requires building spaces for open dialogue between evaluators and stakeholders, while also accepting areas of exclusive domain between parties and respecting the roles and prerogatives of all parties. Therefore, genuine learning is possible within an evaluative construct that is engaging, whilst respecting, one the one hand, the evaluators’ need for independence and their mandate to formulate a final assessment, while on the other hand, recognizing the authority of evaluation stakeholders to take their decision on the recommendation acceptance.

2

In this issue, the insights of what constitutes good evaluation from the perspective of IFAD leadership seeks to generate this conversation. In subsequent editions we shall reach out to IFAD Board members for perspectives as well. We believe this approach is more useful than debating on the oft-invoked dichotomy between accountability and learning. The reality is that both are to be pursued and the specific emphasis will depend on the particular evaluation products. Both the evaluators and the evaluation stakeholders need to make a stronger commitment to building an evaluation culture. Ensuring that the feedback loops are dynamic and iterative throughout evaluation planning, execution, completion and follow-up is a challenge necessary to build organizational credibility.

We trust you will find this edition interesting.

Indran A. Naidoo, PhD.

Fabrizio Felloni

Director Independent Office of Evaluation of IFAD

Deputy Director Independent Office of Evaluation of IFAD

Independent Magazine brings to the forefront of the global development dialogue the major efforts undertaken by the Independent Office of Evaluation of IFAD, while seeking to advance the organization’s vision of vibrant, inclusive and sustainable rural economies, where people live free from poverty and hunger. To present the richness of rural life, and detail facets of local community lifestyle, Independent Magazine also zooms in on cultural activities and landmark occurrences in countries featured by IOE’s evaluations.

@Unsplash/Jabriel Jimenez 3

Editorial Team

Alexander Voccia, PhD

Shaun Ryan

Editor

Copy editor

Writing, graphic design & publishing

Proofreading & revision

@Unsplash/Ivan Bandura 4

CO N T E N T S 06

“Good is not good enough” climate responses struggle

42

Value chain governance and community-driven development

08

Monitoring and evaluation making a difference in Egypt

44

Greater focus on pro-poor targeting called for in Uganda

12

Flexibility needed in countries with fragile situations

46

Adaptation not synonymous with environmental resilience

14

Approaches to design in fragile settings

48

Farmer productivity increased in Senegal with sustainability risks

16

Learning, growing, changing lives - not just a day in the office

50

Evaluations under COVID-19 and pathways to food security

30

New approach to mass mobilization in Ethiopia

52

Social norms challenged: women participate in market activities

32

Time for evidence to contribute to gender equality

54

Change needed for value chain transformational interventions

34

Giant leap for the audit and evaluation community

56

Mechanisms needed to reach poorest in Dominican Republic

39

Centrality of evaluators uncertain in post-COVID world

58

Evaluation must play central role in building food secure world

40

IFAD strategy contributes to resilience in Burundi

59

IFAD’s evaluation function top among development banks 5

“GOOD IS NOT GOOD ENOUGH” Climate responses struggle to meet escalating threats

Donal Brown

“

We are only eight years away from the year 2030. The dire warning of the IPCC’s recent report is upon us. There is an urgent need to act, and ‘good is not good enough’. We need beyond good – we need transformative solutions.”

This, according to Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD, who moderated a side event at the COP26, in Glasgow.

The IFAD-led side event took place on 4 November 2021, under the auspices of the 26th session of the Conference of the Parties (COP26) to the UNFCCC. Entitled ‘Evidence-based Transformative Pathways to Build Climate Resilience in the Agricultural Sector’, the talk-show style event gathered insights from leading evaluation experts and senior international development actors, and blended them with civil society and government experiences. Panellists and speakers included Donal Brown, Associate Vice-President of the Programme Management Department of IFAD; S. Nanthikesan, Lead Evaluator at IOE; Andreas Reumann, Head, a.i, of the Independent Evaluation Unit at the Green Climate Fund; Sven Harten, Deputy Director of DEval; and James Gasana, international specialist in natural resource management and climate change.

Climate change disproportionately affects smallholder agriculture that constitutes 75% of the global farmland and provides more than 80% of food consumed in the developing world. As such, strengthening climate resilience of smallholders is at the heart of IFAD’s mandate to eliminate poverty and food insecurity in rural areas.

6

“Smallholder farmers that have received [IFAD] support are 13% more resilient than their counterparts – that represents 26 million people. To do that, we put dedicated resources right down to where they count, to the most vulnerable. Over the past three years, 35% of our resources have gone to adaptation. In the next three years it will be 40%”, explained Donal Brown.

To reap the benefits of these efforts and those of the international community at large, there is urgent need to go beyond effective measures and seek transformative climate change adaptation (CCA) solutions.

“We support access to knowledge, investing in and using the knowledge of local communities. We put communities at the centre of natural resource management, identifying adaptive solutions with communities. It’s not just about the hard infrastructure, it’s also about investing in the capacity of communities in managing these effectively. Key to this is collecting good data and evidence, and adjusting decisions in real time – adaptive management”, further stated Donal Brown.

Evaluative evidence shows the successes of these and other similar interventions, at local level.

Indran A. Naidoo

“Yet, there remains a shortfall of broad knowledge-base of climate adaptation solutions, as these experiences are not translating into global success”, underscored Dr Naidoo.

Against this backdrop, the experts presiding the side event explored the challenges and oppor-

IOE-led side event at COP 26

tunities to pursue sustainable, viable solutions to build climate resilience. Discussions centred on the need to build an evidence-based knowledgebase, to understand the characteristics of viable transformative solutions, and to prioritize the role of partnerships to build climate resilience rapidly.

S. Nanthikesan

“There is no single pathway to scaling-up. There are common paths leading to successful scaling-up. One of the critical elements will be the commitment of governments, at all levels. A second path to emphasize is partnerships and coordination among donors, among government units, and partnerships with farmer organizations as well as other actors. No scaling up will work if there is no demand from the farmers. They need to be aware of climate risks and familiar with the technologies of adapting. To change behaviour, there have to be conditions that enable farmers to take the necessary risks, such as incentives and community support”, underlined S. Nanthikesan.

Multiple factors have stifled the pursuit of these pathways. James Gasana, Andreas Reumann and Sven Harten put the spotlight on the limited awareness of the importance of CCA to sustain-

able development among decision-makers, on the mutually reinforcing negative impacts of demographic changes and climate change, and on the shortfall of available climate finances. Of the USD 100 billion promised, only a fraction has been made available. How available finances are allocated is also a critical factor, as is the question of whether resources are reaching the most climate vulnerable countries, or whether these finances are directed at the most critical climate priorities of countries.

“Looking at ten years of spending of Germany’s bilateral cooperation, we found that if a country is vulnerable, that increases the likelihood of it receiving funding. However, if a country becomes more vulnerable, that doesn’t mean that it will receive more funding”, observed Sven Harten.

The time to overcome these limitations is overdue. Small-scale producers must be at the centre of climate change discussions as they are greatly underserved by climate finance. Bringing rural small-scale producers out of poverty and food insecurity will make positive, cascading contributions to meeting both the humanitarian and environmental goals of the 2030 Agenda. For this to happen, what we need is clear.

“We need to mobilize finance for adaptation. We need stronger partnerships. We need to institutionalize adaptation practices. And we need to bring the private sector into this picture”, concluded Donal Brown.

7

MONITORING AND EVALUATION: MAKING A DIFFERENCE IN EGYPT

I

OE seeks to expand its excellent collaboration opment partners and senior government officials, with the Government of Egypt, as the Office including H.E. Mostafa Madbouly, Prime Minister plans to further strengthen the monitoring and of Egypt, H.E. Rania A. Al-Mashat, Minister of Interevaluation (M&E) capacity of IFAD programme national Cooperation, and H.E. EL-Said Marzouq countries. Between 23 and 29 October 2021, InEl-Qosair, Minister of Agriculture and Land Recladran A. Naidoo, Director of IOE, presented these mation, with whom they discussed IFAD’s role in opportunities Egypt. to high-level counterparts “A key factor from the Arab for success is Republic of when you have Egypt, as he the highest level joined memof government bers of the support and a IFAD Execvery good colutive Board laboration with and senior IFAD on the IFAD delegation at the artificial insemination center management ground. This was for a high-level visit to the country. and is very much the case in Egypt. We engaged with the central ministries, we met the Prime Min“The Board visit to Egypt was an excellent way to ister, the Minister for International Cooperation, and witness first-hand how IFAD works at the country the Minister for Agriculture. They have all given level, and to see the way it engages with government the highest level of support to IFAD’s intervention and communities. We saw significant evidence – which is financially very significant”, Dr Naidoo to show that the various streams of intervention highlighted. have been successful and continue to grow both in terms of Focusing on the number of possibilities for beneficiaries, future capacity and of having development , impact at the Dr Naidoo outenvironmental lined the possilevel. It was an bility to explore amazing exo p p o r tu n i t i e s perience”, Dr for developing Naidoo statM&E capacied. ty for rural development in Indran A. Naidoo and Dina Saleh at the artificial insemination center The visit alEgypt, as a pilot, lowed Board representatives to experience IFAD’s working with the Global Evaluation Initiative (GEI). work on the ground, and observe the strengths and Dr Naidoo explained that this approach would challenges that the organization faces in carrying expand the work on national evaluation capacity out its mandate. During the visit, Dr Naidoo and that the Ministry of Planning, Monitoring and Adthe members of the Board engaged with develministrative Reform of the Government of Egypt

8

facilitated in the context of the National Evaluation Capacity (NEC) conference.

non-lending activities, in addition to site visits to projects co-funded by IFAD and the Government of Egypt.

“From a broad evaluation point “We are of view, it was proud of our very encouragstrategic and ing to note that highly prothe government ductive partnow has a very nership with strong M&E Egypt that system. We very marks four much look for- Indran A. Naidoo and Ronald Meyer, IFAD Representative for Germany, at decades of ward to working West Noubaria Rural Development Project - Noubaria Israa village a successful with the govmodel of coernment as we plan future assessments”, Dr Naioperation aimed at reducing poverty, addressing doo said. food insecurity and contributing to the resilience and improved livelihoods of more than eight milHeld in 2019, the NEC conference brought tolion rural people,” explained Dina Saleh, IFAD’s Regether over 500 people from 117 nations and govgional Director for the Near East, North Africa and ernments, with trainings delivered by 30 internaEurope division. tional experts, including in The field Arabic. Amina visits alMohammed, lowed Board UN Deputy members to Secretary-Gendeepen their eral, and H.E. knowledge of Hala Helmy El activities on Saeed, Ministhe ground, ter of Planning, and to hold Monitoring and discussions Local beneficiaries at the artificial insemination center Administrative with benefiReform, delivered opening remarks alongside Dr ciaries, farmers’ organizations, cooperatives, priNaidoo. vate-sector representatives and local government authorities. Sites included a livestock and dairy Other topical issues addressed by members of processing farm in West Nubaria, and a metrologIFAD’s Executive Board during their deliberations ical station, aquaponics greenhouses, nurseries with Egyptian policy makers included public poland biogas units in the Kafr El Sheikh Motobus icies on comDistrict.

In munity infraaddition, structure, water Board memmanagement , bers spent microfinance , time at the and agriculturAmreya Aral technologies tificial Inas a means of s e m i na t i o n promoting rural Center, esdevelopment . tablished unThese meetings Indran A. Naidoo and Luis Jimenez-McInnis, Secretary of IFAD, at aquaponics der the West greenhouses, nurseries and biogas units in the Kafr El Sheikh Motobus District were compleNoubaria Rumented by a portfolio presentation by the Cairo UN ral Development Project (WNRDP), and currently Multi Country Office team reflecting lending and operated by the Sustainable Agriculture Invest9

IFAD delegation with H.E. Rania A. Al-Mashat, Minister of International Cooperation of Egypt

ments and Livelihoods (SAIL) project.

“The artificial insemination centre has proven to be a sustainable project. It no longer has IFAD funding, and has been taken over by the government. It is now self-funding and is working extremely well.

Another project that we visited focuses on building the capacity of women in textile production. This has empowered women both in terms of receiving income and also in their ability to engage more with the economy. We also saw several successful projects in the field of agri-business, and projects where deserts are being irrigated and are now producing cash crops, such as mangos” Dr Naidoo reported.

IOE evaluated these projects and activities in 2017, through both the Egypt Country Strategy and Programme Evaluation, and the Project Performance Evaluation of the West Noubaria Project.

“An element that prevailed throughout the mission was the use of the last completed evaluation done by IOE. This was the basis for understanding the progress that has happened over the past few years. The recommendations that IOE made are 10

being acted upon. It is very much a case where evaluation has worked, it has provided evidence, it’s been acted on and can now be tracked in terms of progress. This really shows the impact of IFAD’s interventions both as a funder and a supporter in a very large country”, Dr Naidoo underscored.

IFAD currently has three ongoing projects in Egypt: Promoting Resilience in Desert Environments (PRIDE); Sustainable Agriculture Investments and Livelihoods (SAIL); and the Promotion of Rural Incomes through Market Enhancement (PRIME) project. IFAD is investing US$ 203.47 million in these rural development projects out of a total cost of US$ 284.49 million. The projects are expected to benefit over 616,000 people in rural areas.

“The contributions of our member states are crucial in pursuing our shared goals in supporting rural people, developing rural economies and effectively investing in programmes that will help countries to end poverty by 2030 as part of their commitment to the Sustainable Development Goals,” Ms Saleh noted.

Luxor to become an open museum*

F

inal touches are currently being placed in Luxor’s Al-Kebbash Road prior to its grand opening.

With this massive renovation project, the Egyptian state aims to convert Luxor into an open museum, restoring its historical beauty and luster in a manner that befits its rank as one of the most important historical cities in the world.

A l-Kebbash Road is the road that connects the Luxor Temple with the Karnak Temples. A wide street started from the beach, with statues of sphinxes, which we find in the temples of Karnak, represented in the form of a sphinx with the head of a ram.

A long the 2,700 km road there are 1,200 statues. These statues were carved from a single block of sandstone with a cornice inscribed with the name of the king, his titles and praise for him, placed on a stone base consisting of 4 courses of stone used due to the presence of inscriptions.

@unsplash/ahmed shabana

*source: https://www.egypttoday.com/

11

Project flexibility and dynamic delivery needed in countries with fragile situations

C

ookie-cutter, generalized approaches to programme design are unlikely to meet the developmental challenges of countries with fragile situations. The ability to continuously adapt interventions to evolving needs is critical for success. This was among the key messages that emerged from the learning event on the 2021 Annual Report on Results and Impact of IFAD Operations, (ARRI) which took place on 11 November 2021.

“In response to the challenges faced by countries with fragile situations, we need clear situational and institutional analysis, dynamic planning, flexibility and dynamic delivery on the ground.

We need credible partners whom we trust; we need budgets that are adapted to addressing the situations of fragility; and, of course, we need to empower local communities”, explained

United Nations agencies, programmes and funds, to distinguished members of IFAD’s Executive Board.

“In terms of a long-term project performance trend, the ARRI notes steady improvement in two criteria: environment and natural resource management, and climate change adaptation.

On the other hand, there have been signs of declining trends for the performance of the main partners, the government and IFAD, which the ARRI will continue to monitor in the forthcoming years”, stated Fabrizio Felloni, IOE Deputy Director, setting the stage for discussion.

include adaptation to climate change, efficiency, sustainability of benefits, innovation and scaling up. On the downside, efficiency remained relatively low.

The efficiency criterion, which focuses on how resources are converted into results, suffers from long delays in recipient government review and ratification of agreements, and slow pace of implementation and disbursement, mostly due to weaknesses and delays in procurement processes.

Targeting vulnerable groups remains a hallmark of IFAD’s development strategy. By investing in basic infrastructure and grassroots level institutions, IFAD-funded interventions have succeeded in reaching marginalized populations, women and young people in remote rural areas and fragile contexts.

Fumiko Nakai Dina Saleh

Dina Salah, Regional Director of IFAD’s Near East, North Africa and Europe Division.

Organized by IOE, the on-line webinar brought together a wealth of participants, ranging from practitioners of international institutions, including 12

Nigel Brett The performance of IFAD-funded projects continues to be solid, with the majority of project performance ratings moderately satisfactory or above for all evaluation criteria. Areas of improvement in the recent periods

“Support to inclusive natural resource governance and management comes out as a strength of IFAD in the 2021 ARRI. This is an example of addressing a driver of fragility, which is natural resource-related conflict. Another example of targeting of marginalized groups was found in Cote d’Ivoire, where the project

explicitly provided support to youth of ex-combatant groups,” highlighted Fumiko Nakai, Senior Evaluation Officer and lead author of the ARRI 2021 report.

The virtual webinar also featured a panel of technical experts who discussed ‘how to enhance IFAD’s support, engagement and development results in fragile situations’ – a very timely topic, given that two thirds of people in extreme poverty could live in fragile and conflict-affected situations by 2030, according to recent estimates. Expert panellists included Dina Saleh, Regional Director of IFAD’s Near East, North Africa and Europe Division; Norman Messer, IFAD Country Director for Chad and Mali; and Clement Banse, Chief Evaluation Officer of the Independent Development Evaluation of the African Development Bank Group.

between customary and statuary laws. A fourth is data scarcity and information flows. You are often grasping to get the data.

A fifth is inadequate resources.

A sixth is collaboration among partners. We all have institutional mandates, different interests on the ground, and are then confronted with the fine line between humanitarian and development interventions.”

In response to these challenges, IFAD looks at building blocks and conditions for successful intervention in fragile contexts.

Mr. Messer presented some of these, and discussed the importance of involving the entire rural community from the outset to mitigate conflict and address fragility.

Clement Banse

Norman Messer

The complex and multi-faceted nature of the challenges that countries with fragile contexts face gives birth to a variety of intertwined challenges, as Ms. Salah noted in describing the obstacles of designing and implementing development and agriculture-related interventions with a conflict lens.

“A first challenge - she noted is that fragility is a multi-faceted dimension. A second are the institutional frameworks, with a high turnover of public decision makers and weak policy making settings. A third is the dichotomy

“There are no cookie-cutter solutions and we cannot generalize, as the situations in countries in fragile contexts are quite different - he explained -. With that in mind, one of the building blocks that we need to be aware of is the level of decentralization that the country has because we need to work at the decentralized level. Another condition would be that the government accepts multi-stakeholder and civil society engagement. Also, service providers must already exist on the ground, such as local NGOs that we can work with – we can’t dream them up.”

“The effectiveness of support from all development partners to countries in fragile contexts

depends on the ability to continuously adapt interventions to the evolving needs. In the face of limited resources, development partners have to be selective in directing support to areas in which they have comparative advantage to maximize development impact”, Mr. Banse remarked, also referring to the evaluation of the African Development Bank Group’s strategy for addressing fragility and building resilience.

Roxanna Samii

Building on the findings of the ARRI, IFAD will need solid strategies informed by dedicated conflict and fragility analysis to address both the drivers and consequences of fragility.

“IFAD management is reviewing its engagement in fragile situations to improve our performance in building resilience and engaging better in conflict-affected situations. We are developing special initiatives, for example enhanced engagement in the Sahel and in the horn of Africa. We have a dedicated strategy for small island developing states, which will be going to the Board in April. All of these are fragile contexts. The lessons from these fragile situations are highlighted very well in the ARRI. Business as usual is not going to be sufficient in these contexts”, concluded Nigel Brett, Director of IFAD’s Operational Policy and Results division, speaking on behalf of Donal Brown, AVP for IFAD’s Programme Management Department.

13

Tailored approaches to evaluation design in fragile settings

FURTHER READING

I

n fragile settings, including those affected by conflict and violence, the approach and methodology to design an evaluation needs to be tailored to specific local circumstances. Moreover, evaluation – like development aid itself – can unintentionally exacerbate tensions in ways that can negatively affect people and institutions if care is not taken in designing and approaching it with conflict sensitivity. This argument is at the centre of ‘Evaluation in Fragility, Conflict and Violence’, a book published by the International Development Evaluation Association, and prepared under the leadership of Hur Hassnain, Senior Evaluation Advisor at the European Commission. The book was launched during two live events, on June 29th and July 8th.

Drawing on the insights and personal experiences of many evaluation professionals from around the globe, the book presents real time case examples and practical guidance that show a diversity of perspectives and experiences on sensitive issues when measuring change in unpredictable, complex, and violent situations. Mitigating these challenges can entail significant effort, and can sometimes require redefining the direction, purpose, or scope of the evaluation.

Fabrizio Felloni, Deputy Director of IOE, provided introductory comments during the launch event on July 8th, alongside Gonzalo Hernández Licona, Director of the Multidimensional Poverty Network at the University of Oxford, and member of IOE’s Evaluation Advisory Panel. Mr. Felloni’s remarks centred on how the principles and approaches presented in the book could be applied in a real evaluation scenario in rural development, and what could be the challenges and the way to overcome them.

Simona Somma, Evaluation Officer, who is among the co-authors of this volume, joined other evaluation experts for a panel discussion during the launch event on June 29th.

To emphasize the importance of context-specific evaluation approaches, in the book Somma shared her experiences in designing, piloting and implementing surveys and focus group discussions in Yemen and in fragile areas of India and Mozambique. Somma also contributed to the discussion on the evaluation learning function, with special focus on the dissemination of evaluation results to beneficiaries.

14

To learn more about Simona’s contribution to the book, and her motivation for getting involved in this ambitious initiative, Independent Magazine caught up with IOE’s Evaluation Officer.

Good morning, Simona.

Good morning, Alexander Simona Somma Congratulations on the publishing of this important volume; a complex effort no doubt. What pushed you to accept the challenge of collaborating on the preparation of this ‘how to’ guide to conduct evaluation in fragile settings?

During conflicts and pandemics and in fragile circumstances in general, monitoring and evaluation matter more than ever, and yet very often they are disregarded or perceived as a burden rather than as a way to build evidence and ensure accountability and learning. This is why I felt it was important and timely to contribute my knowledge of evaluation approaches, methods and processes, including in conflict affected contexts, to the preparation of this book.

I assume that the importance of designing a valid approach to evaluations in fragile settings – which you stress – is directly proportionate to the difficulty of doing so successfully, correct?

Yes, precisely. Especially in countries affected by conflicts and crisis, an in-depth analysis of the context is of the utmost importance in the design phase. Likewise, in these situations, the selection of appropriate evaluation methods and tools (including the innovative ones) for primary data collection become even more critical and fundamental for the robustness and validity of the evaluation.

Does this mean that ready-made, cookie-cutter methodological approaches will not suffice in these circumstances?

This is a key point that I felt strongly about addressing in the book. When you design an evaluation to be done in fragile settings, you cannot adopt the same approach and methodology that you would have used in a ‘normal’ context. You need to tailor the approach to the specific circumstances in which the evaluation will be conducted. Elaborating this in the book was something that required a lot of work among the co-authors, because we wanted to make sure to bring to the table something meaningful and useful to evaluation practitioners.

In addition to being involved in the chapters that focus on evaluation design, did you also contribute to those that address the implementation steps?

I am pleased to say that I was able to capitalize on my experience not only in designing, but also in piloting and implementing surveys and focus group discussions in Yemen and in fragile areas of India and Mozambique.

Moreover, I worked on the part of the book that concerns the organization of the field missions and the collection of data, as well as the part that is entitled ‘close the evaluation learning loop’ – a step in the evaluation process very close to my heart as it concerns the dissemination of the evaluation results to the beneficiaries.

Looking now at the volume, are you satisfied with how the final product has shaped out?

Yes, very much, especially since this book is the result of a collaborative effort, to which many evaluation experts provided substantive inputs. This endeavour entailed coordination work and frequent online meetings, especially in the final stages of the drafting. Despite the challenges of embedding in the final product and editing so many different inputs and perspectives, this was for me an extremely enriching learning experience and an excellent example of collaboration across institutions and colleagues with different backgrounds.

The book is available to order [here] and to download [here].

15

WWww

FEATURE STORY 16

LEARNING, GR

Not just anoth

ROWING, CHANGING LIVES:

her [profitable] day in the office

L

e a r n i ng: a word syn onymous wi th gr ow th. The pr ivate s e c tor has em b odi e d th is man tra , over th e ye ar s . Re venues , mar ke t shares and sharehold e r e a rnin gs have skyrocke te d f or bus ines s that have ‘ learne d to learn’, wh ile f ina nc ia l i ns t i tu t i ons have reape d the bene f i t s o f the learning mantra throug h as s e t grow th an d c ap i t al ac c u mulation.

17

IFAD is an international financial institution (one with an AA+ credit rating from Standard & Poor’s). At IFAD, however, learning is not about growing profits. At IFAD, learning means many things to many people. It means strengthening livelihoods, improving nutrition, increasing market access, and enhancing resilience to environmental shocks, to name but a few.

Simply put, at IFAD learning means investing in rural people to enable them to overcome poverty – that’s the growth the organization seeks.

If it is to continue sparking this kind of growth, IFAD must continue to learn. One-size-fits-all approaches won’t cut it. Adaptive, tailored and innovative solutions, and flexible project designs are increasingly being evoked, not least in the wake of the COVID-19 pandemic. To realize these, IFAD looks to become more agile, responsive and effective through virtuous learning processes.

This is where IOE comes into play. With its evidence-based focus on learning, transparency and accountability, IOE plays a key role in the successful uptake of lessons learned and best practices. For this reason, a thorough and rigorous evaluation and the production of a good report are not enough for an evaluation to be useful.

Evaluation is called upon to go one step further. Evaluation must inform decision-making and broaden the knowledge base both within and outside IFAD, bringing value added to the global rural development discourse in a timely fashion. Robust collaboration between IOE and IFAD senior management is essential for this to happen.

To deepen this discussion, Independent Magazine caught-up with IFAD senior management. We sincerely thank Donal Brown (Associate Vice-President, Programme Management Department); Jyotsna Puri (Associate Vice-President, Strategy and Knowledge Department); Dina Saleh (Regional Director, Near East, North Africa and Europe Division); Nigel Brett (Director, Operational Policy and Results Division, and Director a.i. Asia and the Pacific Division); and Sara Mbago-Bhunu (Regional Director, East and Southern Africa Division) for generously taking the time to sit down with us, and for kindly sharing their experiences, insights and perspectives.

Dina Saleh 18

Good morning, esteemed colleagues.

Good morning, Alexander.

Could you briefly describe your organizational role as it relates to oversight and evaluation?

Donal I am responsible for the quality of the results of all IFAD’s projects and programmes. I place a strong emphasis on delivering development results and therefore ensuring that we have an accountable, learning oriented, self and independent evaluation system, because to improve results we need to improve learning.

Jyotsna Evaluation, and especially independent evaluation, plays a very important role because it gives us an unbiased, independent perspective of the direction the organization has taken so far and it gives us a baseline to build strategies that can make IFAD future-fit. In this context, I think that Gilbert (NB: IFAD President), in his wisdom, brought in someone like me, who understands the evaluation perspective, to sit on the management side so that we could create a space within IFAD that appreciates engagement value between evaluation and management.

Nigel The Operational Policy and Results Division is responsible for aggregating results from self-evaluation products. We are responsible for the corporate results management framework, and for reporting to the member states annually through the Report on IFAD’s Development Effectiveness (RIDE). We support an enabling environment between IOE and PMD for self and independent evaluation.

Dina As regional director, my role is predominantly one of oversight in terms of deliverables, in terms of making sure that compliance standards are met. I look at certain performance measures and indicators, and I hold conversations around that to make sure that we evaluate those results properly.

Sara Regional and country teams have very specific roles regarding oversight. We have significant financial resources. We have to monitor the quality of the portfolio, and understand the impacts in terms of transformation.

The interface between evaluation and our oversight is structured. There is a whole framework of accountability. We have agreements at completion point and, as regional director, I have to respond to IOE’s recommendations. These are documented and I am accountable to the audit and evaluation committee.

Donal Brown 19

Within evaluation processes, do you believe that spaces for reflection and conversation generate the insights needed to ‘course-adjust’?

Donal I think that spaces for reflection and conversation help generate the incentives to course-adjust. However, to course-correct, the main issue is to ensure that we have the right evaluation products, and that we have them in a timely fashion. First, we need to be able to learn the lessons and channel them into new project design and new country strategies. In the past, many of the evaluation products have come too late. There has been too much emphasis on the rigour of the product. Second, we need to get the right kind of products. In the past, we have focused too much on accountability rather than on learning. We need the right balance between accountability and learning to allow us to course-correct.

Jyotsna Interaction during an evaluation is extremely important. In my own experience, interaction has never jeopardized the role of independent evaluation. I actually think it strengthens it, because it makes it far more relevant.

For me, this journey is far more important than the destination. It’s the interaction, the richness of the engagement during the evaluation that becomes important to inform staff both on the evaluation and on the management side. I don’t think that a document or an interaction for a couple of hours at the end of a process does full justice to the potential richness of evaluation itself.

Nigel This is one of the challenges that we have been working to address. Since 2016, we have put in place building blocks for better evidence-based decision making. We use M&E data for mid-course corrections whenever possible, and carry out stocktaking workshops and learning events. We also have complimentary tools, including the new restructuring policy that has enabled a much more methodical approach to project restructuring based on results and learning during implementation. A challenge is that sometimes programme management units (PMUs) see project design documents as blueprints. PMU staff need to be empowered and supported to understand that project designs can to be adapted to respond to lessons generated during implementation, and that these changes are sometimes essential to ensure achievement of project objectives.

20

Dina There are spaces for reflection and conversation, of course, but not only through workshops and meetings.

It’s very important that there is an ongoing conversation throughout the evaluation process. The thinking and reflection start at the very beginning. Sometimes it could be that you need to course adjust at the very beginning of the process, not at the end. It’s an iterative process. Sometimes we deepen our conversations and understanding with stakeholders during evaluations – it allows us to have a much sharper conversation and reflection on specific points.

Sara I believe they do. There are a number of workshops and missions during the project life-cycle that allow for conversations and discussion. When we see that our performance indicators are falling short of the standards we have set ourselves, then that triggers an implementation support mission to deepen our understanding of the problems, and then course adjust. The agility that has been built into the IFAD system allows us to proactively restructure, where the context might have changed or a particular component or sub component is no longer relevant.

“To course-correct, the main issue is to ensure that we have the right evaluation products, and that we have them in a timely fashion.”

- Donal Brown

“Sometimes we deepen our conversations and understanding with stakeholders during evaluations.”

- Dina Saleh

@Unsplash/Adele Payman 21

Broadly speaking, would you describe IFAD as a ‘learning-based organization’, or do you see a contradiction between accountability and learning?

Donal I see no contradiction between accountability and learning in the context of evaluation.

IFAD needs to be accountable, both in terms of our internal evaluation mechanisms and also of IOE. Donors provide resources for us to deliver impact and results. We have to be accountable for that. At the same time, to do that, we have to be able to learn. The new development effectiveness framework of IFAD puts learning as a key cross cutting priority. The new leadership of IOE has been very good at trying to get that balance right. The 2021 ARRI report was an example of a much better product in terms of balancing accountability with learning. The revised evaluation manual, which comes out next year, will also stress the importance of learning from evaluation products. There is still a long way to go, but I am quite confident and reassured that the much stronger relationship that I have seen with IOE in the past year, and the enhanced focus internally on learning, has put us in the right direction.

Jyotsna I have never thought that there is a trade-off between accountability and learning. So long as there is an institutional commitment to credibility, to learning, to what the data is showing us, accountability and learning can form a happy marriage. Especially with the advantage of big data, machine learning, real-time data, this marriage is further strengthened because not only you can know at the end if something worked, but you want to be able to adapt as you go along. It is unethical to spend money, today, on something that I might be told did not work, in five years’ time. We are affecting lives, every dollar is becoming so scarce, and thus I need to know what is the biggest impact that I can make now. It is myopic and almost dystopic for us to think that people at the opposite ends of an argument should not be speaking to each other in real time. I think that this leads to dysfunctional organizations. I am hoping very much that with Indran’s [NB: IOE Director] entrance into IFAD there is a greater appreciation of this dialogue process and this engagement, and a greater appreciation of what the potential of a healthy environment can be.

The time for learning and accountability is now.

Nigel Brett 22

Nigel I think the adaptive management approach that IFAD is moving towards really represents the intersection between learning and evidence-based decision making.

Lesson learning for its own sake is not useful. What is important is that lessons need to be applied. When they are applied you have this proactive approach where you are constantly improving based on evidence. Currently, you could characterize IFAD more as a results and evidence-based organization, and I think we need to transform into a more learning-based organization. More effort is needed to ensure learning is prioritized. The problem is that everybody is overstretched and working flat out. There is enormous pressure just to deliver. The challenge for IFAD is to enable staff to carve out time to reflect, to learn and to apply learning – this should be one of the priorities for the organization moving forward.

Dina They are mutually reinforcing. IFAD by its genesis is a learning-based organization.

Through its growth, IFAD has been very focused on piloting and testing new solutions and innovations. If you look at the structure, where we have a country director that deals with several parts of programme management, you can see that IFAD offers a lot of opportunities to learn. We derive many lessons when we pilot innovations at small scale.

Sara IFAD has increasingly become a learning-based organization. It’s a journey. The fact that we have an independent office of evaluation helps us in this journey. We have a big quality assurance process that ensures that we take on board the recommendations from IOE. This accountability mechanism is critical in supporting and fostering our continued growth as a learning-based organization, including insofar as it forces us to have critical conversations and dialogue around what we have to change.

“It is unethical to spend money, today, on something that I might be told did not work, in five years’ time. The time for learning and accountability is now.”

- Jyotsna Puri

“The challenge is to enable staff to carve out time to reflect, learn and apply learning – this should be one of the priorities for the organization moving forward.”

- Nigel Brett

Jyotsna Puri 23

To what extent do you think that internal and external stakeholder buy-in weighs on the extent to which evidence-based lessons and recommendations are internalized at the programmatic level?

Donal Stakeholder buy-in, whether internal or external, is key. If you don’t have stakeholder buy-in you are not going to learn the lessons. Lessons learned is a phrase too easily used and too little understood. Probably 90% of what we do is actually identify lessons rather than learn lessons. To learn a lesson, you have to identify the lesson and then act on it and something has to change as a result. That’s where the stakeholder buy-in comes in. If you don’t have it you will identify lessons but nothing will change. One of the key elements of the development effectiveness framework is that results need to be communicated in a timely and transparent way, particularly to our key external stakeholders, government and development partners. One area where learning will become even more evident will be in the design of country strategies. We really need to build better evidence in there. A COSOP is not an IFAD country strategy, it is a shared strategy between governments and IFAD, so stakeholder buy-in has to be key.

Jyotsna It’s really important that we have a very alive, responsive and aware external set of stakeholders that are holding us to a high standard of credibility and honesty. Frankly, as an international organization, I don’t think we have a choice. We do need to hold ourselves to a very high level of credibility, transparency and honesty. There is no going around this. If I were an external stakeholder that is the standard that I would hold IFAD accountable for – and they are.

Nigel Stakeholder buy in is absolutely essential, both internal and external. For this reason, every year we conduct corporate and regional stocktakes to assess portfolio performance and identify areas where we need to make improvements. The resulting reports are discussed with the membership. In parallel, we also conduct impact assessment on at least 15% of the portfolio. This data is also shared. Under the new development effectiveness

24

framework, we are putting more emphasis on stakeholder consultation and engagement, and ensuring that governments are empowered with more data for evidence-based decision and policy making. To be taken seriously, it is extremely important to be able to engage with stakeholders with credible high-quality data.

Dina It does weigh. We work in countries that in some cases are politically charged. Very diverse countries. You need to know how to navigate those countries. This does not mean that you shy away from producing results and speaking in those countries, but one has to understand the ‘how’. It’s not what we have as evidence but how we have conversations with political stakeholders that gets the buy-in. We have had some very difficult circumstances.

Sara We have a big task ahead of us. We need to improve the data quality at the M&E level. This has to be government driven. We need to look at how we can leverage digital technology to more effectively improve this. Recently, we have tried to introduce triangulation at local level as a way to get direct feedback from beneficiaries, who often do not have digital tools. We have the COSOP, which is a very interesting tool, and which offers all country actors an opportunity to engage and to give us their feedback on how we are doing on our priorities and strategies. These mechanisms allow for critical dialogue.

“It’s not what we have as evidence but how we have conversations with political stakeholders that gets the buy-in.”

- Dina Saleh

“To be taken seriously, it is extremely important to be able to engage with stakeholders with credible high-quality data.”

- Nigel Brett

@Unsplash/Alejandro Rugama 25

Do systems exist to ensure that evidence-based lessons and recommendations are mainstreamed into IFAD strategies, at the corporate level, and into projects at the field level? If so, to what extent do you think these systems are successful?

Donal Systems exist, in principle. What’s really important, more than the systems, and what’s made a difference so far, is the right choice of the evaluation type, scope, focus area and approach. If we have the right products, then the systems exist. For example, the 2018 evaluation of the financial architecture was instrumental in helping IFAD make substantial reforms in its financial policies, and the outcome of that is that we’ve got the AA+ rating from Standard and Poor’s. Similarly, the decentralization CLE. Again, the right product immediately triggered learning. In 2022, we are putting in place an online tracking system to ensure there is better use of evidence for follow-up and to facilitate continuous feedback.

Jyotsna Systems consist of people, technologies, processes and methodologies. They contain all of that. Do we have the people who appreciate that real-time learning and accountability should occur? The answer is “yes”. Do we have the processes, the technologies and methodologies? The answer is a bit less than a full “yes”. For example, I want our IT systems to show me in real-time and at a glance every aspect of a project, and I want that to be available to everyone across the system. Can I integrate in real-time all of the GIS data that helps me to understand what the targeting is and to appreciate if the project design has taken onboard all of the vulnerability assessments? We are still building those systems. The same for methodologies. Impact assessment methodologies were built in a world were data was always very sparse and so you waited for data to be available. Now, we have high frequency and highly dense data and there is nothing in our impact assessment methodology that takes that onboard. We have got to update our methodologies.

Nigel We certainly do have systems in place that facilitate the mainstreaming of evidence. At the field level, and based on personal experience, I can say significant learning happens during supervision missions or midterms reviews. This learning feeds directly into on-going projects, and into the design of new projects. We also have portfolio stock-take exercises, on a yearly basis, where themes are looked at in detail and corporate level lessons are generated, followed by action plans to ensure lessons are applied in practice. At the corporate level, a lot of work has been done to put in place the infrastructure for lessons learning. We have a state-ofthe-art Operational Results Management System (ORMS), where all the data sets and the narratives of project implementation are made available on-line. This is public information. Another tool, the President’s Report on the Implementation Status of Evaluation Recommendations and Management Actions (PRISMA) will also be brought online soon and made public. This will increase accountability and visibility of the follow-up on IOE recommendations. Through continuous upgrade and integration of these systems, there are going to a range of new opportunities to ensure that recommendations and lessons coming from supervisions and evaluations feed back into the design of new country strategies and projects.

Dina Yes, it has been evolving. It’s five years since we introduced the country strategy (COSOP) completion review processes. These have allowed us to look at strategies, take stock of lessons learned, inform stakeholders, and course adjust. However, there is a lot of reliance on manual systems. We could be more successful if we had automated systems. In this regard, NEN is piloting an artificial intelligence project where we can derive automatically lessons learned. We are at the stage where algorithms are almost ready and we will hope to test the system by the end of this year, or the beginning of next year. It was one of those projects that was derived from IFAD’s corporate innovation challenge. NEN took it on board to develop and test it. If successful, we could scale it up to the rest of the house.

Sara Yes, IFAD has made huge progress in putting in place systems. For instance, IFAD maintains a database of agreed IOE recommendations, tracking and reporting of progress and implementation, and this is done 26

through the PRISMA. Looking at this system, it is led from the highest level in IFAD and that forces all levels of management to take IOE’s recommendations very seriously. At the project and local levels, we have the operation results management systems in place, and that’s another database that looks at lessons learned, monitoring, tracking and scoring of results. This ensures that, at the Associate Vice President level, there is an opportunity to see how the results that are coming out are tracked in relation to IOE’s lessons and recommendations. At the regional level there are different platforms, such as the portfolio advisory meetings. The systems are in place. The cycle is there. Everything is well documented. The evidence is there. Can it be improved? I assume it can.

“The systems are in place. The cycle is there. Everything is well-documented. The evidence is there. Can it be improved? I assume it can.”

- Sara Mbago-Bhunu

“What’s really important is the right choice of the evaluation type, scope, focus area and approach. If we have the right products, then the systems exist.”

- Donal Brown

Sara Mbago-Bhunu 27

Looking ahead, what do you see as being the main opportunities for evaluation to further permeate an evidence-based learning culture across IFAD, and for this culture to trigger life-changing impacts for our stakeholders in the field?

Donal Firstly, the world, IFAD and the context in which we operate are changing very fast. One of the key opportunities for evaluation is to be agile, quick, nimble, and to get the right balance of ‘quick and dirty’ versus ‘rigorous and time consuming’. It’s important that we shift the focus of evaluation towards the higher value products and services that can take into consideration and respond quite quickly to changes in IFAD, in the development context, and in the countries where we work. We must make sure that we feed lesson back quickly into the system. Second, I can’t emphasize enough the much stronger collaboration we now have between management and IOE. There is collaboration in defining the timing and structure of the product-mix, sharing data sources, and ensuring the relevance of the evaluation products. This does not compromise the independence of IOE. On the contrary, if we are going to look at the learning function much more, then we need to be able to discuss, debate, and work more closely. I’ve seen really, really strong progress in this area over the past year.

Jyotsna We need to be aware that evaluation is really important for an individual institution, but it’s also a global good as well, very much like climate and health. The more high-evidence we can produce, the more we can help others to become better. We should underscore that there is a service that we are doing to humanity when we produce high quality, credible evaluations. In this context, I would like to see responses to the question “how much did something work?”. For decision making, what is really important is to understand trade-offs. A policy maker is not necessarily thinking “should I introduce cash transfers?”, but rather “should I introduce cash transfers at the cost of setting up a completely separate insurance mechanism?”. So, the overall impact that a cash transfer is making versus an insurance mechanism is the trade-off. To understand that, I need to know how much difference a cash transfer made, and how much difference should I expect an insurance programme to make in the overall resilience of a targeted population before I can start to make those decisions. If evaluation wants to stay relevant in this space, it needs to answer the “how much?” question far better because that’s what will help us to understand cost-effectiveness and trade-offs.

Nigel IFAD will need to adapt its tools to its new decentralized structure. With more staff decentralized we need to make sure that we keep a bridge between the field and headquarters to ensure that the decision

“One of the key opportunities for evaluation is to get the right balance of ‘quick and dirty’ versus ‘rigorous and time consuming’.”

- Donal Brown

28

“We should un that there is a that we are do manity when duce high qua ble evaluation

making that is happening at HQ is informed by learning that is happening in the field. Systems will play a key role in facilitating the collection, analysis and utilization of evidence for decision making. Much of our core business has already been put on line, and we will continue this trend moving forward. We are working towards bringing our country strategies online. This will enable better tracking, monitoring, reporting and learning from the country programmes including from non-lending activities such as policy engagement. Perhaps in closing, I would like to take this opportunity to again underline the importance of enabling staff to have time to think, and to prioritize learning. We need to create space for this to happen.

Dina IOE has brought on this new approach whereby it seeks not to be a disciplinarian, but to feed useful lessons into our work. In the past there was this preconceived bias and prejudice against the evaluation process as something that was going to come and uncover what we got wrong. Now there is a much more open culture towards evaluation. I know that country directors look forward to evaluation prior to doing a new strategy, because there are things that they may miss out, and also to further the difficult conversations with local stakeholders. Going ahead, the main opportunities I see are to make sure that evaluations are timed in such a way that they feed into new programming and strategies. It is also very important to hold learning events jointly with the countries. One has to make people feel that they own the solutions. For evaluation to further this learning culture, there has to be a value addition, without additional workload. The learning needs to be complimentary to the work we do, not additional to it. We need to provide out of the box, state of the art, avant garde solutions, less ‘scientific’ and more context specific.

Sara I see several. One is of course the decentralization process and the emergence of the regional offices.

Also, the move of country directors at the country level, with the increased country footprint. We can leverage this stronger presence to initiate a new type of dialogue with stakeholders on the ground. The second aspect is to learn more about the different instruments that IFAD is utilizing, through its new business model, and understand which is more effective. We have invested in expanding our menu of services.

Maybe we can hear more form IOE about the impact of these services, since we want to move towards more transformational programming under IFAD 12. I see lots of opportunities for IOE to provide credible evidence not only of our successes, but also to guide us in terms of where we should go.

Thank you very much Donal, Jyotsna, Nigel, Sara and Dina.

You are welcome, Alexander.

nderscore a service oing to huwe proality, credins.”

“I see opportunities for IOE to provide credible evidence not only of our successes, but also to guide us in terms of where we should go.”

- Jyotsna Puri

- Sara Mbago-Bhunu

29

@Unsplash/Daniele Levi Pelusi

New approach to mass mobilization of community labour for restoration of degraded natural resources in Ethiopia

T

he Community-based Integrated Natural Resources Management Project (CBINReMP) in the Federal Democratic Republic of Ethiopia was the first to operationalize and build upon Government guidelines related to mass mobilization of community labour for restoration of degraded natural resources. The latest impact evaluation report released by IFAD’s Independent Office of Evaluation (IOE) captures this innovative aspect of the project, and discusses it alongside other important findings.

The impact evaluation used a quasi-experimental approach and combined econometric and qualitative techniques to attribute the impact on beneficiaries of the programme. In addition, use was also made of geo-spatial analysis to assess changes related to selected biophysical indicators.

Approved by IFAD’s Executive Board on 30 April 2009, and officially closed on 31 March 2019, the CBINReMP project set out to enhance access by poor rural people to natural resources, and improve agricultural production technologies, mainly through the adoption of sustainable land management practices. The total project cost was US$27.31 million, of which IFAD financing was US$13.12 million, including a highly concessional loan of US$6.6 million and a Debt Sustainability Framework grant of US$6.6 million.

30

In addition to acting upon the Government’s guidelines, the CBINReMP went one step further by providing an unprecedented incentive scheme in the form of rights to cut-and-carry fodder from communal land. Smallholders also benefited from the innovative approach of including land certification as part of sustainable land management – an aspect that also contributed to women’s empowerment. Moreover, wherever family land was registered, co-ownership was assigned to both husband and wife. This guarantees equal rights and protects women’s rights if their husbands divorce them or pass away. Indirectly, land certification activities also reduced land degradation and decreased communal land pressure by supporting farmers’ investments in their plots.

The evaluation found a statistically significant increase in incomes and dietary diversity of households that participated in higher number of project activities. However, for the rest of the beneficiaries the results were not different from those of non-beneficiaries. The limited impact on incomes is also related to the nature of natural resource management projects that have longer gestation periods, and the low investment of the project per beneficiary household.

The project fostered an effective system of communal pasture governance through informal

community by-laws, and successfully supported adoption of climate-resilient farming practices, including the diversification of farming systems through fruit tree planting in a small number of micro-sheds. Climate change adaptation practices and technologies for on-farm production improvement were also successful, but should have been implemented in all the 650 sub-watersheds.

Opportunities to address the long-term problem of overgrazing on communal lands were missed, as originally envisaged policy and regulatory reform activities were not implemented. At the community level, the project did not invest in supporting institutions such as watershed management committees and land use committees – a requirement to ensure women’s representation. Instead, planning was done through a top-down approach led by the Government, and implementation was carried out through local extension systems that had little or no capacity. In this context, income-generating activities are expected to be unsustainable in the absence of marketing analysis, clear rights of resource usage and sufficient private sector engagement.

In the future, the impact evaluation recommends adopting a master plan for integrated participatory watershed management, to enable the involvement of all stakeholder groups in the management planning and implementation processes. Drawing on other lessons, the report also suggests to align the length of a project’s duration with the time frame of watershed management plans in order to see the effects on beneficiaries’ income, and to embed monitoring and evaluation elements that can better facilitate impact studies in the design of watershed management projects.

In conducting the evaluation, IOE collaborated with the International Food Policy Research Institute (IFPRI).

@Pixabay/microscopic

Africa celebrates fashion*

T

his year’s African Celebrates Fashion Reception (AFR) was held in Addis Ababa, from 21 October 2021. The AFR commemorates African traditions, art, culture and cuisine since 2013.

This year’s event gathered fashion designers, dignitaries, ambassadors and media from over 30 African countries in a celebration of cultural diversity expressed through the artistic fashion discipline, with a theme “Art, fashion and culture.”

The aim of the event was to expand the potential power of fashion as a tool for fighting poverty in Africa by creating wealth through the empowerment of women and youth in the various fashion vocations. It is also set to bring in lots of tourists, hopefully reviving the tourism industry that has taken a huge hit during the pandemic. The event also looked to create self-reliance as it creates jobs across the continent for tailors, creative directors, textile designers, models, cutters, pressers, fashion photographers & writers, hair stylists, make-up artists and alike.

“This is a great initiative. As Africans, we have such a rich culture that the whole world deserves to see; it is a great opportunity to showcase what sort of potential lies within the continent. It is also a great opportunity to show the rich heritage of Ethiopia to the extended African and global tourists the event is set to bring in,” said Lelise Duga, Commissioner of Oromia Tourism Commission. *source: https://www.thereporterethiopia.com/ 31

Time for evidence to contribute to gender equality and women’s empowerment

“I

t is time for evidence to contribute towards achieving gender equality and women’s empowerment.

We are called upon to light a decision-making path for our leaders”, stated Dr Indran A. Naidoo, Director of the Independent Office of Evaluation (IOE) of IFAD, as he opened the United Nations Food Systems Summit (FSS) parallel session on ‘evidence pathways to gender equality and food systems transformation’, on 27 July 2021.

Indran A. Naidoo

Gender inequalities negatively affect food systems, in spite of the fact that women represent almost half of the total agricultural workforce in low-income countries, as underlined by H.E Ms. Céline Jurgensen, Ambassador and Permanent Repre-

atic gender inequalities remain leading women’s contribution to be often undervalued.”

The attention paid to women’s empowerment by international policymakers has fallen short of making the necessary impact.

“Policies adopted to close the gap between opportunities for women and men have not been enough. Experience has shown us that achievements made thus far have been isolated because they are not holistically approached. In many cases, women have entered into what is conventionally seen as ‘men’s world’. The challenge is now for men to enter into what is conventionally seen as ‘women’s world’”, stated Ms María de los Ángeles Gomez Aguilar, Alternate Representative of Mexico to FAO, WFP and IFAD.

María de los Ángeles Gomez Aguilar

Céline Jurgensen

sentative of France to the United Nations in Rome. “Whereas women are key actors in every part of the food chain as farmers, entrepreneurs, processors, traders and consumers, system32

With a global pandemic causing havoc to food systems worldwide, evidence is needed to deliver viable solutions to this paradox as never before. “In these times of conflict and pandemic-driven crisis, evaluation has never been more relevant and necessary. As members of the evaluation community, it is our duty, our responsibility to share findings from evalua-

tions so that we learn and act; so that we contribute to solutions. We provide the scientific level of objectivity needed now more than ever”, Dr Naidoo underscored in this regard. H.E. Ms Alexandra Bugailiskis, Ambassador and Permanent Representative of Canada to FAO, WFP and IFAD, reiterated this point by emphasizing that “evaluation makes sure that we can invest effectively and make good returns”.

Alexandra Bugailiskis

Co-organized by IFAD, WFP, FAO and CGIAR Advisory Services, the gender-focused FSS parallel session brought together over one hundred and fifty among high-level representatives of the diplomatic community, international organizations and civil society for a discussion facilitated by Ms Allison Smith, Director of CGIAR Advisory Services. Discussions focused largely on the findings of a joint report of the four agencies that summarises learning and experience from 47 evaluations addressing food systems-related programming since 2017. The result of this collective effort is a document that highlights nine evidence and experience-based lessons, which will enrich gender-responsive and transforma-

tive approaches to food system programming.

At the core of the report lies the recognition that if systemic gender inequalities are to be addressed, then food systems programming needs to move from trying to ‘address the symptoms’, to tackling the root causes. That is, to shift from a lens of ‘equal participation’ of men and women, to a vision of transformative change and reform. “The focus on trying to directly address and change general norms and values is important to address gender relations and empowering women”, stated Mr Béla Teeken, Associate Gender Scientist at IITA Nigeria in this regard.

Béla Teeken

Creating alliances for change, monitoring progress, building local capacities and scaling-up initiatives are among the cobblestone that pave the way to realizing this vision. “We do not get very far if we do not have the involvement of national and local actors. Partnerships are key.

We need more collaboration with women-led organizations, civil society organizations. We need to build alliances around gender, food security and sexual reproductive health”, stated Ms Jette Michelsen, Deputy Permanent Representative of Denmark to FAO, WFP and IFAD. In the same vein, Ms Maureen Munjua, Country Representative of Tanager in Kenya further noted that “each organization has a footprint, and this footprint

can be used as a basis to scaleup our individual and collective responsibility and commitment to improving gender equality and nutrition”, while Ms Saadya Hamdani, Director of gender and inclusion at Plan international Canada, stressed that “coherent indicators must be set so that we have consistent monitoring of results as we go along our programming.”

Jette Michelsen

Gender transformative approaches represent a shift in how we think about and approach gender. Engaging men and women together as agents of change remains an uphill battle – the importance of which cannot be overstated.

Andrea Cook

“Gender equality and women’s empowerment is a game-changer, a cross-cutter, a catalyst at the core. Activities that support gender equality and women’s empowerment can spur progress towards all other sustainable development goals. Not only will its achievement improve nutrition, health and education outcomes for women, men, girls and boys, but it will also bring wider benefits for families, communities and nations”, summarized Ms Andrea Cook, Director of Evaluation at WFP in her closing remarks.

@Unsplash/Jackson David

33

ONE SMALL STEP FOR THE U.N. One giant leap for the audit and evaluation community

A

udit and evaluation: worlds apart? Arguably, yes. Different paradigms, no doubt, different approaches to research and investigation, without question. The separation of the two functions ensures their independence. Therefore, said separation needs to be safeguarded – one might hastily conclude. But is working in silos really the only way? Maybe not. After all, both audit and evaluation are essential instruments for accountability, and both play crucial roles in the oversight of organizations such as IFAD. And not only that.

If we look at some of the latest literature, research suggests that ‘enhanced oversight’ attained through collaboration among the two functions can promote a holistic three-dimensional view of performance, and create synergies between practitioners from different backgrounds. So where does this leave us? Are we to marry or forever divorce the two?

To answer this thorny question and tackle this important topic head-on, IOE hosted an event on 24 August to discuss the potential for crossover between the two functions, and their respective profes¬sional groups, as well as related incentives and potential barriers. IOE invited Helge S. Osttveiten, Director, Office of Audit and Oversight (OAI) of UNDP; Ana Rosa Soares, Chief of Section, Corporate Evaluation, Independent Evaluation Office (IEO) of UNDP; and Ray Rist, co-creator and co-director of the International Program for Development Evaluation Training (IPDET), and former president of the International Development Evaluation Association (IDEAS).

Together with IOE Director, Indran –former UNDP IEO Director –, Helge, Ana and Ray plaid a prominent role in the ‘Joint Assessment of the Institutional Effectiveness of UNDP’ report. Published in 2017, this first-of-a-kind effort was conducted jointly by IEO and OAI – something that had never been done before [access the full report: here] Independent Magazine caught up with Helge, Ana and Ray to get an insider’s perspective into what proved to be a ground-breaking exercise, in more ways than one.

Good afternoon, esteemed colleagues.

Good afternoon, Alexander.

Helge S. Osttveiten 34

Ana Rosa Soares

Ray Rist

I have a feeling that this is going to be a long and multi-faceted story, with lots of twists and turns. So, let’s start from the very beginning. How did the idea for a joint exercise come about?

Ana In 2013, UNDP started thinking about institutional effectiveness and how to measure it as a central focus of the organization’s strategic plan. In this context, in 2015, the Independent Evaluation Office, and the Office of Audit and Investigation agreed to embark in an approach to oversight that would be different. The objective of the exercise was to assess the extent to which policy and organizational measures enhanced UNDP’s ability to deliver higher quality programmes through results-based management and improve UNDP’s institutional effectiveness.

From a layman’s perspective, might I ask why was the joint nature of this exercise such a big deal?

Ana Historically, both offices have worked according to their own plans. A joint assessment implied working across the traditional professional boundaries that we were very much attached to. It implied developing synergies of approaches, rationalizing the resources that are always scarce, and producing a coherent message to the UNDP management on this critical subject. It was not an easy task – but it proved to be a very rewarding one.

Before even opening the report, the first thing I noticed was that the title of the report refers neither to ‘audit’ nor ‘evaluation’. Instead, you use the term ‘assessment’. What was the thought process behind this choice?

Ana What to call the exercise was the first challenges that generated very heated discussions. Every time we started to mention that we were going to have a joint audit and evaluation people’s eyes opened and they were really scared. In the end, Indran and Helge decided to call it a joint assessment. The time that it took to reach an agreement just on the name is the reflection of a much more significant challenge that needed to be addressed, which were decision rights, communication channels and protocols. That’s when Ray [Rist] had to come in and really help us come together as a team.

I guess that the phrasing of the title was not the only ‘wording issue’ you faced. From dealing with sensitive language to safeguarding independence, how far apart were the two offices when the time come to put pen to paper?

Ana Writing the report and dealing with the sensitive language and the aspect of independence was a challenge.

Audit reports to the UNDP Administrator, while evaluation reports to the Executive Board. As a result, the auditors’ language was different to the evaluators language, generating significant disagreement and severe delays in finalizing the product. We had to go back to the Directors to decide what and how would get communicated.

The Directors were really very much hands-on in helping the team in finalizing the report so that we could say what we needed to say, but were able to do so in a way that did not attack but rather welcome contributions from the organization.

35

To what extent might these communication-related difficulties reflect deeper lying challenges, rooted in different approaches to the overall scoping and design of the exercise?

Helge When it comes to audit and evaluation, it is too simplistic to say that they have different scope. It all depends on what kind of audit you do. A lot of the work we do is in the is in the performance auditing area and there are huge overlaps with evaluation, both in terms of scope – because we all address issues of results, efficiency and economy – and also in terms of methodology.

Ana In this particular case, there were three pillars of the strategic plan that we were assessing, and results-based management (RBM) was one of those pillars. However, we could not assess that pillar on its own. That significantly influenced the way that we needed to scope the exercise and define questions for the assessment. Audit understood RBM much more aligned to monitoring and evaluation and reporting tools for compliance, while evaluation needed to assess the understanding of RBM in the organization as a strategy, not only to manage results but to promote learning to improve results and ensure institutional effectiveness. As a result, auditors wanted to focus on normative questions, while evaluators wanted to address descriptive and cause and effect questions.

If we put aside these challenges for a moment, one thing that both offices have in common is the pursuit of credible evidence. Was at least this something that brought you together and helped you to gel as a unified team?

Ana To be quite honest, reaching a mutually agreed understanding of what constitutes credible evidence was in itself a challenge. This is not surprising though, given the different data collection methods and strategies for triangulation of the two offices. Here, the evaluators learned a lot from the auditors. Evaluators in our office usually favoured using surveys to close triangulation. However, having the surveys early in the process was extremely advantageous to help further limit the scope and to focus additional data collection efforts on what still needed clarity, allowing more time for the ‘why’, ‘what’ and ‘how’ questions. Nevertheless, the surveys could only be used to approach staff, and as evaluators we still needed to broaden the consultation and reach out to the field to engage with partners, governments and beneficiaries, which is something that auditors don’t do so often and broadly. That compromise was a challenge, but we arrived at a decision to go to the five regional hubs, bring staff from ten different countries from each region for an in-depth focus group, and then consult partners and beneficiaries in five countries where the focus groups took place. These interactions proved particularly relevant.

While you were busy grappling and coming to terms with so many challenges within the team, I wonder how people outside the team – such as the recipients of the exercise – felt about the initiative. How did UNDP staff look upon this effort, and what was the reaction of UNDP senior management?

Helge It is correct to say that senior management was not super happy with this exercise. It’s also correct to say that in the beginning they actively pushed back on the whole thing. They felt in the beginning that we were ganging up against them and their response was that they needed to fight back. We had some interesting meetings with senior management along the way. They ignored us when we reported initially. However, I am almost certain that if you had asked them at the end of the process you would have received an honest “we were wrong and we see the benefits coming out of this exercise”. Maybe not all of them, but many of them saw that this was a good exercise for UNDP. It was a learning experience for senior management, a new thing for them.

Ana 36

The joint assessment caused significant fear and discomfort among managers, especially at the regional bureau and headquarters levels. Management first asked the exercise to be postponed very strongly. Then they lobbied to have the scope of the exercise reduced. The UNDP administrator refused to give an interview – she was running for the seat of UN secretary-general at the time and there were fears of the results of this assessment. The staff that we interviewed commented that they had been alerted to be careful of what was told to the assessment team as the report could come to affect the funding of the organization and therefore their jobs. Until the very last minute when the report was first discussed with the executive board, key management officials refused to attend the discussion of the report. So, when the report was formally presented to the executive board, in 2017, the management refused to present a management response to the recommendation.

After so much hard work, and underlying opposition from senior management, what was the final outcome?

Helge It was a happy end. The response that they [senior management] gave us in addressing the recommendations, eventually, was very constructive. My point is that if we had not introduced it we would not have given them the opportunity to initially be a bit scared, but eventually see this as an opportunity for institutional learning.

Ana Regardless of all that [opposition], the assessment was the richest in evidence to this day, highly acclaimed by the executive board and others, and the most read of all products we ever produced. Eventually, management produced a management response and implemented changes. The current administration re-stated its commitment to change and to implement the recommendations of the report. The report made a very strong statement for transparency, calling leadership to promote a results culture that encouraged critical reflection of success and failure as having value for organizational learning to improve effectiveness and stimulate innovation.

If I asked you to identify one defining learning element of this experience, what would be the single most important aspect that two offices took away from the joint assessment?

Ana While there were challenges, the experience of conducting a joint assessment with elements from audit and evaluation was a unique learning opportunity for UNDP. It helped evaluators to look more closely at quantitative analysis and normative questions, and helped auditors to look at performance from the cause and effect angle to people and results. Audit and evaluations complementing each other can better feed the other side accountability and learning engines of an organization to ensure effectiveness.

Helge There is a tendency in the audit community to rely very heavily on numbers and I think that is a great asset.

What we learned is to use more effectively information that is coming through interviews and surveys. This synergistic approach created a product that was rock solid.

In the same vein, if I asked you to identify one defining key to success, what would be the one defining element that you think allowed the assessment to see the light of day?

Ana The assessment was challenging at multiple stages, not least of which was the apprehension of UNDP management of the exercise. What made it possible was that both Directors from both offices, Indran and Helge, were convinced that the joint approach would produce a stronger report and generate much more relevant lessons for the organization.

37

Helge It was a marriage. There were challenges along the way, but we were able to deal to them in a very constructive manner which meant that we were able to address all the concerns.

Ray What is most important to me in this effort is that both groups learned: Indran’s group learned and Helge’s group learned. That’s a powerful statement to make. When we came together, it was a rough marriage for a while. But, in the end, they learned, and both are stronger for it now, having been through that.

Now that a couple of years have passed, what’s left of this tremendous effort? Has this experience affected the audit and evaluation communities, the UN system, and the two offices involved?

Ray It was not an easy effort, but it was an important effort, one that has broken lots of new ground, bringing new perspectives in the UN system about the way that work can get done by different offices together. A dramatic effort at breaking down silos – which I think is one of the longest-term consequences that has come of this.

Ana What followed the joint exercise was an avalanche of questions about the advantages and disadvantages of joint audit and evaluation exercises. There were not many conclusive answers, but we certainly need to continue this discussion. In fact, since this exercise, in 2015, both our offices try to conduct missions together, when possible, and try to exchange information to contribute to each other’s reports. A lot of the lessons we learned I am still using now as we work on other joint evaluations with other UN agencies in different types of exercises.

This is useful not just for audit and evaluation, but also for any kind of joint exercise in the UN. Any final thoughts or words of wisdom?

Ana It is imperative that audit and evaluation work together in some ways to at least better coordinate workplans, ensure synergies and avoid duplication of efforts and wasting of resources. After all, audit and evaluation are part of a continuum. The autonomy of audit and evaluation functions is both historical in nature and institutional in form. It is clear that the nature of the responsibilities of international organizations like UNDP and IFAD are changing with the SDGs, and a recognition that the status quo of working in silos is no longer appropriate is obvious. The question, therefore, is not if but how to integrate: the marriage has to work.

Ray What strikes me about the work that was done is that it was a tremendous effort at organizational learning that allowed both paradigms – audit and evaluation – to come away stronger and with new knowledge. This shows me an organization that is learning, that is vibrant. This is something that we often don’t see in many organizations, where the focus is on routine as opposed to creating new insights.

Thank you very much for your time.

You’re welcome.

38

CENTRALITY OF EVALUATORS UNCERTAIN IN POST-COVID WORLD

“T

he fact that there was almost an absence of strong reaction from the evaluation community in terms of the COVID-19 crisis means that when things change, and this passes, evaluators may not be as centre-stage as they were pre-crisis.” Dr Indran Naidoo, Director of the Independent Office of Evaluation (IOE) of IFAD, shared this reflection with eminent experts of the evaluation community, during an on-line discussion which took place on 1 July 2021.

Commenting on the nature of transformation in evaluative practices, Dr Naidoo highlighted that as the pandemic forces geographic and institutional borders to crumble, evaluators need to operate in a much more sophisticated way. In the wake of the pandemic, many evaluation outfits went into dormancy at government and international level. Fortunately, those units that remained in the forefront have been able to expand the spectrum of evaluative methodologies. The field is opening up, and is becoming much more transdisciplinary.

The on-line event, organized with the support of the David and Lucile Packard Foundation, brought together thought-leaders working at the interface of transformation and evaluation to share, document and map the initiatives, concepts, methods and practices that may help evaluation practice to accelerate large-scale changes and transformations the world urgently needs. Insights centred on utility, complexity, and whether evaluation is for or of transformation. Aaron Zazueta, Independent Evaluator, facilitated the interactions that were enriched

by the contributions of Rasmus Heltberg, Lead Evaluation Officer at the World Bank’s Independent Evaluation Group (IEG), Cristina Magro, Council Member of the International Evaluation Academy (IEAc), and Gabriela Perez Yarahuan, Latin America and the Caribbean Regional Director of the Center for Learning on Evaluation and Results (CLEARLAC).

“I’m excited that we communicate evaluations in a much more creative way than we did when I started, 25 years ago. Social media means we can get out messages, we can have the debate in a creative way. It’s a way of truly presenting the value additionality of our work”, noted Dr Naidoo in discussing the evaluative practices for transformation that make him excited about the future of the field. In the same vein, the Director also remarked that evaluation is allowing us to capture and magnify marginal voices. “Working in IFAD, with the rural poor, means that we are able to help them in very real ways by bringing their voices to the table of the governments we evaluate”.

Guided by a series of open questions designed to encourage exploration and exchange, the on-line discussion was the first in a series of virtual events that will map current and future transformative evaluation practices. The events will visualise connections, clarify concepts and compare how they are understood in different fields by exploring different motivations and ways of working, as well as opportunities to collaborate within themes, approaches or locations.

39 @unsplash/Martin Sanchez

IFAD strategy makes remarkable contribution to resilience in Burundi

I

n a country with fragile situations, IFAD’s strategy has made a remarkable contribution to resilience. These and other findings were presented earlier today, as senior policy makers from Burundi, representatives of IFAD’s East and Southern Africa Division (ESA), and the Independent Office of Evaluation (IOE) met to discuss the findings of the country strategy and programme evaluation (CSPE), carried out in 2020.

“Our evaluation, which covers ten projects funded during the period 2009-2020, found that the country programme showed remarkable resilience despite the fragile situations faced, mainly related to institutional, political, economic, environmental and climate change aspects”, stated Dr Indran A. Naidoo, IOE Director, in his opening remarks.

Sara Mbago-Bhunu Four main factors underpin IFAD’s strong performance in Burundi: the organization’s long-standing presence in the country, the Government’s sustained commitment, the project staff’s dedication, and the local communities’ participatory ‘can-do’ approach. Reported revenue improvements, coupled 40

findings, recommendations and issues emerging from the CSPE, and addressed the opportunities and challenges of the future IFAD-Government partnership.

Mònica Lomeña-Gelis, lead author, Burundi CSPE

with training and nutritional education measures and support for micro projects, contributed to reducing child malnutrition rates. In addition, the program also contributed to strengthening the capacities of producers, infrastructure user associations and development committees, and facilitating cooperation between beneficiaries and public institutions.

Co-organized by the Ministry of the Environment, Agriculture and Livestock and the Ministry of Finance, Budget and Economic Planning of Burundi, and IFAD’s IOE, in collaboration with IFAD’s ESA Division, the on-line virtual workshop brought together a wealth of participants.

These included representatives of public sector institutions and national stakeholders, IFAD management and staff, and multilateral and bilateral partners. During the event, participants focused on the main

By the end of 2020,W the programme led to the protection of 100,000 hectares of hills, the development or rehabilitation of 16,714 hectares of marshlands and 850 km of rural roads. The CSPE report also recognized that the capacity building activities that targeted marsh user associations, and the integrated protection of watersheds, were not sufficient to ensure proper maintenance of the marshland and prevention against erosion and downstream flooding. Furthermore, the sustainability of the results is not yet fully assured, and infrastructure maintenance problems remain.

Excellence Monsieur Déo Guide Rurema

“The participatory approach fostered ownership at local and provincial level. However, the targeting of communes with potentially-productive marshes excluded a large proportion of the vulnerable population. It is only with the last project approved

by IFAD, PIPARV-B, that interventions are going to be focused on the hills and on activities not

requiring so much land”, noted Ms. Mònica Lomeña-Gelis, IOE Senior Evaluation Officer, who led the CSPE evaluation team.

Looking forward, workshop participants noted the need to consolidate a holistic approach to pro-poor value chains, working from the various phases of production to the final market destinations, to prioritize strategies and actions to reduce land pressure and facilitate access to assets for the most vulnerable, and to strengthen the regulatory provisions and financial mechanisms to ensure the sustainability of the results achieved.

Since 1979, IFAD has financed 14 development projects in Burundi at a cost of US$ 668.9 million. In the country, 38.7% of the population lives in a situation of extreme poverty, with a clear concentration in rural areas and higher rates among vulnerable populations (women, youth, Batwa, internally displaced persons and persons with disabilities). The 2018 human development index places Burundi in 185th place out of 189 countries, with large disparities especially between rural and urban areas.

Burundian economist in transformative wood art*

B

urundi’s economic capital, Bujumbura is also the central African country’s artistic and creative powerhouse.

Clovis Mwizero, an economist-cum artist lives here.

With his wife, Mwizero has created La Maison du Bois where he designs and makes decorative objects and furniture out of wood.

“La Maison du Bois is an art company, a family business founded by two people who are passionate about art. It is simply an art house that accompanies couples as they build their homes from engagement, marriage through to old age, I would say,” said Mwizero.

He says wood brings a natural feel to a house and unlocks the imagination, as opposed to other materials used to make decorative or household articles such as plastic, metal, or paper.

“In wood you can find everything, you can find the elements of decoration, you can even make wooden watches, you can make wooden handbags, you can make beautiful paintings of weddings or events”.

Mwizero’s passion for art is shared by his wife Gloria, who mostly specializes in knitting children’s toys and decorative accessories.

With wood as the raw material for their business, the couple has ventured into agroforestry, planting several hectares of trees. Mwizero is also giving woodwork and carpentry lessons to youths waiting to enter university at his workshop.

@pixabay/fietzfotos

*source: https://www.africanews.com

41

BIANNUAL CONF EUROPEAN EVALU

V

Value chain governance intrinsic to inclusiveness of poor or vulnerable groups

alue chain governance is a fundamental condition to the inclusiveness and to the distributional efFabrizio Felloni fects on the particularly poor, as well as on those groups that are vulnerable based on criteria such as education, gender, age and ethnicity. To address the methodological implications of this assertion, Fabrizio Felloni, Deputy Director of IFAD’s Independent Office of Evaluation (IOE), joined other leading experts for an inter-agency roundtable discussion hosted under the auspices of the European Evaluation Society (EES), on 10 September 2021.

Organized as part of the online bi-annual conference of the EES ‘Evaluation in an Uncertain World: Complexity, Legitimacy and Ethics’, the roundtable saw the active participation of Johannes Dobinger, Chief of UNIDO’s Independent Evaluation Division, Thuy Thu Le, Evaluation Officer, UNIDO, and Luisa Belli, who is currently leading the Evaluation of FAO’s Contribution to SDG13 on Climate Action.

During the session, entitled ‘Evaluating the Support to Value Chain Development for Poverty Reduction. Upgrading our Conceptual Framework and Toolbox’, Felloni and the group of esteemed colleagues focused on the notion of governance of a value chain and discussed how this can be addressed in an evaluation, in addition to examining the implications for (re)constructing a theory of change for programmes supporting value chains. Discussions built on the approaches and 42

tools adopted by IFAD, FAO and UNIDO to evaluate these programmes. The roundtable also featured opportunities for two-way interactions with the audience. Special topics for debate included the application to evaluations of key concepts for gender-sensitive value chain development, and the positioning of evaluation at the intersection between value chain development and climate change.

Discussions moved from the premise that value chain approaches have gained traction in the past fifteen years with governments and donor partners. As a result, the number of development programmes supporting agriculture-related value chains has increased and the number of evaluations of these programmes is projected to grow accordingly.

The roundtable was framed within the context of the second theme of the ESS bi-annual conference, ‘Adapting the toolbox: Methodological challenges’, which allowed presenters to discuss an interesting way of using well-known tools to address uncertainty or suggest new tools, and draw the link between tools and challenges, identify main problems of using specific tools, and examine how these can be resolved. The other themes of this year’s conference were ‘The Anthropocene and its complex problems: The role of Evaluation’, ‘Propelling and provoking the agenda: The role and responsibility of evaluators’, and ‘Greasing the wheels of evaluation: the role of evaluators, evaluation commissioners and evaluators funders (donors) in ensuring that knowledge changes practice’.

FERENCE OF THE UATION SOCIETY Community-driven development: potential benefits and measurable impacts.

What is the evidence?

C

ommunit y- driven development (CDD), as a form of people-centred and locally owned developJohanna Pennarz ment used in many IFAD-supported operations, has been very effective delivering tangible benefits to impoverished and marginalised communities. CDD also has shown great potential to address issues such as social exclusion, climate change adaptation and food security through localised approaches. This, according to the 2020 IFAD evaluation synthesis on CDD. However, evidence has been inclusive that CDD has an impact on social cohesion and social capital. Does this mean that CDD does not work in practice? Johanna Pennarz, Lead Evaluation Officer at IFAD’s Independent Office of Evaluation (IOE), tackled this tough question head-on during the course the European Evaluation Society (EES) bi-annual conference, on 10 September 2021.

Entitled ‘Measuring What Matters: Evaluations of Community Development During Times of Global Crisis’, the session to which Johanna contributed as a presenter saw the active engagement Susan Wong, Lead Social Development Specialist at the World Bank, and Hai Ha Vu Thi, Programme Analyst on Youth, Gender and Social Inclusion at IFAD. Dee Jupp, participatory social development expert, moderated the discussions.

IFAD’s evaluation synthesis on CDD supports the overall view that giving communities control over planning decisions and investment resources is a

way to ‘future proof’ investments. Drawing on this body of evidence, Johanna highlighted that CDD offers a cost-effective, sound and environmentally -conscious means to ensure both rapid adaptation and continuation of development efforts despite disruption or discontinuation of external support. However, CDD does not only contribute to stronger community organisations; it also requires community structures to be in place in order to become effective. IOE evaluations often point to the limited sustainability of community organisations, which highlights the need for longer-term engagement.

Against this backdrop, discussions focused on the strengths of the approach and on the challenges that evaluators face when searching for conclusive evidence that “CDD really works”. Challenges include the localised and highly adaptive nature of the CDD approach, which makes it difficult to compare and aggregate the evidence; the difficulties in measuring non-tangible outcomes, such as social capital; and the barriers to accessing the often very remote and marginalised communities, in particular under recent COVID conditions.

The session was framed within the context of the first theme of the ESS bi-annual conference, ‘The Anthropocene and its complex problems: The role of Evaluation’, which allowed presenters to address the bigger conceptual questions, by moving away from a purely philosophical or very pessimistic view, and rather focus on providing constructive perspectives or solutions.

43

Evaluation report calls for greater focus on pro-poor targeting in Uganda

H

@unsplash/Jeff Ackley

ousehold income might have more than doubled for beneficiaries of the Agricultural Technology and Agribusiness Advisory Services Project (ATAAS) in Uganda – but evidence suggests that already well-established groups and better-connected host farmers mostly benefitted from the project, rather than the poorer and more vulnerable members of rural communities. This, according to a new IOE report, published on 11 June 2021.

affected various aspects of the project. While the project was successful in terms of fostering technological innovations and collaborative research, the demand for technology improvements and extension advice remains unmet for 75 per cent of Uganda’s farming households.

Furthermore, farmer groups and NGOs had little opportunity to influence how extension should be provided during the project. As a result, many groups have since collapsed.

The project’s aim was to enhance the performance of the agriculture sector through support for technology development, extension services and stronger linkages between farmers and the market. ATAAS was a successor to the National Agricultural Advisory Services Programme (NAADS). Although the project was able to meet the 1.68 million people planned, most beneficiaries came from the inherited and well-established NAADS groups, and thus ATAAS did achieve its pro-poor targets.

The expected shift to innovative, private sector-led approaches failed to materialize, and instead there was a reversion to a more conventional model of public sector-led services combined with subsidized inputs. In spite of this, ATAAS did succeed in building the capacity of private sector seed companies and farmer group seed producers to deliver improved seed materials. The project was also relatively successful in terms of equitable participation of women, although there is little evidence to suggest that this led to wider changes in the roles of women.

Targeting challenges reflected the project’s centrally managed extension system, which 44 @unsplash/Keith Kasaija

For the future, IOE’s evalua-

tion report recommends that IFAD ensure that its comparative advantage is adequately leveraged and its target group is sufficiently and effectively reached, even when it is a minor contributor in large projects. In addition, the organization should pay greater attention to political drivers in project design, especially when projects are largely funded from government resources.

Uganda is a low-income country with a GDP per capita of US$794 in 2019 Past economic growth contributed to reducing poverty in the country according to national estimates from 56.4 per cent in 1993 to 24.5 per cent and 19.7 per cent in 2009 and 2013, respectively. However, Ugandans remain vulnerable to slipping back into poverty – for every three Ugandans who escape poverty, two fall back in. More worryingly, national estimates show that poverty levels have worsened in recent years, rising to 21.4 per cent in 2016. Similarly, the proportion of people living in extreme poverty, based on the international poverty line, increased from 36 per cent in 2012 to 41 per cent in 2016.

Ugandan cultural troupe wins trophy in India’s National Tribal Dance Festival*

U

ganda’s cultural team, the Kiconco Cultural Troupe won first prize for their stellar performance at this year’s National Tribal Dance Festival in Chhattisgarh Raipur, in India.

“We are proud to have participated at the National Tribal Dance Festival in India where we took a cultural group which ably represented us and showcased our national heritage. We call upon the ministry of tourism to give priority to culture because it is the only thing that shows our uniqueness,” said Robert Musiitwa, Public Relations Officer at the Uganda National Cultural Center (UNCC) The National Tribal Dance Festival is an Indian annual event that seeks to bring together all the tribal states and union territories of India and tribes from other countries.

@https://www.bukedde.co.ug/

“The team also took time to practice to be able to create a fusion of the local dialect Hindu with local dialects in Uganda. This is what made the Uganda team favourite of the audience such that whenever they stepped on stage the audience would go crazy”, said Francis Ojede, UNCC Executive Director.

Led by Juliet Nuwamanya, the team showcased different dances, sounds, music, marriage, and initiation acts.

Dances performed included Runyege, Ntogoro, Bakismba, Larakaraka, Otwenge, Mwanaga and Kitagururo, among others.

Other countries that participated were Swaziland, Nigeria, Mali, Sri Lanka, Uzbekistan, Palestine, and India which was the host.

*source: https://www.newvision.co.ug/ and https://chimpreports.com/ 45

CLIMATE ADAPTATION WITH ENVIRONME

“I

t is the belief of many that a climate change adaptation intervention should be naturally good for the environment. It is assumed S. Nanthikesan that climate resilience is going to build environmental resilience. We have found that this is not always the case. Climate resilience may not always work towards environmental resilience without conscious efforts to marry them. The challenge is to achieve environmental, climate and economic resilience together”. S. Nanthikesan, Lead Evaluation Officer at the Independent Office of Evaluation (IOE) of IFAD, put this finding on the table during the event entitled ‘Integrating Environment into Evaluations’.

The finding emerges from IOE’s recent Thematic Evaluation of IFAD Support to Smallholder Farmers’ Adaptation to Climate Change. In his intervention, Nanthikesan highlighted the thematic evaluation’s approach and methodology to assess the environmental sustainability of climate adaptation interventions and to analyse the complex nexus between human systems and ecosystems.

Held on 16 November, the Evaluation Practice Exchange (EPE) learning event of the United Nations Evaluation Group (UNEG) was organized by the Working Group on integrating environmental and social impact in all development evaluations. The session was organized to take stock of the progress within the UN development system and to bring experiences from outside such as the ongoing work of the Footprint Evaluation initiative of the Global Evaluation Initiative (GEI). Juha Uitto, Director of the Independent Evaluation Office at the Global Environment Facility (GEF), moderated the event which featured introductory remarks by Michael Spilsbury, Director of the Evaluation Office of the UN Environment Programme (UNEP). S. Nanthikesan’s intervention was complemented by those of other high-level experts, including Johannes Dobinger, Chief of the Independent Evaluation Division of the United Nations Industrial Development Organization (UNIDO), David Todd, technical specialist in evaluation for several UN evaluations and member of the UNEG Coordinating Committee, and Andy Rowe representing Footprint Evaluation.

46

N NOT SYNONYMOUS ENTAL RESILIENCE Discussions moved from the stocktaking exercise by the UNEG Working Group on Integrating Environmental and Social Impact into Evaluations (ESI) which revealed that environmental aspects were inadequately covered in most evaluations by UNEG members. Against this backdrop, in the past year, the ESI Working Group has focused its efforts to starting to develop guidance that would help UNEG members to enhance their performance in this respect. IOE is one of the founding members and co-coordinators of the ESI working group.

“IFAD has put in place efforts to integrate environmental considerations not only in evaluations, but also in the evaluation function itself. The first enabling condition was that management integrated environmental considerations in all its interventions. Mirroring this and around the same time, the evaluation policy and the evaluation manual stipulated that we should look at environmental considerations in all IFAD evaluations. The evaluation manual went one step further and introduced support to climate adaptation and environmental natural resources management (ENRM) as an evaluation criteria (in addition to the more familiar international criteria) to be assessed in all evaluations of IFAD’s operational interventions. The Annual Results and Impact Report (ARRI) of IOE was submitted to the governing bodies that aggregated the performance related to ENRM and climate adaptation of all IFAD projects. Thus, IFAD ensured that evaluations worked towards maintaining accountability for environmental sustainability”, noted Nanthikesan in presenting the systemic approach pursued by IOE and IFAD to mainstream environmental impact in all evaluations.

Juha Uitto, the event moderator, noted that IFAD was one of the pioneers in the UN system to recognize the need to evaluate the intended and unintended environmental consequences of all development interventions. Today, IFAD’s commitments continue to ratchet up the importance assigned to mainstreaming environmental considerations in its interventions while its evaluations are already testing out the cutting-edge approaches to assessing environmental impact – which are still emerging as ideas elsewhere.

47 @unsplash/Pat Whelen

Farmer productivity increased in Senegal as risks threaten long-term sustainability of IFAD-supported project

T

he ‘Support to Agricultural Development and Rural Entrepreneurship Programme’ (PADAER) has had a positive impact on agricultural production, rice yields and the income and food security of farm households in the Republic of Senegal. Looking ahead, the sustainability of these and other benefits remains uncertain. The report published by IOE provides substantive insights in this regard.

the achievements of other IFAD projects in Senegal, adjustments to the programme’s design shifted its focus more towards rice production rather than rural entrepreneurship development.

The incomes of farmers and beneficiary rural small and microenterprises benefitted from the programme’s initiatives, most notably from the infrastructure and irrigation activities.

However, there was only limited impact on women’s economic empowPADAER was approved in September erment, and the programme did not 2011, and ran from March 2013 to June include specific activities to promote 2019. With an initial budget of US$51.71 the participation of women in lomillion, complemented by US$5.98 cal decision-making processes. PAmillion additional financing, the Eoghan Molly, DAER’s main technological innovaprogramme set out to improve lead author, Senegal PPE tions were in the livestock sector, and the food security and income of involved the introduction of improved smallholders (farmers and livestock prodrinking troughs for animals. This improved the ducers) and create sustainable rural employment conditions for watering herds, saved time for liveopportunities, particularly for women and youth. stock producers, and led to technical upgrades in Although PADAER was originally conceived as a the vaccination facility model in use. flagship programme to consolidate and scale up

48 @unsplash/Eyelit Studio

The Evaluation concluded that PADAER has contributed to the Senegalese government’s goal of achieving self-sufficiency in rice production, although the rice yields obtained remain below expectations and the support to rice cultivation concerned only a limited number of beneficiaries.

For the results achieved, sustainability remains an issue of concern. The management committees of new pastoral units are not yet fully functioning, smallholders continue to face barriers in accessing markets, and the absence of specific programme activities to link farmers to bank financing has left them without the means to procure supplies and other inputs in the longer term.

The IOE report recommends addressing the sustainability challenges, including by ensuring that the programme’s second phase (PADAER-II) supports all infrastructure management committees and pastoral units. In broader terms, the PADAER experience highlights the importance of issues such as access to finance and agricultural insurance, as well as the need to design interventions based on thorough and commodity-specific value chain analyses.

Since 1979, IFAD has supported 16 programmes and projects in Senegal, investing US$216.4 million and directly benefiting more than 455,000 rural households. Poverty is more widespread in rural areas, where 75 per cent of poor households live. Rural women are almost 70 per cent of Senegal’s workforce and produce 80 per cent of the country’s food.

@https://cepell.it/

Senegalese wins Goncourt prize*

A

new victory for Senegalese and African literature as writer Mouhamed Mbougar Sarr is named winner of The Goncourt 2021 prize.

The Goncourt, the most prestigious French literary prize, was awarded to Sarr, who at 31 years old becomes the first writer from sub-Saharan Africa to be honored by this prize. Sarr received six votes from a jury of 10, for “The most secret memory of men” (published by Philippe Rey), a novel inspired by the cursed fate of the Malian writer Yambo Ouologuem.

His win has elicited proud reactions in Senegal.

“As Senegalese, we are happy and very moved that one of our compatriots has won the Goncourt Prize.

After all, it is a source of motivation for other writers too. We can say, in a general way, that we are more than proud”, said Mousatpha Sarr, a student of Modern literature in Dakar.

The distinction crowns a dazzling career for the Senegalese writer who launched his first book, The Hold, in 2014.

“I have just read the novel and it is really extraordinary; this journey through Africa, Europe, the Americas. Especially this quest to pay tribute to Yambo Wologhem. It is a beautiful revenge for Africa and especially a beautiful promotion for Senegalese literature, which, it should be noted, was in lethargy” stated Moussa Sagna, research Professor at the Chiekh Anta Diop University Dakar.

The young Senegalese writer is now the pride of an entire country, an entire continent after his coronation at the Prix Goncourt 2021. A pride all the more great because the 31-year-old author is the first from sub-Saharan Africa to have won this prestigious prize. A consecration that comes 100 years after that of Rene Maran, the first black author to have won the Goncourt Prize.

49 *source: https://www.africanews.com

ASIAN EVALUATION WEEK

T

he constraints that the COVID-19 pandemic has posed on evaluators as well as the opportunities to introduce changes in international evaluation practices, and the factors impacting rural poor farmers’ ability to adapt to climate change were the two salient issues that senior representatives of IFAD’s Independent Office of Evaluation (IOE) brought to the discussion table during the 2021 Asian Evaluation Week (AEW).

This year, the AEW took place virtually from 6 to 10 September 2021. The event was jointly sponsored by the Ministry of Finance of the People’s Republic of China, through the Asia-Pacific Finance and Development Institute (AFDI), and the Asian Development Bank’s Independent Evaluation Department.

Speaking on behalf of IOE were the Office’s Deputy Director, Fabrizio Felloni, and Lead Evaluation Officer, S. Nanthikesan.

‘Transformative pathways to Food Security in the 21st Century – Resilience to Climate Change’ was the title of the session to which Nanthikesan contributed on 6 September. In his presentation, Nanthikesan put the spotlight on the state-of-the-art evaluative evidence regarding the socio-technical, societal and policy barriers limiting the ability of

50 @unsplash/Peter Burdon

rural poor farmers to build resilience to climate shocks. The business-as-usual approach to climate adaptation is over.

Success in scaling up adaptation interventions will largely depend on government ownership, high-profile partnerships, and engagement at design.

However, many governments face significant challenges to incentivize sustainable climate adaptation responses of smallholder farmers.

S. Nanthikesan

Juha Uitto, Director of the Independent Evaluation Office, Global Environmental Facility, moderated the session, which also featured presentations by Jerry Knox, Professor at Cranfield University, and Laura Silici, Research Analyst and Independent Consultant, and drew insights from distinguished panel members, H.E. Sani Redi, State Minister, Ministry of Agriculture of Ethiopia, and Sheikh Md. Mohsin, Additional Chief Engineer, Ministry of Local Government of Bangladesh.

Evaluations under COVID and and pathways to food security at forefront of discussions ‘Two years of evaluations under COVID: lessons and opportunities’ was the title of the session to which Fabrizio Felloni contributed on 7 September. The approach taken by IOE to offset the challenges and capitalize on the opportunities stemming from the COVID-19 pandemic was the focus of the presentation that Felloni delivered alongside Roberto LaRovere, Independent Consultant with IOE. Virtual interviews work

Fabrizio Felloni

well with many stakeholders, and are now socially accepted by senior government officials.

At the same time, GIS data and images are useful for projects on infrastructure, irrigation, pastoral corridors and agroforestry, although some changes are not detectable from the sky. However, totally virtual evaluations, without national consultant visits, should be pursued only in extreme cases. There remains no perfect substitute for country and field visits.

pendent Evaluation Group of the World Bank, further enriched a session that benefitted from the perspectives of panel members Luthfur Rahman, Additional Chief Engineer and Project Director, Local Government Engineering Department of Bangladesh, and Sanjeev Sridharan, Country Lead, Learning Systems and Systems Evaluation at the India Country Office of the Bill and Melinda Gates Foundation. Maya Vijayaraghavan, Methods Advisor, Independent Evaluation Department of the Asian Development Bank, moderated the discussions.

The theme for the 2021 AEW was ‘Transformational Evaluation: Moving from Uncertainties to Resilience’. This recognizes the role of evaluation in providing evidence-based information, lessons and recommendations on development issues to help address short and long-term challenges and current uncertainties towards resilient and sustainable development. Government officials, representatives of international organizations, and evaluation experts interested to exchange knowledge, experience and the latest thinking and innovation in evaluation attended the event virtually, connecting remotely from across globe.

A presentation by Jos Vaessen, Methodology Advisor, Inde51 @unsplash/LinkedIn Sales Solutions

Social norms challenged as women participate in market activities in Pakistan

O

wnership of small housing units and land plots as well as livestock has become reality for some women in extremely poor rural households in Pakistan, thanks to IFAD co-financed projects.

The projects contributed to the social and economic empowerment of women by enhancing their access to resources, assets and services, allowing them to partake in activities that were not seen as women’s domains traditionally. This was one of the findings of the country strategy and programme evaluation (CSPE) that covered the period 2009 – 2020, which senior policy makers from Pakistan, representatives of IFAD’s Asia and the Pacific Division, and IOE discussed on 2 June 2021.

Ms Samar Ihsan, Senior Joint Secretary, Economic Affairs Division

“The portfolio had important achievements in women’s social and economic empowerment, in light of the challenging gender context in Pakistan”, stated Dr Indran A. Naidoo, IOE Director, in his opening remarks.

The projects provided livestock, skills training and small housing units to be registered in women’s names. Women also 52

Fumiko Nakai, lead author, Pakistan CSPE

benefitted from interventions such as water supply schemes, which reduced their workload and improved the health and well-being of household members. Social mobilization and the establishment of women’s community organizations enhanced social spaces for women and their participation in decision-making processes at community level.

Organized by IFAD’s IOE, in collaboration with the Economic Affairs Division of the Government of Pakistan and IFAD’s Asia and the Pacific Division, the on-line virtual workshop brought together a variety of participants. These included government representatives at federal and provincial levels, national partner agencies, IFAD staff and international development agencies.

The objective of the event was to discuss the main findings

and recommendations of the CSPE, and reflect on strategic priorities for IFAD’s next country programme and related opportunities and challenges. Workshop participants recognized a strong poverty focus and the positive impact of IFAD’s interventions on human capital through infrastructure support.

Interventions such as drinking water schemes, drainage and sanitation, and link roads contributed to increasing access to basic services and improving health and living conditions for beneficiary communities. The fact that many of these schemes were planned and implemented through community-led approaches further added to their value. In the earlier part of the evaluation period, the portfolio also contributed to strengthening of microfinance service providers, and to knowledge generation and policy issues in the microfinance sector.

Nigel Brett

“The improved roads resulted in major travel time and cost savings, and had a visible impact on household incomes from reduced wastage of fish catch and perishable crop produce”, Ms Fumiko Nakai, IOE Senior Evaluation Officer, highlighted.

Challenges, however, remain.

In a number of projects, implementation arrangements had weak linkages with relevant institutions, as limited attention was paid to fostering connections between the target group and service providers. The CSPE also noted an overreliance on poverty scorecards – based on observable indicators – for geographical and household targeting with inadequate reflection on drivers of poverty and structural constraints, limited promotion and adoption of improved agricultural techniques, and weak alignment of technical and vocational training with market needs and contextual realities.

Alvaro Lario - Associate Vice President, Chief Financial Officer and Chief Controller

“The evaluation found inadequate attention to food security and nutrition in programming and monitoring”, Ms. Nakai stated.

Looking forward, workshop participants noted the importance of placing greater emphasis on inclusive market systems development with due attention to climate resilience and natural resource management.

Other recommendations included the need to articulate a strategy to promote innovations and scaling up for greater rural poverty impact; to place more emphasis on strengthening and linking with institutions, policies

and systems for greater likelihood of sustainability; and to adopt a more flexible and differentiated approach in targeting and programming.

@https://ilfarosulmondo.it/

Syed Abrar Hussain - Additional Chief Secretary, Planning and Development Department

“The programme needs to reflect more carefully how best to leverage sustainable changes in the local economy around agriculture and food systems that would benefit the rural poor”, Dr Naidoo noted.

Since 1979, IFAD has approved the financing of 27 projects in Pakistan, for a total cost of US$2.58 billion. During this time, the Government has also had a number of anti-poverty initiatives. Stemming from these efforts, significant progress has been made in reducing the poverty level over the past two decades. Challenges remain, as about one quarter of the population still lives under the national poverty line and about 39% in multi-dimensional poverty. Pakistan ranked 151st out of 153 countries on the Global Gender Gap Report in 2020.

Anniversary of national poet Allama Iqbal*

O

n 9 November 2021, Pakistan observed the 144th birth anniversary of the national poet, Dr Allam Iqbal.

Popularly known in Pakistan and other parts of the world as “Poet of the East”, Dr Allama Muhammad Iqbal was born in Sialkot on November 9, 1877.

Iqbal was an acclaimed poet and philosopher.

His poetry was translated into Spanish, Chinese, Japanese, English and many other languages.

The Indians, Pakistanis, Iranians and other foreign literary scholars revered Iqbal as a famous poet. His first poetry book, The Secrets of the Self, was published in the Persian language in 1915, and other poetry books include The Secrets of Selflessness, The Message from the East and the Persian Psalms.

*source: https://nation.com.pk 53

ORGANIZATIONAL CHANGE NECESSARY TO SUPPORT VALUE CHAIN TRANSFORMATIONAL INTERVENTIONS

“T

o support transformational interventions, organizations need first to transform themselves”. This succinct, yet incisive observation was the cornerstone of the presentation that Fabrizio Felloni, Deputy Director of IFAD’s Independent Office of Evaluation (IOE) delivered alongside Girma Kumbi, Principal Evaluation Officer of the Independent Development Evaluation of the African Development Bank, during the Wilton Park dialogue series.

Held on 8 September 2021, the Wilton Park dialogue was the first, in a series of three, scheduled two months before COP26 in Glasgow. Rob D. van den Berg, member of IOE’s Evaluation Advisory Panel, and Professor at King’s College, London, delivered welcoming remarks together with Robin Hart, Senior Programme Director at Wilton Park. ‘Pathways towards transformational climate action: connecting the worlds of research, evaluation and policy for action’ was the theme of this fist dialogue, which moved from the notion that while transformational climate action requires collective action, many practical, political and structural barriers exist to improved collaboration and aligned networks across the worlds of research, evaluation and policy.

54 @pixabay/stefan_bernsmann

This was the backdrop against which Felloni and Kumbi carved their presentation. Drawing on lessons from evaluating support to pro-poor rural value chains, the presentation emphasized the importance of exploring conceptually the system that we wish to transform, in order to plan for transformational changes. Development organizations may not be able to continue with the status quo ante. Focusing on sub-systems might prove an effective first step, in this regard, especially in those instances in which we may not need or be able to act on an entire system.

The dialogue brought together about 50 participants from the worlds of policy, evaluation and research, with the aim of clarifying the dimensions of transformational climate action in the context of the COVID-19 pandemic, which has revealed and exacerbated inequality, poverty, unemployment, and vulnerability. Partnering institutions and entities included, in addition to IOE, the International Development Evaluation Association (IDEAS), Climate Investment Funds (CIF), Global Environment Facility (GEF), Green Climate Fund (GCF), University of Sussex, Institute of Development Studies (IDS), United Nations Development Programme Independent Evaluation Office (UNDP IEO), and the International Evaluation Academy.

FURTHER W READING

W

O

Evidence Summary on COVID-19 and Food Security

fficially launched on 16 July 2021, the IDEAS book ‘Transformational Evaluation for the Global Crises of Our Times’ features a chapter on evaluating the support to value chain development as a (potential) transformative change, co-authored by IOE Deputy Director, Fabrizio Felloni, and Girma Kumbi, Chief Evaluation Officer at the African Development Bank, and an endorsement by IOE Director, Indran Naidoo. “‘Business as usual’ is a thing of the past. The COVID-19 pandemic may be the latest in a series of era-defining challenges – it will not be the last. If humanity is to move forward, transformational change is desperately needed”, stated Dr Naidoo.

In their chapter, Fabrizio and Girma examine the contribution of AfDB and IFAD to agriculture-related value chain development, based on evaluations conducted by the respective organisations. Independent Magazine is pleased to offer its readers 6 recommendations that Fabrizio and Girma share on how international development organisations could better support value chain development*.

1 2

Organisations need conceptual clarity on what a value chain is and what are the critical requirements to make them viable, sustainable and inclusive.

Value chain development requires long-term engagement. In many financial institutions, this will often entail providing support throughout several project phases.

3

Projects need to actively promote outreach to poor and very poor groups and gender equality, rather than assuming that trickle-down mechanisms will be operating, like an “invisible hand”.

4

Projects need to promote inclusive value chain governance and an inclusive policy and regulatory environment by establishing or strengthening multi-stakeholder platforms and inter-professional associations.

5

In addition to ‘conventional’ approaches to rural finance, there is a need to devise ways to provide financing along the value chain, including producers, buyers, processors and retailers.

6

Development agencies need to strengthen partnerships with other organisations, including private sector ones that have strong value chain expertise.

The book, co-edited by Rob D. van den Berg, member of IOE’s Evaluation Advisory Panel, is available to download on the IDEAS website [here]. Printed copies are available through Amazon.

55

Mechanisms to be developed to reach poorest target groups in Dominican Republic

A

dequate mechanisms and competencies need to be developed to ensure the inclusion of the poorest in projects based on the value chain approach. The experience of the Rural Economic Development Project in the Central and Eastern Provinces (PRORURAL Centre and East) in the Dominican Republic shows that to ensure social inclusion in value chains it is necessary to scale geographic targeting, plan adequate implementation timeframes, consider domestic and local markets, tailor capacity building to beneficiary needs, and link projects to national systems and programmes that combat poverty and inequality.

This, according to IOE’s, released on 31 May 2021.

Approved in April 2010, the PRORURAL Centre and East project was implemented over the period 2013-2018, and yielded mostly mixed results.

The project pursued a public-private partnership approach to promote and support the Dominican agribusiness sector. This innovative approach, implemented for the benefit of all value chain actors, succeeded in transferring public resources to a private institution to invest in rural development.

The project also contributed to creating a space for promoting rural development, and supported the financing of business plans for 78 smallholder producer organizations. This effort, aimed at positioning the organizations in domestic or international markets, did not however promote sufficient dialogue on policies or foster synergies with other ongoing programs in the country. Likewise, while the project contributed to a better knowledge of climate risks by producers and financed several early warning systems, the application of adaptation measures at the farm level will require more time and support.

@unsplash/Richard Cordones

@unsplash/Anthony Lebron

@unsplash/Kelvin Moquete 56

@pixabay/Hyparxis

With regard to the improvement of agricultural productivity and food security of the producers who participated in the project, the evaluation did not uncover conclusive data. The scenarios for the project’s flow of benefits to continue after its closure are thus uncertain. Windows of opportunity

exist, as the new Government Plan includes attention to rural populations and agricultural development.

The contemporary female figure in a colorful way* Mònica Lomeña-Gelis,

lead author, Dominican Republic PPE

Drawing on the above, the IOE report recommends that future projects carry out territorial diagnoses and make the approach and tools related to business plans more flexible, and explore expanding partnerships to implement and support rural organizations. Some of the lessons learned from this experience were incorporated into the design of the new Inclusive and Youth PRORURAL. Since 2008, the Dominican Republic has been the Latin American economy that has grown the most, at an annual average of 5.06%. There has been a significant reduction in poverty in the country, between 2009-2019, both at the urban level (from 37.6% to 17.2%) and in the rural area (from 50.6% to 26.2%). National and rural extreme poverty rates have dropped accordingly.

Improvements are also reported in the human development index. In 2019, the country was among those with high human development.

A collection of works of art in acrylic and oil on canvas, which shows the vision of the artist Ana Sofía Batlle about female identity, was exhibited at the Fernando Peña Defilló Museum, from June 24 until August 8, 2021.

Body Landscapes aimed to explore and question the preconceived ideas that exist today about women and invited the public to establish their own criteria about the meaning of women in today’s society.

Ana Sofía Batlle uses a mixed technique to recreate and give spontaneity to her works, and she mixes oil with elements like airbrush. The Dominican artist, who graduated in plastic arts from the School of the Art Institute of Chicago (USA), has been developing her technique for several years, with which she represents female figurations within an illusory, colorful, playful, and contemporary landscape. ‘’It is an opportunity to express my point of view through the art of a reality that I consider relevant in today’s society, at the same time it allows the public to know a new stage in my evolution as an artist ’’ highlights the young plastic artist.

This solo exhibition was Batlle’s second in the Dominican Republic. The works that were exhibited included All The Single Ladies, inspired by corporeal landscapes with which she expresses her perception of what it means to be feminine in today’s society, in which, according to the artist, preconceived ideas such as voluptuousness become decorative through visual patterns.

*source: https://dominicantoday.com/

@http://www.anasofiabatlle.com/

57

Evaluation must play central role in building post-pandemic food secure world Lessons drawn from evaluation evidence can foster food security, equity and sustainability in the aftermath of the COVID crisis. This premise will be at the centre of this year’s gLOCAL evaluation week, which took take place between May 31st and June 4th.

sults, especially to the poorest and most vulnerable. To this end, the events organized under the gLOCAL 2021 umbrella brought together academics, government officials, non-profit organizers, policymakers, evaluation practitioners, students and researchers at a global level, both in the private and public sectors.

tance of people-centred evaluations, and provide examples of ways in which adaptive programming and inclusive evaluations have been undertaken.

Eoghan Molloy

Johanna Pennarz

The overarching objective of gLOCAL 2021 was to prompt lessons and reflections on how to ‘build forward better’ in the post-pandemic world. ‘Building forward better’ calls for advances in planning, systematic monitoring and rigorous evaluation to ensure that projects, programs, and strategies are informed by evidence. Evidence will be key to help revitalize economies affected by the pandemic, and ensure that policies are implemented effectively and bring the best possible re58

Prashanth Kotturi

The Independent Office of Evaluation (IOE) of IFAD was at the forefront of this global effort, with its staff sharing the wealth of experiences, knowledge and reflections garnered over the last year in several webinars.

On June 2nd, Johanna Pennarz and Prashanth Kotturi, respectively IOE Senior Evaluation Officer and Evaluation Officer, joined Dr Indran Naidoo, IOE Director, to discuss the impor-

The following day, Eoghan Molloy, IOE Evaluation Officer, presented insights into how the pandemic has increased reliance on national consultants, what opportunities can be leveraged in this regard, and what risks need to be managed.

The need to take a step back and rethink the basics of evaluation work was the starting point of the discussions that Monica Lomeña-Gelis, IOE Senior Evaluation Officer, joined on June 4th. By reflecting on what has been learned during this pandemic year, the session considered more sustainable evaluation practices, with a

lighter carbon footprint of international travel, and stronger national ownership.

Monica Lomeña-Gelis

Fabrizio Felloni, IOE Deputy Director, was involved in two events. On June 3rd, Mr. Felloni joined experts from the the Consiglio per la ricerca in agricoltura e l’analisi dell’economia agraria of Italy, in a webinar organized by the Italian National Rural Network. The webinar discussed approaches taken to capitalize the use of evaluative knowledge for improving programme design and implementation of public policy. The following day, IOE’s Deputy Director moderated a panel discussion focusing on the type of data that can be used to simulate or assess the intensity of shocks caused by the crisis on agriculture and to help shape policy responses, drawing on the experience of the Italian agriculture and food sector.

EVALUATION FUNCTION TOP AMONG DEVELOPMENT BANKS

T

he quality of IFAD’s evaluation function is the highest among multilateral development banks, according to the highly regarded and influential Quality of Official Development Assistance Report (QuODA).

“I welcome the high rating, which attests decades of leadership and investment in the evaluation function”, stated Dr Indran Naidoo, IOE Director.

IFAD’s evaluation function ranks second behind the European Union in the line-up of multilateral development organizations, and is fourth among all providers of official development assistance world-wide, “well ahead of other [multilateral organizations] on evaluation”*. The World Bank’s International Development Association, which is “the nextbest multilateral agency”, placed 12th.

“I seek to sustain and even improve on this affirmation as I work closely with IFAD’s Executive Board, members States and IFAD management”, Dr Naidoo further underscored.

Evaluation is one of the four dimensions that determine the quality of official development assistance, as measured by the QuODA report. Specific indicators associated with this dimension include the quality of evaluation, institutional learning, and results-based management systems. The other dimensions of the report are prioritisation, ownership, and transparency and untying. At the corporate level, based on these four dimensions, “IFAD ranks 1st on QuODA overall” IFAD’s close partners the African Development Fund and IDA took second and third place respectively in the rankings.

“IFAD’s high ranking is a testament to the importance we place on ensuring that every dollar spent has a long-term impact on tackling the hunger and poverty experienced by the world’s most vulnerable people,” said Gilbert F. Houngbo, President of IFAD.

Fabrizio Felloni

The gLOCAL evaluation week is an M&E knowledge sharing initiative that aims to support the exchange of M&E knowledge and experiences.

Multilateral agencies outperformed bilateral donors. The report concludes that using the multilateral system can help ensure funding reaches those in greatest need. *Source: Mitchell, I., Calleja, R., and Huges, S. (2021). The Quality of Official Development Assistance. Center for Global Development.

59

IOE FAMILY

60

FUN ‘N GAMES ENERGIZE TEAM SPIRIT

O

n 1 6 – 1 7 No vember 2 02 1 , the I O E f ami l y re charge d i t s bat t eri e s w i th an energi z i ng re tre at . Organi ze d b y I O E ’s ‘re tre at t e am’ (L aure Vi daud , Mas si e l Ji menez , Prashanth K o t turi , Chi tra D e shpande and Ma x K odj o K oue s si ) and f ac i l i tat e d b y C i arán B e ar y, the e vent f e ature d a ho s t o f e xperi ent i al t e am bui l di ng e x erc i se s . Hi ghl i ght s i nclude d yo ga w i th Dani e l a A spre l l a and a Me so -A meri c an danc e l e s son w i th Mas si e l !

61

Independent Office of Evaluation International Fund for Agricultural Development Via Paolo di Dono, 44 - 00142 Rome, Italy Tel: +39 06 54591 - Fax: +39 06 5043463 E-mail: evaluation@ifad.org www.ifad.org/evaluation www.twitter.com/IFADeval www.youtube.com/IFADevaluation

62