Independent Magazine - Issue 6, 2023

EMOTIONAL INTELLIGENCE

The missing link between evaluators & management

HARNESSING NEUROSCIENCE

Evaluation through the lens of brainscience

EVALUATION PRINCIPLES

Moving beyond what is right or wrong

Welcome to the 6th edition of Independent Magazine. We appreciate and recognize the feedback received from around the world to our magazine, especially the comments from increasingly diverse audiences. We continue to strive to address all elements that make evaluation, and IFAD, more effective. This edition of the Magazine, in particular, addresses some of the boundaries we have been pushing to make evaluation more integrated, comprehensive and dynamic.

Much of the interest relates to the unique mandate of IFAD and its central role in supporting food security through its focus on smallholder agriculture. The sector has become even more prominent after COVID-19, given massive disruptions to interlinked economies that were forced to recast their approaches to development. Climate change has also had a disproportionately adverse effect on the agricultural sector. IOE assessments show tangible progress and remaining challenges in smallholder climate change adaptatio. In this context, which is affected by declining resources, independent evaluation becomes even more critical to ensure timely feedback for course correction.

Although evaluation as a discipline has come of age, as demonstrated by the number of professional networks and vibrant schedule of events, its full integration into the institutional fabric of organizations remains work in progress. This applies as much to governments, who have come a long way over the past two decades, as it does to international and bilateral institutions, many of whom still fall short of the full requirements of sound evaluation principles.

Within IFAD, the overarching architecture for evaluation is governed by the Executive Board, which holds responsibility for adherence to the evaluation policy, and for safeguarding the full independence of the function, which ensures its credibility. Equally important for the credibility of the function is methodological rigour, which is another aspect that we have heavily invested in, including through our Evaluation Advisory Panel. This notwithstanding, evaluation involves more than just applying a methodology and producing a report; the processes are as important as the final product.

In my own words

We have already spoken about the need to integrate feedback into management and oversight systems, creating spaces to test claims rather than accepting them as unassailable. An evidence-based institution benefits from oversight that is structurally, behaviourally and operationally independent. In addition, by virtue of it not being interest-driven, independent evaluation has other benefits such as the ability to access voice, and to report without censorship or approval from interest groups. It is a public good insofar as it seeks to connect citizens directly to policy makers and decision-makers, and thus helps empower people by ensuring deeper democratic practice, transparency and accountability.

Our magazine continues to stimulate debate on all these areas, as it is only through frank and robust conversations that progress can be made.

In this edition, we focus on a much spoken about yet less visible element: the people who engage in the evaluation construct. We move into the realm of brain-science, which helps us to focus on fundamental questions of receptivity, and to look at how we engage with performance feedback. We firmly believe that understanding people’s mindsets, focusing on their thought processes and unpacking the reasons that determine how they respond to a given stimulus, are key to optimizing the real potential of the evaluation discipline. In this regard, our work with the Innovation Hub of IFAD, supported by EvalForward, a Rome-based evaluation network focusing on food security, has been very stimulating.

We emerged from this process with what is known as ‘the Evaluation Pills’: thirteen doses of insight into the full spectrum of the evaluation world. In the Pills, by applying neuroscience-based principles, we capture and examine key elements seldom spoken about, from both the evaluator’s and evaluand’s perspective. These elements include the challenge of reconciling institutional loyalty with what may be perceived as critical messaging from evaluators; the question of what constitutes objectivity and whose evidence matters; the difficulty to recognize and deconstruct the biases that we bring to the practice; and the need to debunk some of the myths relating to how change happens. In this edition, we also share a wealth of lessons from completed evaluations, alongside presentations made at highly distinguished academic venues, including Yale University and the European Institute of Innovation for Sustainability, among others.

We hope you will enjoy this edition, which aims to redefine boundaries.

Dr. Indran A. Naidoo Director Independent Office of Evaluation of IFAD

INDEPENDENT OFFICE

GUIDING PRINCIPLES

Accountability, learning, independence and partnership are the principles that guide our work

MISSION

Our mission is to promote accountability and learning through independent, credible and useful evaluations of IFAD's work.

HISTORY

The IFAD’s evaluation function has evolved over more than 40 years of existence, from being discharged by a unit internal to the IFAD management into a fully-fledged independent evaluation outfit.

QUALITY ASSURANCE

We have supported two external peer reviews of IFAD’s evaluation function conducted by independent professionals in the field of evaluation (2009 and 2019).

IOE DIRECTORS

Fabrizio Felloni Acting Director | 2020-2021

Kees Tuinenburg Officer in Charge| 2013-2014

Ashwani Muthoo Acting Director | 2012-2013

NEW IOE VISION

FORGE CORPORATE CULTURE

Contribute to forging IFAD’s corporate culture as a transparent, learningoriented and accountable organization.

Luciano Lavizzari

Director

1999-2012

Oscar A. Garcia Director

2014-2020

BUILD EVALUATION DIALOGUES

Engage with Management, Member States and external partners to support evaluation capacity and use within and outside IFAD.

IMPROVE COVERAGE

Improve evaluation coverage and promote transformative evaluations that reflect the scale and scope of IFAD operations.

Indran A. Naidoo Director

2021

RETAIN & DEEPEN LEADERSHIP

Retain and deepen IOE’s position as an internationally recognized leader in the evaluation of rural development programmes, policies and strategies.

EVALUATION OF IFAD (IOE)

IOE EVALUATION PRODUCTS

[click on icons to access database]

Aggregation level

Sub-regional Evaluations

Primary objective

Assess strategy, common intervention approaches and IFAD organizational set up in a set of countries that share salient characteristics.

Main users

Regional and country director(s), technical advisors, operational staff, and government counterparts.

Aggregation level

Evaluation Synthesis Thematic Evaluations

Primary objective

Contribute to knowledge generation by consolidating findings from past evaluations.

Main users Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Project Completion Report Validations

Primary objective

Validate the project completion reports prepared by IFAD Management.

Main users

IOE and IFAD Management for reporting (ARIE and RIDE) and feedback.

EVALUATION POLICY

Project Performance Evaluation

Primary objective

Assess the performance and results of projectlevel operations funded by IFAD.

Main users

Regional and country director(s), technical advisors, operational staff, and government counterparts.

Primary objective

Provide evidence of development effectiveness, performance and results of operations in a thematic topic.

Main users Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Corporate Level Evaluations

Primary objective

Assess the organizational performance and institutional effectiveness of IFAD.

Main users Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Impact Evaluation

Primary objective

Provide a rigorous quantitative assessment of the impact on rural poverty of selected IFAD’s operations.

Main users Regional and country director(s), technical advisors, operational staff, and government counterparts.

Project Cluster Evaluations

Primary objective

Assess the experience of several projects that have a common theme or common major component.

Main users Regional and country director(s), technical advisors, operational staff, and government counterparts.

Aggregation level

Annual Report of Independent Evaluation

Primary objective

Report all of IOE’s evaluation activities in a given year, and presents a synthesis of IFAD’s performance, lessons and challenges.

Main users

Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Aggregation level

Country Strategy and Programme Evaluation

Primary objective

Assess performance and results of country strategy and operations and provide lessons and recommendations to guide preparation of next country strategy

Main users

Divisional and country director, country team, and government.

INDEPENDENCE

Guarantees the avoidance of conflicts of interest by upholding the provisions that safeguard the behavioral, organizational and structural independence of the Office.

EVIDENCE-BASED MANAGEMENT

Reflects IFAD’s increasing focus on embracing a culture of evidence-based management to maximize development effectiveness, in which evaluation has a critical role to play

LEARNING & ACCOUNTABILITY

Clarifies that accountability and learning are objectives of evaluation, and emphasizes the effective use of evaluation products to this end

EVALUATION COMMITTEE

Presents the roles and responsibilities of the Evaluation Committee and, of its Chair, in the evaluation function on behalf of the Executive Board

Editorial Board

Proofreading Revisioning

Independent Magazine brings to the forefront of the global development dialogue the major efforts undertaken by the Independent Office of Evaluation of IFAD, while seeking to advance the organization’s vision of vibrant, inclusive and sustainable rural economies, where people live free from poverty and hunger. To present the richness of rural life, and detail facets of local community lifestyle, Independent Magazine also zooms in on cultural activities and landmark occurrences in countries featured by IOE’s evaluations.

Dr. Alexander Voccia

Laure Vidaud

Profile

Shaun Ryan

Profile

Nene Etim

Profile

THE (MISSING) LINK BETWEEN EVALUATORS

“You can be strong technically, but if your emotional intelligence is not adequate you will neither be a good evaluator nor a good manager”. Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD (IOE), drove home this message during an interactive session with students of the European Institute of Innovation for Sustainability (EIIS). Dr Naidoo explained that while having strong technical skills is fundamental, because they help cement the legitimacy of an evaluator – case in point, 50% of IOE professional staff have a PhD – these intellectual assets are only one element of what makes for a good evaluator.

Held on 27 April 2023, the questions and answers session with the IOE Director was an integral component of the ‘Measuring and Evaluating Food Sustainability’ module of a new a post-graduate certificate programme on ‘Food and Sustainability’. Co-designed by IFAD and EIIS, the certificate programme is structured to help shape the future leaders of food by providing them with the necessary tools to analyse the food system, understand its complexity and develop solutions to concretely fix it at the com-

pany, consumer, and institutional levels. To this extent, the programme covers all the main aspects related to food, from sustainable agriculture to nutritional challenges, food policy, food

INTELLIGENCE: EVALUATORS AND MANAGEMENT

education and many more. In addition to Dr Naidoo, programme instructors include, among others, Jeffrey Sachs, Director of the Center for Sustainable Development at Columbia University, and David Katz, Founding Director at Yale-Griffin Prevention Research Center.

Investing in emotional intelligence is necessary in order to pursue ‘co-creation’ among evaluators and management. Co-creating implies forward looking dialogues built on processes of genuine reflection, with the common aim of serving the poorest of the poor. Dr Naidoo is striving to introduce the concept of co-creation at IFAD, and is doing so by demystifying the evaluation process and by customizing entire reports on the basis of user needs.

“ We have moved away from purely technical exercises, to embrace open conversations. Naturally, this requires humility on both sides. Evaluators need to become much more sensitive, understand their audiences and recognize that people feel the pressure of the evaluative proces”.

- Dr Indran A. Naidoo.

Judgement is a word that triggers negative connotations. To overcome these, IOE is following a three-step process. The first is to shift the conversation from ‘you’ to ‘the organization’. It is the programme that is being evaluated, not the individual. The second step is to allow the evaluand to talk more. The third part is to enable space for self-evaluation, which implies asking the evaluand to fill in questionnaires, and engage in a process of genuine reflection.

“Management needs to look at evaluation not as being punitive, but as providing information that they cannot get. This requires a mindset shift, and the willingness to put aside egos. If management were to spend less time worrying about judgement, we could focus on how we can improve the future”, clarified the IOE Director.

In this context, Dr Naidoo underscored that while evaluation, by its nature, is about what happened in the past, IOE looks at the past in order to provide information for a better future. This is done through a set of action-oriented and practical recommendations, which form an integral part of the Office’s reports to IFAD’s Executive Board.

“The pressure from the donors is for quick results and perfect programme delivery. However, results take a long time and progress is never linear. IOE looks at a timeframe of ten years for country programme evaluations: that’s how long it takes for results to be achieved in a sustainable manner”.

- Dr Indran A. Naidoo

Ultimately, success needs to be defined from the perspective of rural people. For this reason, IOE always gives voice to the local rural beneficiaries. Results are not necessarily always quantitative. In some cases, while outputs may not be measurable, for example in terms of increased food production, evaluators may observe qualitative indicators related to happiness, to a sense of belonging, or to people feeling that they have a greater sense of agency and voice. IOE merges quantitative and qualitative information.

DEMYSTIFYING EVALUATION THROUGH COMMUNICATION AND ENGAGEMENT

“It’s important to communicate evaluation in a manner that demystifies it. To do so, I am a firm believer that independent evaluation must stimulate dialogue to help programme improvement. For this reason, over the past two years, we have almost doubled the moments of engagement between the Office and the evaluand”. Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD (IOE) highlighted this nexus between demystifying, communicating and engaging during a video interview, on 28 February 2023.

The video interview is an integral component of the ‘Measuring and Evaluating Food Sustainability’ module of the new a post-graduate certificate programme on ‘Food and Sustainability’.

“Evaluation tends to create tensions in organizations and people because they feel that it’s a judgement of themselves. What we are trying to communicate is that we are evaluating the programme, and in doing so we want to provide feedback to the implementers so that they are better able to do their work”.

- Dr Indran A. Naidoo.

For every evaluation, IOE has a series of tailor-made products that allow the Office to reach a vast gamma of audiences, and to keep the conversation going.

IOE ensures that the entire evaluation function at IFAD follows internationally recognized good standards and practices, and reflects the latest methodological breakthroughs. In this context, IOE will test evaluative approaches that draw

from the field of neuroscience, working alongside internationally renowned brain scientist, Dr Srini Pillay, in the coming months. Moreover, IOE recognizes that there may be scope for further growing and expanding this use, and to apply it to the field of evaluation. Evidence suggests that doing so would likely help evaluators to better identify the mechanisms that make policies, programmes and interventions work.

“We want to move the culture from evaluation phobia to evaluation love, so that everyone begins to understand that the reason we have rigorous evaluations is to help ensure that IFAD has the highest quality programmes for the benefit of the millions of people whom we serve around the world”, Dr Naidoo concluded.

One of the positive outcomes of independent evaluation is that it promotes good governance, which leads to transparency and accountability. It demonstrates to the world that IFAD is an organization that is not afraid to acknowledge its challenges so that it can work on them. That, in itself, is a factor of confidence

Watch the video here

Positive changes in economic opportunities, natural resource management and climate change adaptation achieved in G5 Sahel countries

hrough its portfolio of co-financed operations, IFAD has contributed to positive changes in economic opportunities, natural resource management and adaptation to climate change in Burkina Faso, Chad, Mali, Mauritania, Niger and Northern Nigeria, between 2010 and 2021. These results have been instrumental in enhancing the resilience of beneficiaries and their communities by building their absorptive and adaptive capacities, while the building of transformative capacities was ongoing. During a learning workshop, Kouessi Maximin Kodjo, Lead Evaluation Officer of the Independent Office of Evaluation of IFAD (IOE), presented these and other findings of the evaluation report titled ‘Sub-regional evaluation of countries with fragile situations in IFAD-West and Central Africa’.

“This was the first sub-regional evaluation carried out by IOE. The aim was to reinforce lessons learned based on empirical data. We worked on the topic of fragility for the G5 Sahel and Northern Nigeria. IFAD-supported interventions explicitly addressed poverty, degradation of natural resources and social inequalities”.

- Dr Kouessi Maximin Kodjo

TOrganized by IOE on 27 February 2023, the online virtual workshop brought together a wealth of high-level attendants, including Mr Diamoitou Guessibo Boukari, Secretary General of the Ministry in charge of Livestock of the Republic of Niger, and Mr Kouldjim Guido, Deputy Executive Secretary of the G5 Sahel. Over 70 participants joined the event, representing the Government of Nigeria and the Governments of the G5 Sahel countries – namely Burkina Faso, Chad, Mali, Mauritania, and Niger –, and regional bodies, such as the G5 Sahel Secretariat and the Permanent Inter-State Committee for Drought Control in the Sahel. The meeting also benefitted from the contributions of NGOs, including “Réseau Billital Maroobé et partenaires”, IFAD Senior Management and staff, and other partners met during the evaluation.

Dr Kouessi Maximin Kodjo Lead author, Sub-regional evaluation

“

Let me highlight the importance of this kind of event, which is core for the utility of the evaluation. Indeed, its purpose is to promote the effective usage of recommendations, by facilitating a discussion between critical stakeholders on the evaluation findings. As such, IOE is always keen to organize learning events to conclude such important evaluations”, underscored Indran A. Naidoo, Director, IOE.

Sub-regional evaluations (SREs) were introduced into IOE’s evaluation portfolio in 2021, to support evidence-based learning. SREs evaluate intra-regional issues or common development challenges within a defined geographical zone to identify common strategic and programmatic lessons. The sample countries of the IFAD-WCA SRE were selected, after engagement with IFAD Management, due to the similar fragility challenges they are facing, which pose threats for achieving the Sustainable Development Goals.

“This evaluation is very enlightening, very insightful, and the issues are very common across a number of regions. Fragility is poverty, it is violence, it’s a vicious cycle because fragility can cause poverty and conflict, and vice versa”, stressed Dina Saleh, Regional Director, of IFAD’s Near East, North Africa and Europe Division.

The event featured a discussion panel comprising key actors with expertise in the evaluation themes, in addition to a questions and answers session and high-level opening and closing statements. The discussions focused on a number of cross-cutting issues. These included, first, how IFAD can support governments in addressing the challenge posed by weak public institutions in fragile contexts, to ensure effectiveness and sustainability of operations. Second, how IFAD can contribute to addressing transboundary issues on a regional basis, given that cross border trade and trahumance are of great importance in the subregion, and cannot be only linked to one country strategy. Third, what strategies and operations in WCA can learn from other IFAD regions, such as Near East, North Africa, Europe and Central Asia, in terms of the nexus conflict and poverty.

“The G5 Sahel group brings value added in the region. Our actions are localized and our approach is regional. Our strategy for development and security focuses on regional coordination, and all of our projects involve at lease two countries”, highlighted Mr Kouldjim Guido.

The event also afforded opportunities to delve into the strengths uncovered by IOE’s forthcoming SRE report. In particular, participants

noted that improved farming practices led to increased yields and reduced yield variability; thus, promotion of improved cropping and animal husbandry techniques and adoption of asset-building strategies were determinant in those fragile contexts. Furthermore, capacity building and non-financial support were critical in developing the necessary human and social capital of individuals and groups, necessary in fragile situations. In addition, support to customary credit and saving groups was instrumental for smallholders’ resilience-building strategies. Other important aspects included that empowering producers’ and farmers’ organizations to deliver effectively and sustainably was instrumental to building absorptive and adaptive capacities, and that IFAD support created favourable conditions for farmers’ organizations to participate in relevant policy discussions.

“The IFAD approach in Niger, via economic development funding, has led to a number of solutions for agro-silvopastoral products and intra-state transhumance. IFAD has also helped with the coordination of local actors. This coordination has helped obstacles between borders to lessen, noted Mr Diamoitou Guessibo Boukari.

Challenges were also addressed during the

meeting. In this regard, the discussions presented the missing nexus approach addressing poverty and conflict in IFAD-supported operations in the G5+1 context. Other matters of concern included the mixed results achieved for functionality of water users’ associations for small-scale irrigation schemes, where local management committees had to play important roles, despite intensive efforts by projects in Burkina Faso, Chad, Mali and Niger. In addition, the report found that pastoralism – which is an important issue in the Sahelian context – received insufficient focus in IFAD-supported operations, over the reviewed period.

Looking ahead, it is recommended that IFAD develop a comprehensive resilience framework for the subregion or region to guide assessments, designs and implementation of operations at field, national and regional levels. It is also recommended to revisit approaches for value chain development support within the subregion to further improve the inclusiveness, and to build on community-driven approaches that were effective in highly fragile areas. Other recommendations include the need to further promote the resilience of rural communities through support to producers’ and farmers’ organizations and community-based organizations to effectively deliver services and strengthen their capacity to engage in policy dialogue on topics related to them.

EVALUATION: MOVING BEYOND WHAT IS RIGHT OR WRONG

Guest lecture by:

Dr Indran A. Naidoo

Director

Independent Office of Evaluation of IFAD

MGT 839: Program Evaluation, Spring 2023

“The Independent Office of Evaluation of IFAD [IOE] focuses on both learning and accountability. It’s not just about saying what is right or wrong. We are dealing with social data, where it is sometimes not easy to cast a judgement, and often times we are dealing with a motion picture, not a simple snapshot”, stated Dr Indran A. Naidoo, Director

of IOE, explaining the complexity and ramifications of the evaluation function during a lecture at Yale University, on 5 April 2023.

Dr Naidoo delivered the lecture as part of the course ‘Program Evaluation, Spring 2023’, of the Yale University School of Management. Led by Dr Beth Osborne DaPonte, the course introduces students to the concepts and tools used to evaluate programmes and policies. The course considers evaluation from an equity perspective and focuses on evaluation techniques and approaches that are used in practice to evaluate programmes and initiatives of non-profit, agencies, NGOs, and governments. A com-

“

IOE embraces and understands new content and methodologies that provide both real time information on change and impact. There is no gold standard, methods should feed the question, and issues need to be framed in the context of the multi-disciplinary world that we live in.

Dr Indran A. Naidoo Director, IOE

bination of Business Administration, Public Health, Development Economics, and Forestry students attended the lecture.

The nexus between learning and accountability in the context of independence was at the heart of the session. Dr Naidoo underscored that, within the rubric of a fully independent office of evaluation, it is possible to engage in learning, which is a reflective activity that comes from receiving frank and direct feedback. The prerogative of IOE is to provide this feedback, in order to ensure that the product received by IFAD’s beneficiaries on the ground is the best they can get.

“The reason for independence is a critical one. You want a unit, within an organization, that is able to conduct its work without fear, favour or prejudice, that is not under any political or administrative pressure”, Dr Naidoo clarified.

IOE advances a process of principled engagement. Traditional, static and linear approaches move from learning to sequencing. Having recognized that engagement is a part of credibility, IOE has moved beyond a transactional approach to evaluation between evaluator and evaluand. IOE’s approach hinges on a dynamic and engaging process of triangulation that is based on independence, credibility and utility.

In this context, IOE is creating a culture of receptivity to results through engaged processes, and through a new approach to strategic communication. For the first time in its over forty-year history, IOE is able to communicate with full independence, in a move that bears few precedents among evaluation offices of international financial institutions and United Nations entities., highlighted Dr Alexander Voccia, Coordinator of the Evaluation Communication Unit at IOE, who took the floor during the lecture.

Seeking to further build on these achievements, IOE is crafting communication and out-

reach strategies that build on social neuroscience foundations. This, as evidence suggests that applying these principles to evaluation would likely help evaluators to find the mechanisms that make policies, programmes and interventions work, and enhance the impact of their communication efforts.

To ensure that the substantive content that forms the basis of these processes is of the highest possible standard, Dr Naidoo explained that IOE embraces mixed-methods evaluation processes, driven by methodological pluralism. IOE considers the environment and ensures methods allow for downward accountability, moving beyond the idea that impact is the only criterion to be focused upon, and revising the obsolete principle of a ‘golden standard’ of randomized controlled trials and quasi-experimental design.

The lecture also offered an opportunity to address the conundrum of evaluation and audit offices in multilateral organizations as a seemingly insurmountable divide for joint efforts. Drawing on his experience as former Director of the Independent Evaluation Office of the United Nations Development Programme, Dr Naidoo observed that joint assessments could help promote a holistic three-dimensional view of performance, which looks both inside and outside organizations to see whether inputs matched with outputs and outcomes.

Dr Alexander Voccia Coordinator, ECU

“

The new communication product mix responds to the particular nature of IOE’s work, which calls for enhanced synergies between the Office and its stakeholders. The online solutions foster greater user engagement by speaking to the core of what IOE’s audience looks for.

Dr Indran A. Naidoo: ‘‘Research in action through independent evaluation for change’

Opportunity seized to introduce agricultural innovations in Cuba

IFAD has introduced several innovations in the agricultural sector in Cuba through the implementation of the Cooperative Rural Development Project in the Oriental Region (PRODECOR). This, as the Fund was able to take advantage of a window of opportunity that arose in the country to address the technological, productive and commercial lag in the agriculture sector. The latest report published by the Independent Office of Evaluation of IFAD (IOE) illustrates these important steps forward.

PRODECOR was the first project implemented by IFAD in Cuba following the suspension of activities in the country during the 1980s, and, consequently, has also been the first project to be evaluated by IOE in the country. The project aimed to increase the production and productivity of maize and beans (i.e. two strategic crops) and improve living conditions for producers organized into cooperatives and their families. IOE’s project performance evaluation report (PPE) analysed the project impact at the micro, meso and macro levels on the population and institutions assisted.

The cooperatives supported by the project offered mechanization services and support

for commercialization to other cooperatives with fewer resources or less organizational capacity. The establishment of this collaboration among cooperatives was an innovation in the Cuban context – but not the only one.

PRODECOR also introduced seeds of different bean and maize varieties that had been tested in the field under the leadership of the Grain Research Institute of Cuba, to optimize the use of inputs. Another innovation in the Cuban context, at the value chain level, was the grain production contracting system and payments to producers by processing plants. This system facilitates decisions by the National Bank of Cuba to grant credit to producers, as it

presupposes a degree of certainty as to product destination and selling price.

These innovative efforts were complemented by several successful undertakings. These included financing the construction of grain processing plants using modern technology, with a laboratory for quality control and grain drying. This contributed to reducing post-harvest losses and generated 320 direct jobs, beyond the 200 initially projected. In addition, the project strengthened participatory planning, including technical aspects and consultation on sowing, input procurement and financial instruments; improved the quantity and quality of technical services, extension and research; and contributed to improvements in production knowledge about the application of crop techniques, with very good adoption rates.

The evaluation found that, despite the support for farm mechanization and extension, the increase in yields was limited. Moreover, the processing plants are only at 20 per cent to 30 per cent of their operating capacity given that the volume of grain is inferior to expectations.

The key problem was that, despite the support for farm mechanization and extension,

Fabrizio Felloni Lead author, Cuba PPE

Profile

the increase in yields was limited. The first three to four years of project implementation saw a substantial rise in average yields. However, yields have declined since 2019. This occurred following the restrictions on imports of production inputs (fertilizers and pesticides), fuel and spare parts for machinery, as a result of the international sanctions imposed since 2017. Equally important to note is that PRODECOR paid little attention to the learning generated in other countries to benefit its interventions. In terms of sustainability, the project did not have a concrete exit plan with resource mobilization to address the major risks faced by the project. To date, there is no strategic plan in place to address the risks that could threaten the project achievements.

To address these and other related challenges, the IOE report recommends that future interventions in Cuba support the operationalization of national reforms on value addition to products, markets and value chains. The PPE also suggests strengthening knowledge management and the exchange of experiences with international cooperation in Cuba and with other countries in the subregion, and adopting immediate measures to strengthen the sustainability of project results.

Cuba is one of the largest small island development states. In 2013, the agricultural sector accounted for about 18 per cent of gross domestic product and employed 940,000 workers, approximately 20 per cent of the workforce. Due to undercapitalization and a lack of technological innovation, the agricultural sector has untapped potential. Currently only 2.6 million out of 6.3 million hectares of cultivable land are in use. IFAD recently resumed operations in Cuba after more than a 20-year hiatus. Given the challenges the agricultural sector faces, IFAD is serving as one of the country’s strategic partners, contributing to the ongoing modernization process.

Capturing the Havana Street Opera Co. Performing*

The Havana Street Opera is a musical theatre company, directed by Ulises Aquino. Among the functions it performs is that of training young amateur artists, who stand out for their talent. Although it is based in the Arenal theatre in Playa municipality, sometimes it performs in street locations.

Three numbers were recently presented in the middle of the street, capturing the attention of passers-by, and even interacting with several of them. The streets of Old Havana are ideal for this type of presentation, which mixes various musical genres with opera, dance and acting. Everyone who was able to witness them appreciated the freshness, joy and good taste that these talented young artists freely displayed.

Aquino, like all of Cuba, has suffered the exodus of many of the company’s members but, luckily, we can still enjoy scenes like these on the streets of Havana.

*source: https://havanatimes.org

@ Havana Times

INDEPENDENCE REMAINS UNMOVABLE BEDROCK OF EVALUATION FUNCTION

The experience of the Independent Office of Evaluation of IFAD (IOE) shows that it is possible to assert and solidify the independence of an evaluation outfit whist increasing the impact of its evaluative work by promoting engagement with senior management. Thus, safeguarding the independence of an evaluation function within a large organization, on the one hand, and fostering engagement within said organization, on the other, are not irreconcilable aspirations. Indran A. Naidoo, IOE Director, explained how IOE achieved these results during his panel presentation at the Evaluation Cooperation Group (ECG) Spring 2023 meeting.

The ECG Spring meeting took place at the headquarters of the African Development Bank – current Chair of the ECG – in Abidjan, Côte d’Ivoire,

from 27 to 28 March 2023. As customary, the event brought together the heads of the evaluation offices of the ECG member institutions and observers. The event offered participants the opportunity to focus on the ECG strategic directions, in addition to addressing four specific discussion themes, namely opportunities and challenges to improve evaluation influence; evaluating policy-based loans and budget support; development effectiveness; and innovative methods in evaluation. The two-day gathering was followed by an ECG Spring webinar titled ‘Food security and broader resilience using an evaluation lens’, on 29 March 2023.

Dr Naidoo and Fabrizio Felloni, respectively Director and Deputy Director of IOE, represented IOE during the ECG meetings. In particular, Dr Naidoo delivered a presentation during the first

discussion theme of the event and served as the Chair of the spring webinar, while Mr Felloni acted as a panellist in both the fourth discussion theme of the event and in the spring webinar.

In his panel presentation, on 27 March 2023, Dr Naidoo delved into the approach that IOE has taken to ensure a successful marriage between ‘independence’ and ‘engagement’. Over the past two years, IOE embarked on several joint undertakings with IFAD Management. These included a joint IOE-Management task force to prepare the 2021 Evaluation Policy, and the co-development of the 2022 Evaluation Manual – a first in the history of the organization –, in addition to closer coordination and cooperation in learning events, workshops and capacity building. A transformative approach to the Office’s communication and outreach has supported these efforts, through a suite of cutting-edge tools, resources and platforms that have created an enabling environment in which opportunities for dialogue and exchange have flourished.

This innovative approach to heightened engagement and communication is well-balanced with the safeguards to IOE’s continued behavioural and organizational independence, enshrined in the Evaluation Policy, which ensure that the office continues to deliver its work without any fear, favour or prejudice.

On 28 March 2023, in the context of the fourth theme of the event ‘innovative methods in evaluation’, Mr Felloni delivered a panel presentation titled ‘Harnessing opportunities from information technology for evaluation – with some words of caution’. Mr Felloni discussed the experience of IOE in the application of geo-based data for evaluation before, during and after the COVID-19 pandemic, with examples drawn from evaluations carried out in Georgia, Mexico, Nepal, Bangladesh and Kyrgyzstan. In this context, IOE’s Deputy Director also shared insights on the use of remote interviews, including remote participatory videos.

Following the closure of the ECG 2023 Spring meeting, the ECG webinar titled ‘Food security and broader resilience using an evaluation lens’ offered an opportunity to focus on what interventions, projects, and programmes are most effective and efficient in ensuring food security

and resilience. Dr Naidoo moderated the event, which moved from the premise that appropriate evidence and lessons learned from past evaluation exercises of food security interventions need to be drawn for better design of interventions and quality enhancement of future food security programmes. Discussions looked at four fundamental aspects of food security, namely availability, access, utilization, and stability.

Speaking at the seminar, Mr Felloni shared the experience of the Evaluation Offices of FAO, IFAD, UNIDO and WFP, who worked on an evaluation evidence summary on food security during times of crisis, in 2020. In his presentation, the IOE Deputy Director addressed interventions aimed at: ensuring social protection and basic services; supporting economic response and recovery; and supporting social cohesion and vulnerable groups.

The ECG was established in 1996 to promote a more harmonized approach to evaluation methodology. Members work together on joint and meta-evaluations and discuss such evaluation issues as the independence of evaluation offices in multilateral development banks, evaluability assessment, the results agenda of MDBs, and lessons learning and dissemination. The African Development Bank is the current Chair of the Group. IFAD served as Chair of the ECG in 2017, and is scheduled to serve as Chair again in 2025.

Dr Indran A. Naidoo: ‘Independence and Engagement at IOE’

Fabrizio Felloni: ‘Harnessing opportunities from ICT’

Fabrizio Felloni: ‘‘Evidence from a UN Evaluation Group Summary on Food Security During Crisis Situations’

Rural enterprise development supports increased household

incomes and employment creation

IFAD-financed projects supporting rural enterprise development have contributed to income diversification and risk mitigation for entrepreneurs, mostly by creating or strengthening pre-entrepreneurial activities and very small microenterprises. As a result, the project strategies have fallen short of supporting enterprises with potential for larger employment impact. Fumiko Nakai, Senior Evaluation Officer at the Independent Office of Evaluation of IFAD (IOE) presented these findings during an online learning event, on 31 March 2023.

“ Overall, there was often lack of clarity in whether the project focus was on supporting very small microentrepreneurial activities for household income increase and diversification and family member employment, or enterprises with more growth potential for generating employment opportunities for non-family members in the rural landscape – or both. These require different strategie s”, stated Ms Nakai.

Organized by IOE, the virtual learning event offered an op -

portunity to share and discuss the findings, conclusions and lessons of the project cluster evaluation on (PCE) on rural enterprise development, which was conducted over 2021 and 2022. The session brought together over fifty participants, and featured a panel discussion that explored issues around the overarching question of ‘how can IFAD contribute to improving the availability, quality and sustainability of business development services to support new and existing rural enterprises?’.

The PCE is a new evaluation product, the aim of which is to enhance the learning aspect of project-level evalu -

ations through comparative analyses of a small number of projects sharing common features. The evaluation presented at the learning event covered four projects that had a focus on rural entrepreneurship, enterprise development and employment creation, and that included support for non-financial services, access to finance and an enabling environment for rural micro and small enterprise development.

“ Projects generally entailed an implicit assumption that many supported entrepreneurs would grow enterprises and create jobs for others, but this was over-optimistic: in reality, for many, micro-scale entrepreneurial activities are not a choice but a necessity”, noted Indran A. Naidoo, IOE Director.

The evaluation found that the introduction of new technologies and practices was effective in improving the performance of existing enterprises through improved productivity, on-and offfarm. At the same time, there was less contribution to enterprise upgrading by better business practices or access to markets.

Fumiko Nakai Lead author, Project Cluster Evaluation

Profile

The event highlighted that the factors affecting project performance in enterprise establishment and growth included the sequencing and intensity of advisory and follow-up support, the process of screening and selecting participants with entrepreneurial aptitude, synergy with financial services, as well as types of enterprise and markets. Also, projects often underestimated the effort and time required to create, strengthen and sustain entrepreneurial activities and enterprises. However, the evaluation also positively noted that one of the projects covered in this evaluation (Cameroon) pursued the incubation approach and business coaching with a focus on youth with well-sequenced and intensive support.

“Creating and growing enterprises requires systematic, longer-term support. Efforts to improve rural entrepreneurs’ access to finance must be based on the assessment of their needs, as well as policy and institutional bottlenecks” , explained Ms Nakai.

The evaluation also underlined the importance of longitudinal and granular data on enterprise performance to better understand who participated and who benefited and to what extent, and which project interventions were most effective and for whom.

HARNESSING NEUROSCIENCE TO UNLOCK THE TRANSFORMATIVE POTENTIAL OF EVALUATION

There is scope for applying the principles of social neuroscience investigations to the field of evaluation. Evidence suggests that doing so would likely enhance the transformative potential of evaluation by strengthening evaluators’ resilience, and helping them to find the mechanisms that make policies, programmes and interventions work. Having recognized this untapped resource, the Independent Office of Evaluation of IFAD (IOE) has started to apply methods that draw from the field of brain science, and is now working alongside world-renowned keynote speaker, lecturer and author, CEO of Neurobusiness Group, and Harvard trained psychiatrist, Dr Srini Pillay. To discuss the implications and potential ramifications of this cutting-edge approach, Dr Pillay joined Dr Indran A. Naidoo, IOE Director, for an insightful discussion during the thirteenth instalment of the IFAD Innovation Talks series, on 22 March 2023.

Why neuroscience?

In the context of the United Nations, the Secretary-General recently urged all agencies, programmes and funds to explore and apply behavioural science in programmatic and administrative areas. In a similar vein, Dr Naidoo and Dr Srini Pillay believe that the time has come to apply neuroscience principles to the field of evaluation – with a view to transforming the discipline and yielding unprecedented results. Moreover, these principles are already being used to great effect in the fields of crime, justice, security, child development, education, mediation, health, and social well-being and social cohesion.

By their very nature, evaluations are meant to shine a light on the strengths and challenges that people face. While the goal may be to inspire change behaviour in the people who receive those evaluations, several bodies of research from the biological sciences indicate that the approach taken is often misfocused. Emphasizing weaknesses, flaws, or other shortcomings, or even trying to “fix” the problem has an opposite effect. Moreover, this approach will likely activate the negative emotional attractor, cuasing people to defend themselves and, as a result, to close down. Not exactly the desired outcome.

During the Talk, Dr Pillay also explained that people are more likely to reach a destination and appreciate the steps along the way if they are not simply told what to do. Being told what to do creates weaker cognitive maps in the brain because there is no figuring out process that contributes to the plan. Furthermore, many studies have also demonstrated that strengthsbased performance appraisal motivates people to change and align with goals much more than one that looks at strengths and weaknesses or weaknesses alone. While this does not mean that evaluators should only emphasize strengths, it does suggest that the feedback should highlight the capacity to grow that strength.

What is IOE doing?

IOE has already begun a process of using the principles of neuroscience. Examples are the new Evaluation Manual, which was co-created and jointly signed by IOE and IFAD Management – a first in the history of the organization. IFAD also has a new Evaluation Policy, which

reflects both independent and self-evaluation, as well as a host of dynamic communication products. These products demonstrate how it is possible to enhance engagement in a constructive fashion, without undermining the principle of independence.

These efforts have been supported by IOE’s unprecedented approach to strategic communications, which has radically evolved during the past two years, revolutionizing the Office’s capacity for dynamic user engagement. Today, the articulate communication offerings present users with an immersive experience, spanning the full spectrum of the evaluation function. Intuitive and easy to navigate, the communication product mix fosters greater user engagement by speaking to the core of what IOE’s audience looks for, bolstering direct and flexible avenues for interaction, and inviting exciting opportunities for people to create a personal connection with the Office in a dynamic fashion. Opportunities for real, tangible change have emerged, as the products are facilitating greater impact of IOE’s work.

The significance of similar undertakings notwithstanding, these results are the proverbial tip of the iceberg as IOE plans to bolster the synergetic use of neuroscience-based principles in its evaluative offerings during the course of 2023. For the first time, the Evaluation Manual will have a section on communication and

engagement, which will allow to design evaluation modules that help evaluators to engage in a way that brings about the level of dialogue that IOE is seeking. The section will be followed by a virtual online training course focused on ensuring impactful evaluative communication, and a subsequent training workshop.

About the event

Organized at IFAD’s headquarters in Rome, and live-streamed globally through the Fund’s corporate broadcast channels, the hybrid event saw the participation of over 160 attendants, including delegates and high-level representatives of the Executive Board, as well as a host of development partners, counterparts, stakeholders, and IFAD Senior Management and staff. Renata Mirulla, Facilitator of the EvalForward community of practice, moderated the session.

The IFAD Innovation Talks are a series of learning and knowledge-sharing sessions, lasting one hour. They feature innovative approaches, tools, products and services developed by IFAD, its partners and the members of the IFAD Innovation Network in order to further the aims of the network.

FEATURE STORY THE BRAIN Putting emotional

SCIENCE REVOLUTION

emotional intelligence centre stage

Being an evaluator is no simple task. That’s not only due to the knowledge and experience required to embrace this complex profession. Undoubtedly, methodological expertise, analytical acumen, data collection training, and solid writing skills are all part and parcel of what makes for a strong and sound evaluator. However, there is more – much more.

Evaluators are often met with a mixture of cautious concern and anxiety by the evaluands. The reasons for this are well-known, and are found in the fear of being judged which permeates most of mankind. It thus comes as no surprise that for evaluators soft skills are essential. Being able to communicate effectively, having a problem-solving attitude, knowing how to lead by example when necessary, and fostering open and inclusive engagement processes are abilities that all good evaluators should hold dear in their toolbox. Arguably, it is these attributes that help ensure that an evaluation report actually makes a difference, that its findings are looked upon with genuine interest, and that its recommendations are followed-up on with concrete actions. In other words, it is these attributes that help to bring an evaluation report to life. The importance of these soft skills notwithstanding, these abilities are unpredictably difficult to come across in the evaluation world. This being the case, an out-of-the-box approach is called for to break the impasse.

IOE has identified the application of neuroscience-based principles to evaluation as the cornerstone of this approach, and is moving along full steam to implement a series of mindset-changing initiatives. The previously discussed Innovation Talk, a soon-to-be-published neuro-science focused annex to the IFAD Evaluation Manual, and a virtual training on the same subject are but some of the efforts currently underway. But what does this all mean in practice? How can neuroscience actually help transform the attitudes and approaches of young and seasoned evaluators alike? To answer these and many other related questions, there are no two better people to ask than the main players in the ‘IOE brain science revolution’: Dr Srini Pillay (Assistant Professor of Psychiatry at Harvard Medical School), and Dr Indran A. Naidoo (Director, IOE).

How can we apply neuroscience-based principles to improve evaluation processes?

Srini

As an evaluator, what you really want is change, and you want change in a positive direction. If you can understand what is going on in the brain of the evaluand, you have a greater likelihood of improving that change. Research suggests that 70 per cent of change initiatives fail. Part of the reason for this is that when there is any kind of change, it activates the conflict centre of the brain. As a result, the brain is forced to go back to old ways of behaving. Organizations that develop initiatives that change mindsets, like IOE is doing right now, are two times more likely to be effective. In addition to that, if aspirations are aligned, you are 4.4 times more likely to be effective.

How is IOE testing application of neuroscience principles to improve IFAD’s effectiveness?

Indran

I always say that the evaluation process is more important than the report. This is because the learning takes place in the process, not at the end. If we have evaluators that are more sophisticated in brining about the dialogue, then we can bring about ‘evaluative conversations’. We are dealing with very complex issues, and nobody from Management is doing less than the best. Allowing voice to come in is more important than what is simply written on the last page of a report. An evaluation is much more than a score card. To help shift this mindset, we are also communicating in a more sophisticated way, with a view to make evaluation more appealing and move away from evaluation phobia.

How we can communicate evaluation findings based on neuroscience to trigger uptake of recommendations?

Srini

Traditionally, very strict evaluators come to a meeting thinking “I am going to give it to these people. They have not delivered and are not doing what they should be doing”. They feel that by being strict they can get people to do something. The problem is, we all have mirror neurons in our brain. This means that if you have the intention, albeit not overtly, to attack somebody, or you have an emotion of anger that you are not showing, this will automatically mirror in the other person’s brain. This means that the other person will go into defensive mode, and will not be able to hear you. The notion of coming to a meeting with a positive intention is very important, because you want that to mirror in the other person’s brain.

Secondly, the moment you create stress, this turns on the habit circuits in the brain and people go back to doing what they are used to, as opposed to being open to change. Third, some evaluators feel that they should tell the evaluand exactly what they think. On the surface that sounds fine. However, studies show that receiving a negative appraisal diminishes people’s trust, it puts them on defence, and they start to focus on the past rather than thinking about the future. Fourth, some evaluators think that delivering feedback based on strict goals is an effective approach. We assume that if people clearly understand a goal that they are given, then they will achieve it. However, we need to recognize that people have a multitude of goals, which are context specific. Studies show that goal-directed coaching activates the ‘fight or flight’ mechanism, and the brain goes into panic mode. Therefore, we should be aiming towards compassion-oriented coaching. Furthermore, we know that if you frame goals as “do not do this”, under stress the brain will do the exact opposite. Therefore, how you frame the goals makes a big difference.

How do you reconcile the independence construct & desire to build engagement with the evaluand?

Indran

When I first joined the UN, coming from outside the international system, I noticed that there was a trend to defend the organization at all costs, rejecting all and any critique, and shooting the messenger. Changing this mindset is important. The starting point is to understand that we share the common goal of serving the millions of poor beneficiaries in the field. I managed to walk this tightrope effectively in UNDP, where we were independent and we were constructive. There were products that we worked collaboratively on, including the National Evaluation Capacity conference, which was the world’s largest event by government participation in the evaluation context. At IFAD, the latest edition of the Evaluation Manual was done in collaboration with Management. These are small steps in moving the culture, trying to get people to become more self-critical rather than defensive. We have a very constructive dialogue with the Executive Board and Management, who take evaluation seriously. This is shown by the very large demand for evaluation products that we receive – which reflects the credibility of what we do.

“ We all have mirror neurons in our brain. This is why, as an evaluator, the notion of coming to a meeting with a positive intention is very important, because you want that constructive emotion to mirror in the other person’s brain.

- Dr Srini Pillay

What enables people to positively affect change?

Srini

There are three factors that allow people to feel motivated to change. The first is autonomy, which deals with how free people are to do what they need to do in the system. The second is competence, which deals with the skills that are needed to affect change. The third is social relations. Often, in evaluation people only focus on competence. Instead, we also need to understand if people are free enough and are being supported enough to achieve their objectives.

What hinders people from positively affecting change?

Srini

There is a concept called ontological insecurity, that postulates that when there are a large number of changes occurring at a rapid pace, this makes people feel insecure. When you are insecure, you will likely become more fixed and rigid in your own opinion.

How can an evaluator foster positive change?

Srini

Psychological safety matters. In the brain, the trust circuits and the fear circuits have the opposite impact. When there is fear, there is heightened activation in the amygdala region of the brain. Trust, on the other hand, decreases activation. People are likely to share with you what they perceive as being obstacles to their safety to the extent that they perceive that you are aligned with them. A practical consequence of

this is that evaluators should, first, examine their intentions. Moreover, studies have shown that the person that is chosen as a leader in a group is the person that first synchronizes his or her brain with others. In other words, the leader comes into a group with the intention to be on the same page as the people who form said group. For evaluators, this means that it is important to have the same intention as the evaluand, which is to affect change, and to want to help the evaluand to overcome some of the obstructions that are impeding that change.

To what extent are the principles of engagement applied across the UN system in the evaluation context?

Indran

The reality is that there is a cookie cutter approach. Reports are wanted in a certain format, so many days are allocated, and rigid deliverables are requested. In this current process, which are highly bureaucratic in the UN, it is difficult to find the spaces for deeper discussion and engagement. There are very few offices that have that latitude. In collaboration with Srini, at IOE we are coming up with a training module that will reflect on how best to integrate neuroscience practices with evaluation. However, we are the exception to the rule.

How can we overcome habit pathways that inhibit positive change?

Srini

From a neuroscience perspective, we must start by looking at how we can make the process more collaborative and enjoyable, even if it’s serious, and make it such that we recognize that all human beings have some kind of vulnerability. In addition, we must recognize that fundamental to change is agility. The world is changing at an alarming pace. Moreover, any plan or strategy is destined to change and evolve. Therefore, we need to ask ourselves how we can create a culture that flourishes in agility. First, we must recognize that decreasing stress will decrease habits. Second, we must appreciate that it’s important to have a culture of agility. Third, we must understand that agility is linked to responding faster to failure, which happens when people have growth mindsets. Studies show that if you believe that you can course-correct after you have made a mistake, this can help you respond to errors more quickly.

How important is the human element in evaluation?

Indran

Evaluation is a human business. It’s not a technological application, where you type in a function and you get a result. More important than the result is the conversation that brings about the change. Evidence has shown that over 170 countries in the world have some kind of evaluation function, and have embraced the notion of evaluation. With that said, most evaluations look at one or more issues related to good governance. Clearly, we do not have good governance around the world. We see high levels of unemployment, corruption, human rights abuses, etc. This means that having information does not translate to being able to change the situation.

In light of this, evaluators need to be modest, given that they play an important but small role in the over-

“ We aim to co-create findings that help the organization to improve effectiveness. We are moving away from the ‘them and us’ construct, since there is space and capacity to have a more creative and constructive dialogue. ”

- Dr Indran A. Naidoo

all ecosystem that brings about change. They need to be humble in terms of what they can and cannot do. They need to recognize that just having good research and good methodology on its own will not bring about change. They need to recognize the importance of chemistry, which is the ability to negotiate in the real context and facilitate the change that needs to happen. They need to understand that evaluation is a professional function that operates in a political context.

How can we handle conflicting interests between management and evaluation by applying brain science principles?

Indran

We agree to disagree, agreeably. We need to recognize that you are not going to get full consensus. There are different individuals, there are different perspectives. That’s fine. In a democracy it’s fine. I don’t subscribe to the notion that there needs to be full consensus. I think that there will always be different interpretations to issues, provided the interpretations are not so broad that there is total disagreement. If you look at the evaluations that flow out of IFAD, over 70 per cent of the recommendations are agreed upon. The other 30 per cent may have partial agreement or disagreement. That’s in-line with international organizations; it’s not a problem. What is important is to recognize that once we have the result, we need to find a way to co-create a future to act on the results.

As a professional outfit, our job is to examine performance. We put this on the table because someone else will implement our findings and take them to the next stage. For this to happen effectively, Management needs to appreciate that evaluation is no about them, and to get away from the notion that a policy written in Rome or New York is going to translate into brilliant results across the world. It does not work that way. Everything that a multilateral development organization does is mediated by multiple factors and, eventually, a result, whether good or bad, has very little to do with the person that put it on the table. The challenge is given by the element of ego: “this is the policy we are putting in place, it must work, and therefore any result that shows that it doesn’t must be flawed by wrong measurements”. There needs to be a re-think in the international system to recognize that, whilst multilateral institutions have high profile, there are multiple players on the ground, including civil society, NGOs and governments, all of whom contribute to the final result. Therefore, we must get away from the policy of the UN that brings about the

change – this is not how things work in reality.

How can we decrease stress and anxiety in the context of an evaluation?

Srini

There are principles, based on brain science, that can help us decrease anxiety. Once structure that I use is called C.I.R.C.A. (there is also a free download of the CIRCA app online). The first ‘C’ stands for ‘chunking’. When you open your e-mail box, and you see a long list of unread messages, you should chunk it up into what you want to do immediately, what you must do, what you may do, and what you will do later. The ‘I’ stands for ‘ignore mental chatter’, which is a mindfulness practice. Stress in the workplace can create change in our genes that eventually lead to chronic diseases like cancer, heart disease, stroke, and neurodegenerative disease.

Mindfulness can protect the genes. Mindfulness is sitting back in the chair, paying attention to your breath and focusing on the here and now. The ‘R’ is ‘reality check’, which relates to the notion that ‘this too shall pass’. Sometimes you get bad news, there is no good way of looking at it, and you just need to remind yourself that something better will come tomorrow. The second ‘C’ is ‘control check’, which relates to the

need to distinguish between ‘what you can control, what you cannot control, and how do you know the difference’. If you can discern these two, you will not be occupying your brain’s attentional centres with irrelevant information. The last ‘A’ is ‘attentional shift’, which is ‘how do you shift your attention from the problem to the solution’.

What constitutes objectivity? Indran

We often think that, as evaluators, we are empirical, factual, and what we present is the reality. However, evidence is proving that qualitative information is just as important as quantitative facts and numbers. In fact, the range of what constitutes evidence is much more complex than what we may typically look at. There is no one methodology or one-size-fits all approach that may be universally valid. We hope that this conversation on neuroscience-based principles is going to make people around the world more reflective, more thoughtful, so that before they go into the process of conducting or presenting an evaluation they are more reflective and understand how information is received on the other side. We believe that the marriage between evaluation and brain science can work, and we are going to advance it as best we can at IOE.

IFAD contributes to Kyrgyzstan’s pioneering pasture reform and privatization of veterinary services

Kyrgyzstan is considered a pioneer in terms of the pasture reform as well as the privatization of veterinary services, in the region. IFAD’s support, in effective collaboration and coordination with other international development partners such as FAO, GIZ and the World Organization for Animal Health, has made a visible contribution to these achievements in the country. Fumiko Nakai, Senior Evaluation Officer of the Independent Office of Evaluation of IFAD (IOE), highlighted this aspect during the learning workshop that presented the findings, conclusions and recommendations of IOE’s Country Strategy and Programme Evaluation (CSPE) report in Kyrgyzstan, on 1 March 2023.

“ There were a number of innovations supported in the country programme, which were complemented by other partners. These include support to pasture reform implementation and institutional strengthening, veterinary law permitting the private sector veterinary reorganization, and scholarships given to young students to help them become veterinaries, among others ”, explained Ms Nakai.

Profile

Organized by IOE, in collaboration with the Government of Kyrgyzstan and IFAD’s Near East, North Africa and Europe Division (NEN), the online virtual workshop brought together over 60 participants drawn from the Government of Kyrgyzstan, partner organizations, research institutes, non-governmental organizations, associations, development partners, and IFAD senior management and staff. Mr Nurdin Alisherov, First Deputy Minister of Agriculture, delivered an opening remark representing the Ministry of Agriculture.

“ The important areas of the country programme highlighted by the evaluation in -

clude systematic support to the livestock sector, pasture reforms, veterinary institutions and private veterinarians, training and preparation of young veterinarians, food security and food safety, animal identification and tracking, early warning for pasture users, livestock breeding, and value chain development ”, stated Mr Alisherov.

Meeting participants confirmed that interventions around pasture management and veterinary services were comprehensive and encompassed multiple levels, from policy and legislative frameworks, institutional development, research and education at national level, to concrete activities at field level. The impact on institutions and policies around pasture management and veterinary services was particularly far-reaching.

“ The evaluation noted major achievements as a result of IFAD’s strategic orientation. Notable results on the ground included improved veterinary services, reduced incidence of animal and human diseases, and better access to remote pastures ”, affirmed Indran A. Naidoo, IOE Director.

Fumiko Nakai Lead author, Kyrgystan CSPE

Workshop discussants also noted that the IFAD portfolio facilitated a more balanced use of pasture ecosystems with seasonal rotation, but that this has not been sufficient to reverse – or even to halt – deterioration of pasture productivity over a long term. There is a general consensus that a continued and substantial increase of livestock number in the past years is the most plausible explanation for this decline.

The support to value chain development has also faced numerous challenges. Overall, there was a lack of conceptual clarity in terms of how the interventions were expected to leverage investments and facilitate pro-poor value chain development, instead of subsidizing the operations which were ongoing or would have occurred anyway without the project.

There was also a lack of strategic approach at country programme and project level to promote gender equality and women’s empowerment. The portfolio did not make adequate efforts to challenge the social norms, which have limited women’s participation in project activities and decision-making.

To address these challenges, it is recommended that, in preparation for the new country strategy, IFAD conduct a diagnostic analysis of rural poverty and livelihoods. Other recommendations include adopting a strategic approach to propoor value chain and cluster development, articulating

the additionality and impact pathways for the rural poor, focusing on consolidating the achievements in pasture management and veterinary services and their sustainability, and strengthening the approach to supporting gender equality and women’s empowerment.

@

Nurdin Alisherov, First Deputy Minister of Agriculture of Kyrgyzstan.

“ The findings and lessons of this evaluation will directly feed into the upcoming design of the new country strategy. The country strategy will take as its starting point the Government of Kyrgyzstan’s priorities for the agricultural sector set out in the National Sustainable Development Strategy 2018–2040, which aims at supporting poor rural people to improve their productivity, competitiveness, and diversify their income ”, highlighted Ms Dina Saleh, NEN Regional Director.

After its independence in 1991, Kyrgyzstan implemented a series of structural reforms to transit to an open market economy. Since 1995, seven investment projects have been approved for a total cost of US$254 million, with IFAD financing of US$129 million.

Kyrgyz film wins two awards at international festival in Los Angeles*

The film by Kyrgyzstani Guzel Duishenkulova won the International Film Festival of Women Directors in Los Angeles, the Cinematography Development Fund of Kyrgyzstan said.

Her film “Kelin Tash” (Bride Stone) brought victory in two categories: “Best Feature Short Film” and “Best Female Director”. The script of the film is based on the story of Topchugul Shaidullaeva.

In a Kyrgyz village there is a stone that looks like a woman. According to local beliefs, a young woman who had argued with her husband and father-in-law runs away from home. The father-in-law puts a curse on the woman, who supposedly turns to stone.

The IMDb LA Independent Women Film Award is in its tenth year. It is open to women authors of both American and international media projects and films.

Approaches to assessing Lessons from a recent

Strengthening the capacities of the most marginalized who are facing crises is a complex and challenging endeavour. The pathways to help them achieve the necessary adaptive, absorptive and transformative capacities should match the responses at the individual, community and institutional level to address the needs of these marginalized.

Profile

Dr S. Nanthikesan, Lead Evaluation Officer at the Independent Office of Evaluation of IFAD (IOE), highlighted this point during the 2023 EvalXchange event, on 11 May 2023.

Organized by the World Food Programme (WFP) Office of Evaluation, the EvalXchange session to which IOE’s Lead Evaluation Officer contributed to was titled ‘Changing lives: Rising to the challenge of evaluating country capacity strengthening’. The event brought together experiences of different organizations and experts to take stock of the current evaluation practices and tools to assess country capacity strengthening (CCS) to implement the SDGs, and the challenges that remain. A group of expert panellists addressed the event, which built on interactive question and answer segments. Over 90 participants joined online, including evaluation practitioners, development partners and other stakeholders.

Speaking at the event, Dr Nanthikesan drew from the experience of IOE’s ‘Thematic Evaluation of IFAD’s Support to Smallholder Farmers’ Climate Adaptation’ to illustrate the conceptualization and method of evaluating capacity. In his presentation, IOE’s Lead Evaluation Officer

Dr S. Nanthikesan Lead Evaluation Officer IOE

assessing capacity development: IOE thematic evaluation

outlined multiple response pathways to climate threats, including support to absorbing the damage, strengthening preparedness, and enhancing learning and facilitating system change. Each pathway involved action at the individual, community, and institutional levels. This multi-pronged approach has the potential to increase the capacity to absorb climate shocks while continuing to function; to face hazardous events and reorganize after the events; and to shift to a new mode of system behaviour when continuing along the same trajectory becomes untenable.

Multiple interrelated issues were addressed during the event. In particular, discussions included the strategies to understand the complex and interconnected nature of country capacity strengthening; the critical importance of approaches that start with and end in addressing the demands of those who are left behind; and the context-dependence of the solutions to capacity development.

This was the third edition of the EvalXchange series. Other sessions of the 2023 edition included ‘Timely evidence for humanitarian support: Early learnings from impact evaluations using high- frequency data’, and ‘Disability inclusive practices in evaluation: Where are we and what can we learn?’. Cumulatively, the event drew the interest of close to 600 participants. EvalXchange is a series of virtual learning events, organized by the Office of Evaluation of WFP.

IFAD helps rural citizens in Colombia regain trust in the State

IFAD funded interventions in Colombia have been particularly successful in helping rural citizens and beneficiaries to begin to regain trust in their own organizations and in the state. This has been a key element in advancing the peace process. The gradual increase in coordination – through agreements and joint activities supported by IFAD projects – among local associations and institutions at the territorial level was instrumental in this regard. This was one of the findings of the Country Strategy and Programme Evaluation (CSPE) report in Colombia, soon to be published by the Independent Office of Evaluation of IFAD (IOE), which was discussed during a learning workshop on 21 March 2023.

“IFAD-supported interventions have strengthened the social tissue and the social fabric, and promoted the empowerment of the different vulnerable population groups. This is most important for the Government, because our aim is to achieve total peace”, affirmed Jorge Ricardo Torres Rueda, Director of International Cooperation, Ministry of External Relations of Colombia.

Co-organized by the Government of Colombia and IOE, in collaboration with IFAD’s Latin America and the Caribbean Division, the online virtual

workshop brought together a wealth of high-level attendants, including Mr Torres Rueda. Over sixty participants joined the event, representing the government of Colombia, key project implementing partners, development partners, representatives of beneficiaries associations, and others, as well as IFAD senior management and staff.

ban/rural dynamization.

“IFAD-funded interventions helped restore social capital and cohesion. This helped create a more dynamic environment for creating jobs and income opportunities in rural Colombia. It also promoted rural-urban economic interactions and economic growth in intermediate towns” stated Indran A. Naidoo, IOE Director.

The evaluation covered the period 2008–2022, and was the first CSPE to be carried out in the country. IFAD-supported projects have been oriented towards rural populations with few assets, post-conflict territories (in recent years), women, youth, Indigenous Peoples and Black, Afro-Colombian, Raizal and Palenquera communities present in poor territories. Support was also provided for initiatives and business plans for citizens’ associations located in medium-sized towns and cities, key areas for greater ur-

Through these interventions, the evaluation found that IFAD has been a pioneer in bringing approaches and mechanisms into the public policy sphere to address rural poverty through a non-assistance-based model. For two decades, it has promoted projects aimed at small-scale producers and entrepreneurs, with broad and diverse national coverage. These efforts have responded to the demands of local actors, facilitating their access to public resources and ownership of initiatives, through an approach aimed at rebuilding the social fabric and strengthening the resilience of households and associations.

In highly insecure and conflict-affected rural scenarios, strengthening of the resilience of families and associations was an important accomplishment, which was made possible by a set of coherent innovations that included participatory approaches that

Jorge Ricardo Torres Rueda, Director of International Cooperation, Ministry of External Relations of Colombia

were suited to meeting the varying demands of smallscale producers and entrepreneurs. These resulted in increased beneficiaries’ income, jobs and physical assets and improvement of food security through diversification of production and income-generating activities. The projects also promoted a high level of participation by women. This translates into thousands of women involved in economic, political and citizen empowerment activities.

sented in the evaluation report. These include limitations in the continuity and quality of access to technical and financial services, the absence of an explicit strategy for scaling up and low synergy with other agencies and private and public entities, which affects the maximization of impacts. As a result, it was thus difficult to see gradual progress in institutionalization within the Ministry of Agriculture and Rural Development or the securing of new resources.

Another aspect in need of greater attention is the lack of a broad environmental, natural resource management and climate change adaptation strategy at the level of territories and areas of intervention. In part, this hindered the use of lessons learned and alternatives to develop adaptation and resilience strategies in fragile agroecosystems.

“Vulnerable people are at the centre of our programmes. Funding helped to promote rural development that put at the centre the small-scale producers who live in poverty. Providing opportunities to vulnerable groups was made possible by the virtuous combination of funding and capacity building necessary to be able to manage resources. For the first time, many of these resources were managed directly by the beneficiaries”, said Donal Brown, Associate Vice-President, Programme Management Department, IFAD.

Meeting participants also discussed the challenges pre-

Looking ahead, it is recommended that IFAD channel a larger portfolio of resources to expand and dynamize its country strategy and programme, substantially increasing its portfolio in Colombia. IFAD may also consider developing a dialogue with the new Government and key partners to agree, design and implement a relevant co-financed portfolio of projects and grants, as well as building a communication strategy aimed at sharing and using results as public goods.

During the period covered by this evaluation, the total portfolio cost was US$125.1 million, with total IFAD financing of US$50.3 million.

Donal Brown Associate Vice-President, Programme Management IFAD

Cali Salsa declared Intangible Cultural Patrimony of Colombia*

The fast steps, gyrating hips and sultry moves of Cali’s legendary dance – salsa caleña – has been recognized by Colombia’s Ministry of Culture as Intangible Cultural Patrimony of the Nation; and important step toward the eventual listing of the dance, with all its cultural manifestations, on UNESCO’s List of Intangible Cultural Hertiage (ICH).

“For Cali this is historic, and has global resonance. For decades we have received the unconditional nickname of World Capital of Salsa and this patrimonial exercise allows us to ratify our international reputation,” said Ronald Mayorga, Cali’s District Secretary of Culture. “We are willing to take care of what has been achieved, to preserve and protect all these narratives, stories, and productions that Cali has created over the years. The challenge ahead is to take new steps, to think about the future of salsa and salsa for the world,” he added.

In a city home to salsa academies, dance clubs, music ensembles and traditional record shops with collections of vinyls, salsa caleña is recognized among the fastest given that music is interpreted at 45 rpm compared to other tropical genres at 33 rpm. The announcement declaring salsa as intangible patrimony was made by Minister of Culture Angelica Mayolo to mark the city’s 486 anniversary of its founding.

From the old-theques or viejotecas where Sunday dancing in a romantic slow-paced style unravels to the melodies of musical greats such as Richie Ray and Bobby Cruz, to the sprawling dancehalls in nearby Juanchito with their crossover rhythms and frenzied crowds, the departmental capital of Valle del Cauca is alive to the sound of music.

“Salsa speaks to the people,” believes Carlos Trujillo, founder of one of the most respected dance academies in Cali. At Rucafé, dance lessons start in the morning and continue into the late hours of the night. After having participated in international salsa congresses, Trujillo has seen his city evolve into the world capital of salsa. “It’s a lifestyle which breaks through class divides,” says the professional choreographer.

With more than 8,000 professional dancers in a city known as “the outpost of heaven,” many dancers have hit the global performing circuit to teach their steps to others. For Trujillo, who produced the acclaimed Delirio dance/musical with its Broadway-style show, “salsa caleña is the great social integrator.”

During the 1970s, Cali’s predominantly white, European, social fabric was transformed when a wave of black migrant workers from the Pacific arrived in the valley to harvest sugarcane. With this movement of displaced ambition and dreams, salsa took root in small towns overlooking the banks of the Río Cauca, and physical divide between an affluent industrialized city and the ramshackle huts of the dispossessed.

Jairo Varela bridged this social divide composing songs etched in the Colombian promise. The release of ‘Al pasito’ in 1979, launched the Afro-Colombian poet, novelist and lyricist as the country’s ambassador of salsa. With the acclaim of Varela’s first album, the internationalization of Colombian salsa began, and one of the city’s legendary ensembles, Grupo Niche, catapulted to international stages, accompanied by fleet of dancers.

Other big bands followed with their own brassy sounds, chart-stopping hits and heart wrenching refrains. Cali’s Golden Age of Salsa was born. But only ‘maestro’ Varela could baptize an entire city with a song “Cali pachanguero,” and the panchanga (word that fuses ‘dance’ with ‘party’), lives on, and now indelibly emblazoned as patrimony of a nation.

PERFORMANCE IS KEY TO STRONG EVALUATIVE CULTURE

“An organizational evaluation culture is one which appreciates and deliberately demands data, seeks empirical information on the achievement of its outcomes, and uses findings to improve its performance”. With this statement, inspired by the works of John Mayne*, Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD (IOE), began his presentation during the seminar titled ‘Building an Evaluation Culture: the experience of IFAD’s IEO’, on 23 January 2023.

Organized and hosted by the Evaluation Unit of the Pan American Health Organization (PAHO) Planning Budget and Evaluation (PBE) department, at its headquarters in Washington D.C., the event afforded the fast-maturing Evaluation Unit an opportunity to learn about UN experiences in advancing an evaluation

culture, and to share insights and solutions about challenges in building said culture.

The list of participants of the seminar included the second highest ranking PAHO official, Deputy Director Mary Lou Valdez; as well as Luis Jimenez, Director of External Relations, Partnerships & Resource Mobilization; Rony Maza, Director of Planning, Budget and Evaluation (PBE); Roberto La Rovere, Senior Advisor, Head of PAHO’s Evaluation Unit, who moderated the event, and a host of senior professional staff covering a wide range of areas of technical expertise. In addition to Dr. Naidoo, IOE was represented by Fabrizio Felloni, IOE Deputy Director, who delivered a presentation alongside the IOE Director.

The IOE presentation focused on principles and practices to strengthen an organizational evaluation culture, improve transparency, and build ac -

countability. Dr. Naidoo and Mr. Felloni spoke about the 2021 IFAD Evaluation Policy, including the common core principles of independent and self-evaluation, and the safeguards to IOE’s continued independence. The presentation also addressed the key elements of the Office’s first multi-year evaluation strategy, and touched upon the significance of the new edition of the IFAD evaluation manual – the first to be cosigned and jointly published by IOE and IFAD management. The new strategic approach to evaluation communication, which has led to a complete overhaul of IOE’s digital presence, online reach and perceived brand value, rounded off the discussions.

As part of the knowledge exchange session, the IOE delegation visited the Incident Management System Team (IMST), which is the operational heart from where all PAHO health emergency responses are coordinated, including the COVID-19 pandemic. Through the regional IMST, created in January 2020, PAHO – also serving as WHO’s Regional Office in the America - provided ministries of health and other national authorities with direct emergency response to the pandemic.

The PBE department implements PAHO’s evaluation policy, applying a results-based management approach in line with the organization’s mandates, policies, and strategies. In particular, the evaluation unit in PBE evaluates how and why results were or were not achieved and how corporate performance can be improved.

PBE, as the custodian of the evaluation function, in PAHO implements the Corporate Evaluation Work Plan, approved by the Executive Management on the basis of organizational requirements, relevance or utility. The PBE Department is accountable to the Director and Deputy Director of PAHO, and the evaluation function and Unit at PAHO collaborates with the WHO Evaluation Office and with the United Nations Evaluation Group (UNEG).

Seminar presentation by Indran A. Naidoo, IOE Director, and Fabrizio Felloni, IOE Deputy Director

COVID-19 EXPERIENCE UNLOCKS INNOVATIVE METHODS FOR EVALUATION

The Independent Office of Evaluation of IFAD (IOE) did not back down during the height of the COVID-19 crisis, and continued to carry out its evaluative work.

To do so, IOE made use of a number of innovative solutions, whilst maintaining methodological rigour and credibility. Today, these resources offer important opportunities for the evaluation world to enhance its toolbox. Fabrizio Felloni, IOE Deputy Director, presented these experiences during the course of a high-level conference titled ‘Picking up the pace: Evaluation in a rapidly changing world’, on 6 March 2023.

Organized by the Evaluation Division of the European Investment Bank, the conference addressed how evaluation functions are adapting to respond to an increasing demand for just-in-time evaluation insights and how this is fostering innovations in the field of evaluation. Participants included high-level delegates and practitioners from multilateral and bilateral finance institutions, the European Commission, governments, and academia.

Mr Felloni acted as a speaker during the first breakout session, titled ‘Innovative digital tools for evaluation – Remote sensing’. In his presentation, IOE’s Deputy Director provided technical and methodological insights into the use of innovative approaches for enriching data collection methods during crisis response and volatile times.

What were challenges, at the onset of the COVID-19, became opportunities to revisit some of the evaluation practices and processes that can be considered not only for future crises but also for ‘normal’ times, to add evidence and simplify processes.

For IOE, these included rediscovering under-utilized secondary data; conducting remote interviews with national counterparts and mini-surveys where feasible; increasing the use of national consultants; introducing additional steps and processes to strengthen quality and validity of data; and testing alternative methods for primary and secondary evidence collection and validation, such as remote sensing, GIS, and satellite data for evaluation, which was especially useful for projects investing in physical infrastructure.

Mr Felloni further confirmed that, in terms of using information communication technology, for IOE the emphasis currently remains on the use of low-cost tools, involving time-bound consultant input, such as Geo-based data, as an additional source of evidence analysis to complement traditional evaluation methods. Looking ahead the Office plans to make use of artificial intelligence, which offers opportunities to synthesize from a very high number of sources and interface between different databases.

The experience of IOE in conducting evaluations during Covid-19.

Fabrizio Felloni, Deputy Director, IOE

Profile

More responsive evaluations needed to understand environment-development nexus

“We need to evaluate the nexus between environment and development. It cannot be one or the other. We need to understand the unintended consequences of any development intervention”, affirmed Juha Ilari Uitto, Director of the Independent Evaluation Office (IEO) of the Global Environment Facility (GEF), during meeting at IFAD headquarters, in Rome, on 1 March.

Organized and hosted by the Independent Office of Evaluation of IFAD (IOE), the in-person knowledge exchange session provided an opportunity to discuss the role that evaluation can and should play in better understanding the nexus between human and natural ecosystems. In this context, Dr Uitto – who started his illustrious career as a Junior Professional Officer with IFAD, many years ago – presented the approach being taken by the GEF, broadly, and by GEFIEO in particular.

“We need faster evaluation cycles, more responsive evaluations, more formative evaluations, real-time feedback, and more collaboration with programme proponents, without sacrificing independence which is crucial for credibility. We need to narrow down questions that we can answer more quickly. Not every evaluation needs to be a massive enterprise, that uses all DAC criteria”, emphasized Dr Uitto.

Moving from the well-documented premise that people and the environment interact in many

complex manners, international organizations cannot continue doing development projects in a narrow silo, in a business as usual manner. The focus of attention must shift to researching how elements are interlinked, and how human and natural systems are connected. Luckily, there is plenty of existing evidence on these interlinkages from science and past studies, so we do not have to start from scratch.

“You can’t separate people from the environment, since virtually all root causes of environmental change are in the human domain”, further explained Dr Uitto.

Cognizant of the above, the GEF is now focusing its efforts on integration. The new GEF-8 strategy is built on the Healthy Planet, Healthy People framework, recognizing the interdependence between human wellbeing and a healthy environment. The Facility’s investment through integrated programmes is not only to generate global environmental benefits, but also to create innovative pathways for transforming these systems toward durability and resilience.

To support these efforts, GEF-IEO conducts accountability and learning-focused evaluations that generate lessons learned for the GEF.

EVALUATION PRACTICE EXCHANGE

Gender transformative approaches offer a key toward equality in rural development

Wseek to tackle inequalities by engaging women and men together in addressing underlying gender barriers. In particular, they can shift constraining gender norms, which are a critical leverage point for enhancing gender equality.

Dr Mónica Lomeña-Gelis Senior Evaluation Officer IOE

hile gender transformative approaches have become increasingly recognized as important, there are critical gaps in evidence regarding how they work and their outcomes. But, if better evidence can be built and applied, gender transformative approaches offer a potent opportunity for development programs and investments to make lasting progress toward equality. Moving from this premise, Mónica Lomeña-Gelis, Senior Evaluation Officer at the Independent Office of Evaluation of IFAD (IOE), delved into the challenges of evaluating transformational change towards gender equality in rural development, during an Evaluation Practice Exchange (EPE) seminar of the United Nations Evaluation Group (UNEG).

Held virtually on 23 March, the EPE session prompted reflection on the challenges of evaluating gender transformational change, shared experiences on tackling these challenges, and discussed how IFAD is currently evaluating Gender Equality and Women’s Empowerment (GEWE) and gender transformative approaches.

Gender transformative approaches represent a shift in how we think about and approach. They

“Examples of GEWE practices contributing to gender transformative change include functional skills training to women, gender awareness training, including engaging with men and traditional community leaders, leadership skills training to women, and policy engagement at regional and national level to ensure gender issues are on the agenda, among others”, noted Dr Lomeña-Gelis.

From an evaluative perspective, the question arises as what indicators are being used to assess gender transformative change. More broadly, the issue becomes how to ensure a shared understanding of ‘transformative change towards gender equality’, with a view to avoiding the concept losing credibility by being co-opted or mal-used.

In her presentation, Dr Lomeña-Gelis explained that IFAD IOE has been evaluating GEWE as a stand-alone criterion since 2011. The new 2022 IFAD Evaluation Manual includes more information about how to assess GEWE in IFAD operations, as well as those projects including Gender Transformative Approaches. The Manual calls for evaluators to consider differential impacts by gender and the way they interaction with other forms of discrimination, using an intersectional lens. These include age, race, ethnicity, social status and disability.

Evaluating sustainable pathways to climate resilience

n the context of food security, it is imperative that evaluations of climate responsive agricultural interventions consider adapting to climate change together with their consequences to the surrounding ecosystem. This requires evaluators to have the relevant methods, guidance and tools necessary to evaluate interventions in food systems. Dr S. Nanthikesan, Lead Evaluation Officer at the Independent Office of Evaluation of IFAD (IOE), delivered this message during an Evaluation Practice Exchange (EPE) seminar of the United Nations Evaluation Group (UNEG).

Titled ‘Evaluating sustainable pathways to climate resilience: Recent experiences from evaluations of IFAD, FAO and GEF‘, the event was held virtually on 29 March 2023. Over 70 participants joined the EPE session, which explored emerging experience from recent evaluations in terms of methods, guidance, and tools to help evaluators assess the climate change related aspects of interventions in food systems.

Climate threats are increasing in frequency and intensity, and their consequences are disproportionately affecting the most marginalized

Ipeople in the agricultural sector. At the same time, agri-food systems account for more than one third of global greenhouse gas emissions that caused climate change. For this reason, and in light of the fast-approaching targets of the Agenda 2030, there is an urgent need to ensure that food security is attained along with strengthened resilience of people and ecosystems and, at the same time, that agri-food systems are transformed to be part of the solution. However, very little evaluative experience is available in this area.

Against this backdrop, Nanthikesan discussed the qualitative approaches developed to evaluate the effects of agricultural interventions on the eco system (human – eco system nexus), as part of IOE’s recent Thematic Evaluation of IFAD’s Support to Smallholder Farmers’ Adaptation to Climate Change. Considerations were given to eco system aspects pertaining to water and air quality, water and land use, and soil health.

Based on the findings of the Thematic Evaluation, Nanthikesan stressed the importance of moving away from anthropocentric approaches to agriculture. In particular, the evaluation found that in light of the accelerating climate change effects, there is an urgent need to develop climate adaptation solutions that simultaneously promote not only climate and economic resilience, but also environmental resilience. Where that is not feasible, we should aim to ‘do-noharm’ or better, with the necessary offsets to compensate for the damage that is being done to the ecosystems.

Dr S. Nanthikesan Lead Evaluation Officer IOE

GEO-BASED TOOLS COMPLEMENT TRADITIONAL

“The new geo-based tools at our disposal allow us to deal with evaluative questions that we could not address in a satisfactory manner in the recent past. This is especially true for natural resource management and climate change projects”.

Fabrizio Felloni, Deputy Director of the Independent Office of Evaluation of IFAD (IOE), made this point during the course of a podcast. In the interview, Mr Felloni explained that geobased tools are an important complement to existing evaluation techniques.

The interview with IOE’s Deputy Director was part of the Evaledge Podcast Series, which is an initiative of the European Evaluation Society (EES). The series focuses on the use of frontier technology in evaluation and on methods to evaluate innovations. In the latest episode, the Evaledge team presented IOE’s experience in using innovative technologies such as geospatial data in evaluation.

Today, geo-based tools and satellite imagery present multiple interesting opportunities for evaluators all over the world. An important starting point is to recognize that some projects are more amenable to these techniques, such as those that have produced visible infra-

structure that can been seen from the sky. Other examples of areas of use may include soil characteristics, vegetative cover and climate change impacts and interventions. For all of these, free and easy-to-access geo databases may complement investigations on-the-ground with valuable evidence. In the future, geo-based tools could even help to make field visits more cost-effective and time-efficient, including by allowing to analyse project sites in advance.

IOE’s interest in the use and application of geo-based tools for evaluation dates back to in 2017, when they were first used to compare deforestation rates in Mexico in sites that were supported by IFAD projects as opposed to locations that were not. The analysis showed a big difference. Since then, IOE has adopted a simple but effective approach.

“We have not discontinued the traditional way of conducting evaluations. We still think that it is best to go on-site, see the reality on the ground and talk to people. At the same time, we recognize that availability of geo-based data allows us to look at things that we could not access in the past. What we do is to look for solutions that have low costs and involve limited inputs from external specialists”, Mr Felloni explained.

Guidance note on the use of Geospatial tools and applications

TRADITIONAL EVALUATIVE APPROACHES

For example, last year, during an evaluation, IOE used geo-data to ascertain that the carbon content of the soil in a certain location was very low. This information – which was obtained at no extra cost –corroborated the insights gathered through the field interview process.

Limits to the application of geo-based tools by evaluators persist, nevertheless. While in some cases the nature of a given project may not be suited to the use of these resources, or evaluative questions may be such that geo-data cannot offer any assistance, in other cases evaluators may simply not be aware of the geo data that exists, and which can be accessed at little or no cost. Moreover, there may be a certain degree of scepticism towards these innovative approaches, especially among evaluators who may not have been exposed to these technologies.

For this reason, it is important to have guidance that help introduce people to these resources. In this regard, IOE has recently published its first guidance note of the use of geospatial tools and applications. This open access document helps people to better understand what tools are currently available, and what kind of data can be gathered through these resources.

“In our guidance on using geo-tools, we have tried to match existing tools with evaluation questions and criteria. This is important, because the technological tools at our disposal will only work for us to the extent to which we know how to use them and adapt them to our needs. The key is to first define your questions, then to look for the best technological solution to assist you”, noted Mr Felloni.

EvalEdge, is the EES podcast that provides insights into the implications of new and emerging technologies in evaluation, and vice versa. The Podcast explores frontier technologies such as big data, machine learning, open data, geospatial analysis, blockchain and Internet of Things (IoTs). The purpose is to identify the role of evaluators and the field of evaluation in shaping how these technologies can be adapted in international development and in larger society. It also looks into a wider application of the traditional methods to evaluate innovations, and seeks to introduce novel methods having potential to be applied in the evaluation field.