Independent Magazine - Issue n.3, 2022

Independent

Learning. Transparency. Accountability.

MAGAZINE

T R I A D COM E S F U L L C I R LC E Spotlight on IFAD’s member states

NEW WEBSITE Access to evaluation transformed

Issue n.3

2022

ZERO HUNGER Untapped opportunities revealed

1

In o ur ow n wo rds

W

elcome to the third edition of Independent Magazine. In this edition, we complete the circle in terms of evaluation effectiveness. In our maiden edition, we spoke about the importance of policy, strategy and a competent cadre of evaluators to ensure that the highest standards of evaluation are adhered to, and to guarantee that evaluation adds value at multiple levels, including across government and management. In the second edition, we welcomed the perspectives of senior management vis-à-vis the evaluation function, of which they are among the primary users. This, as we continuously recognize the importance of having an open dialogue to enable evaluations to remain relevant and impactful.

In this edition, we focus on the most critical element of the effectiveness triad, which is governance. To do so, we are bringing forth the voices of representatives of IFAD’s Executive Board. IOE reports directly to the Board, which plays a critical role in the life of the Office. The Board protects the independent nature of the evaluation function, in addition to approving IOE’s budget and workplan. IOE recognizes the value addition that the Board makes. The Board also plays a critical role in funding IFAD, to which evaluations are very central.

We further elaborate, in this issue, on how independent evaluation works in IFAD, including by unpacking different evaluation products and presenting their relevance. Over the past year, we have focused on improving the understanding of the relationship between evaluation and development. Over twenty ‘Coffee Talks’ later, we have moved beyond the boundaries of traditional in-house seminars, bringing international speakers to present cutting-edge research.

As always, the people of IFAD remain the Fund’s most critical asset. Fully recognizing the importance of experience and knowledge, we are pleased to share the insights, thoughts and perspectives of people leaving the organization after a life-time of commitment to the Fund and of dedication to the people it serves around the world.

2

We also present the launch of our new website, which houses over six hundred evaluation reports. We are grateful for the support received from IFAD in the development of the site. This effort further entrenches the independence of our Office and of our work. For the first time in its over forty-year history, IOE independently manages all content production, publishing, display and presentation, in an effort to guarantee full communication transparency. The launch represents a major milestone in the life of IOE – one which we are very proud to share with the readers of Independent Magazine.

We trust you will find this edition inspiring.

Indran A. Naidoo, PhD.

Fabrizio Felloni

Director Independent Office of Evaluation of IFAD

Deputy Director Independent Office of Evaluation of IFAD

@Unsplash/nandhu kumar 3

INDEPENDENT OFFICE OF E IOE PILLARS

GUIDING PRINCIPLES

HISTORY

Accountability, learning, independence and partnership are the principles that guide our work

The IFAD’s evaluation function has evolved over more than 40 years of existence, from being discharged by a unit internal to the IFAD management into a fully-fledged independent evaluation outfit.

MISSION

QUALITY ASSURANCE

Our mission is to promote accountability and learning through independent, credible and useful evaluations of IFAD's work.

We have supported two external peer reviews of IFAD’s evaluation function conducted by independent professionals in the field of evaluation (2009 and 2019).

1978

02 1994 Monitoring and Evaluation Unit

IOE JOURNEY

03 Office of Evaluation and Studies

Monitoring and Evaluation Unit

2011 Independent Office of Evaluation

Office of Evaluation

04 1982

2003 05 IOE DIRECTORS

Luciano Lavizzari

Ashwani Muthoo

Kees Tuinenburg

Oscar A. Garcia

Director

Acting Director

Officer in Charge

Director

1999

NEW IOE VISION

2012

2013

2014

Fabrizio Felloni Acting Director

2020

FORGE CORPORATE CULTURE

BUILD EVALUATION DIALOGUES

Contribute to forging IFAD’s corporate culture as a transparent, learningoriented and accountable organization.

Engage with Management, Member States and external partners to support evaluation capacity and use within and outside IFAD.

Indran A. Naidoo Director

2021

IMPROVE COVERAGE

RETAIN & DEEPEN LEADERSHIP

Improve evaluation coverage and promote transformative evaluations that reflect the scale and scope of IFAD operations.

Retain and deepen IOE’s position as an internationally recognized leader in the evaluation of rural development programmes, policies and strategies.

EVALUATION OF IFAD (IOE) IOE EVALUATION PRODUCTS [click on icons to access database]

Aggregation level

Aggregation level

Sub-regional Evaluations

Evaluation Synthesis

Thematic Evaluations

Corporate Level Evaluations

Primary objective Assess strategy, common intervention approaches and IFAD organizational set up in a set of countries that share salient characteristics.

Primary objective Contribute to knowledge generation by consolidating findings from past evaluations.

Primary objective Provide evidence of development effectiveness, performance and results of operations in a thematic topic.

Primary objective Assess the organizational performance and institutional effectiveness of IFAD.

Main users Regional and country director(s), technical advisors, operational staff, and government counterparts.

Main users Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Main users Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Main users Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Annual Report of Independent Evaluation Primary objective Report all of IOE’s evaluation activities in a given year, and presents a synthesis of IFAD’s performance, lessons and challenges.

Main users Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Aggregation level

Aggregation level

Project Completion Report Validations

Project Performance Evaluation

Primary objective Validate the project completion reports prepared by IFAD Management.

EVALUATION POLICY

Main users IOE and IFAD Management for reporting (ARIE and RIDE) and feedback.

Primary objective Assess the performance and results of projectlevel operations funded

by IFAD. Main users Regional and country director(s), technical advisors, operational staff, and government counterparts.

Impact Evaluation

Project Cluster Evaluations

Primary objective Provide a rigorous quantitative assessment of the impact on rural poverty of selected IFAD’s operations.

Primary objective Assess the experience of several projects that have a common theme or common major component.

Main users Regional and country director(s), technical advisors, operational staff, and government counterparts.

Main users Regional and country director(s), technical advisors, operational staff, and government counterparts.

Country Strategy and Programme Evaluation Primary objective Assess performance and results of country strategy and operations and provide lessons and recommendations to guide preparation of next country strategy Main users Divisional and country director, country team, and government.

INDEPENDENCE

EVIDENCE-BASED MANAGEMENT

LEARNING & ACCOUNTABILITY

EVALUATION COMMITTEE

Guarantees the avoidance of conflicts of interest by upholding the provisions that safeguard the behavioral, organizational and structural independence of the Office.

Reflects IFAD’s increasing focus on embracing a culture of evidence-based management to maximize development effectiveness, in which evaluation has a critical role to play

Clarifies that accountability and learning are objectives of evaluation, and emphasizes the effective use of evaluation products to this end

Presents the roles and responsibilities of the Evaluation Committee and, of its Chair, in the evaluation function on behalf of the Executive Board

Editorial Board

E D I T O R

Writing Graphic design Publishing

[online profile]

C O P Y E D I T O R

Alexander Voccia, PhD

Laure Vidaud

[online profile]

Proofreading Revisioning Shaun Ryan

[online profile]

T E A M Nene Etim

[online profile]

Independent Magazine brings to the forefront of the global development dialogue the major efforts undertaken by the Independent Office of Evaluation of IFAD, while seeking to advance the organization’s vision of vibrant, inclusive and sustainable rural economies, where people live free from poverty and hunger. To present the richness of rural life, and detail facets of local community lifestyle, Independent Magazine also zooms in on cultural activities and landmark occurrences in countries featured by IOE’s evaluations.

@Unsplash/stephan louis 4

CO N T E N T S 08

Access to independent evaluation transformed at IFAD

30

Whole value chain approach found to be unique in Indonesia

12

IFAD support bolsters development of rural finance sector

32

Strategic facilitation services and livelihood activities

14

Evaluation reveals opportunity towards zero hunger

34

Evaluations face challenging times

15

Transformational change for people and the planet

35

Evaluations under COVID-19

16

Triad comes full circle

36

Ad maiora: Chitra, Prashanth and Eoghan

24

Transformation indispensable for climate action to take place

41

IFAD striving to increasingly define beneficiary target group

25

Business-as-usual approach to climate adaptation is over

42

A lifetime of service to the rural poor

26

Loan financing to horticulture and dairy in Uzbekistan

45

Evaluation capacity building is a collaborative effort

28

Evaluation as a driver of democracy in Africa

46

Communicating, independently

29

Evaluation capacities to advance sustainable development for all 5

ACCESS TO

AT

IFAD

TRANSFORMED

INDEPENDENT EVALUATION [Access website here]

I

n a move that bears few precedents among International Financial Institutions and United Nations entities, the Independent Office of Evaluation (IOE) and the Information and Communications Technology (ICT) Division of the International Fund for Agricultural Development (IFAD) jointly announce the official launch of a standalone website dedicated to IFAD’s independent evaluation function.

[Online profile]

For the first time in its over forty-year history, IOE independently manages all content production, publishing, display and presentation, in an effort to guarantee full communication transparency. The new website – launched twelve months after the arrival of Dr Naidoo as Director of IOE – ensures that the right evaluative products reach the right people, and that the most relevant messages are brought to the forefront through appropriate online resources and integrated channels. Opportunities for real, tangible change are fast emerging, as the website facilitates greater impact of IOE’s work.

“We are very excited about the launch of the first dedicated IOE website. This further entrenches the independence of the Office. I wish to make a special commendation to Thomas Bousios, Director of ICT, and the ICT team for working with us to design and launch a website within record-time. This helps to advance the commitment of IFAD to transparency, as all evaluative information will now be easily accessible, and in the public domain.”

- Indran A. Naidoo, PhD, Director, IOE

6

“We are very pleased to have worked with the IOE team and to see the release of their new interactive and easy to navigate website. The IOE team’s strong professionalism and pragmatic focus on the project scope, coupled with their leadership in prioritizing and taking decisions promptly, allowed for an excellent collaboration, and for the smooth and quick release of the new tool.

We are delighted for this fruitful experience in partnering with the IOE, and look forward to continuing our close collaboration.”

[Online profile]

- Thomas Bousios, Director, ICT

As the Fund’s official gateway to all independent evaluation-related knowledge and information, this one-stop-shop encompasses the multi-faceted range of IOE’s product mix.

The website’s articulate offerings present users with an immersive experience, spanning the full spectrum of the evaluation function. Everything is now just one click away.

7

Intuitive and easy to navigate, the website responds to the particular nature of IOE’s work, which calls for enhanced synergies between the Office and its stakeholders. To this end, the platform fosters greater user engagement by speaking to the core of what IOE’s audience looks for, bolstering direct and flexible avenues for interaction, and inviting exciting opportunities for people to create a personal connection with the Office in a dynamic fashion.

Innovation and tradition merge in the site’s visual branding, which draws inspiration from the most innovative and graphically appealing visual solutions adopted by award-winning designers, while maintaining continuity with IFAD’s corporate identity.

In addition to ensuring immediate access to the complete repository of IOE’s reports, searchable through the new dashboard interface, the website features a suite of tailor-made digital solutions, customized to maximise the end-user experience.

These include unprecedented access to over six hundred evaluation reports, staff professional profiles, work programmes and budgets, quality assurance processes, evaluation policies, manuals, on-line trainings, the multi-year strategy, and ratings database, in addition to a wide selection of books, case studies, briefs and learning notes.

A rich media centre rounds-off this portfolio, with a fast-paced turnaround of stories, events, blogs, videos, podcasts, newsletters and magazines, complete with full social media integration.

“Great website. Beautiful and easy to navigate with rich contents.”

Osvaldo Néstor Feinstein, Professor, Universidad Complutense de Madrid.

“This helps to advance the commitment of IFAD to transparency.”

Carlos Alberto Amaral, Minister, Counsellor, Deputy Permanent Representative of Angola to IFAD.

“Congratulations, it is terrific.

Easy to navigate and very clear.”

Pearl Eliadis, Associate Professor, McGill University.

“Congratulations for this initiative and for making IFAD the first agency to get there!”

Paolo Silveri, Country Director, IFAD.

“The new IOE website features a brand-new interface.

Existing content has been reclassified for easier discovery and many new sections have been created to host additional independent evaluation knowledge and information. It has truly been a very positive and smooth project journey, thanks to the productive collaboration with the IOE team and their strong dedication to the project output.”

- Federica Verri, Digital Workspace and Collaboration Tools Coordinator, ICT 8

“It is a pioneering effort, which has broken the mould in record-setting time. ‘Clean, crisp, fresh and modern’ is what people are looking for – and that’s exactly what we are striving to give them. For us, cookie cutter solutions are a thing of the past. The web-based resources, now at the disposal of our stakeholders, open wide countless pathways to genuine connection, interaction and engagement through an immersive dynamic.

This is what really excites me the most.”

[Online profile]

- Alexander Voccia, PhD Coordinator, Evaluation Communication Unit (project lead), IOE

9

IFAD support bolsters development of inexistent rural finance sector in Eswatini

“I

FAD-supported strategies and programmes have contributed to setting the building stones for the national rural finance system, and enabling small-holders access to financial products suited to their needs”, stated Indran A. Naidoo, Director of the Independent Office of Evaluation (IOE) of IFAD, in introducing the findings of the Eswatini Country Strategy and Programme Evaluation (CSPE), during the virtual national workshop held on 28 January 2022.

According to IOE’s latest evaluation report, which covered the period 2000 – 2021, the partnership between IFAD and the Government of Eswatini over twenty years has been constructive and fruitful and has produced tangible positive results and impacts. IFAD-supported programmes addressed very diverse development challenges and engaged with a variety of intervention models.

Hon. Jabulani Mabuza MInister of Agriculture “During the period of implementation of IFAD interventions, the country has achieved a ten percent reduction in the prevalence of poverty. We be10

from Ministries of Agriculture, Finance, UN agencies, private sector, research organizations, including CGIAR, farmer organizations, IFAD supported projects’ implementers and IFAD staff attended the workshop.

Kouessi Maximin Kodjo,, PhD lead author, Eswatini CSPE [online profile]

lieve that this is attributed to a large extent to the long-term consciousness ignited to inform policy and activate self-help activities of the direct beneficiaries of the programmes”, highlighted the Honourable Jabulani Mabuza, Minister of Agriculture of the Kingdom of Eswatini, during his opening remarks.

The CSPE started under the guidance of Mònica Lomeña-Gelis, IOE Senior Evaluation Officer, and transitioned to the leadership of Kouessi Maximin Kodjo, IOE Lead Evaluation Officer.

Co-organized by the Government of the Kingdom of Eswatini, and IFAD’s IOE, in collaboration with IFAD’s East and Southern Africa Division (ESA), the on-line virtual workshop brought together a wealth of high-level attendants, including at the Ministerial level. Approximately 70 participants

During the workshop, participants noted that IFAD’s strategies for Eswatini have contributed to major achievements such as the development of an inclusive rural finance sector in the country, and the participation of smallholder farmers into a variety of value chains and the promotion of participatory community development.

Dr Kodjo presented successful undertakings. Examples include establishing and developing the national rural finance sector, the establishment of smallholder Farmer Companies concepts and approaches, the introduction of the Chiefdom Development Plans and community development committees, which have enabled the participation of rural communities and smallholder producers to decision-making process on issues of high relevance to their livelihoods. IFAD was also innovative in bringing to Eswatini new crop varieties, for instance the Orange-Fleshed Sweet Potato, and in piloting of on-farm feed production associated to mini beef-lots, to be mixed with other residues to produce lowcost animal feed and fatten animals for the market.

Country Strategy and Programme Evaluations

These achievements of the IFAD-Eswatini partnership did not benefit to all target groups equally. Most vulnerable households may not be reached by interventions that are market focused, considering that participants are rural smallholder producers who have some assets, such as access to land and workforce available in the household. However, nowadays, those smallholder farmers are facing risks of increasingly diminishing returns from their main productive activity, which might jeopardize all positive achievements so far. Moreover, important threats are emerging with regards to the economic, institutional and technical sustainability of the sugar cane smallholders’ production scheme, which also poses risks to the sustainability of major investments in the industrial, export-oriented sugar value chain.

“While Eswatini has recently achieved strong progress in reducing poverty, there are still several challenges faced by rural people in the country, including: increasing vulnerability to climate change, especially drought; low agricultural productivity and diversity; limited access to financial services; and high post-harvest losses”, underscored Mr. Donal Brown, Associate Vice-President, Programme Management Department of IFAD, summarizing some of the main challenges encountered by IOE’s evaluation report.

Going forward, IOE recommends that IFAD should address the fundamental constraints that prevent rural smallholder producers, women and youth, from achieving more sustaina-

ble livelihoods, and should collaborate with the government to develop a programme aimed at tackling the challenges faced by the producers of irrigated sugar cane and other crops.

Ewatini promotes indigenous arts and music*

Hon. Neal Rijkenberg Minister of Finance “We of course need the support of our stakeholders to achieve the desired improvements in these areas, and I count on the commitment of our friends here today to support our efforts to implement more efficient, effective and sustainable projects in Eswatini”, remarked Ms Sara Mbago-Bhunu, Regional Director, ESA, in this regard.

Closing the meeting, the Honourable Neal Rijkenberg, Minister of Finance of the Kingdom of Eswatini, praised IFAD’s focus and its modus operandi in tackling rural development and rural transformation. He highlighted that the Fund should be building on successful and proven models, whilst recognizing that there is a need to put in place a clear and robust rollout plan to ensure seamless implementation.

Since 1983, IFAD has supported six projects and programmes in the country for a total amount of US$ 54.4 million, benefiting 41,555 poor rural households.

Rural poverty was estimated at 70.2 per cent in 2016, largely associated with agriculture.

Access full database [here] @ ioe.ifad.org

Two indigenous musicians, Gogo Tfobhi Shongwe and Vusi Sibandze, were invited to perform at the North West University (NWU) of South Africa on 9 March 2022, during a show dubbed ‘Sounds from Eswatini’. The two were joined by pianist Cara Stacey, who is the University’s senior lecturer in African Music.

The artists showcased the beauty of Eswatini indigenous music. Gogo Tfobhi Shongwe played sitolotolo (mouth harp) and makhoyane (musical bow player), while Vusi Sibandze is a guitarist and vocalist.

Vusi is also a music teacher at Sitsembiso Sebunye High School.

The concert forms part of the North-West University Annual Concert Series. “The trio brings forward new arrangements of traditional repertoire and newly composed Swazi music from Mbabane, Mshingimshingini and eMphini,” reads the letter of invite.

Sibandze said they were excited about the invitation, stating that for them to be recognised by such a reputable institution meant their contribution in the preservation of indigenous music was an honour. *source: http://new.observer.org.sz/

11

FIRST OF A KIND EVALUATION REVEALS OPPORTUNITY TO BOLSTER PROGRESS TOWARDS ZERO HUNGER Joint Evaluation on the Collaboration among the United Nations Rome-based Agencies [[[ Full report on-line here

T

here is real potential for the three United Nations Rome-Based Agencies (RBAs) to enhance progress towards eliminating hunger world-wide. The key to unlock this untapped potential is enhanced inter-agency collaboration – which responds to a genuine commitment among RBA staff. This, according to the first ever joint evaluation of the collaboration among IFAD, WFP and FAO.

The evaluation, which bears no precedent, systematically assesses the potential value added of RBA collaboration. To this end, the report provides credible evidence of the extent and quality of joint RBA performance towards shared objectives, and an explanation of the reasons for that performance. The evaluation focuses on all forms of collaboration at country, regional and global levels, and draws on the views expressed by over four hundred informants during interviews and meetings.

Collaboration between the RBAs is a daily reality, reflecting the shared strengths and commitment of these distinctly different organizations. RBA staff routinely act on the advantages of this collabo12

Corporate Level Evaluation

ration, but not to the extent necessary to unlock its full potential impact on SDG 2. Moreover, at present, efforts to promote RBA collaboration are not grounded in an accurate understanding of the conditions in which it is most effectively pursued.

Despite the daily collaboration reality, there is widespread ambivalence about the concept.

Complex layers of doubt and reluctance lie steeped beneath the strong official commitments – a situation worsened by the formal systems and procedures to promote, co-ordinate and report on collaboration, which add little value, are often not followed through usefully, and frustrate staff more often than they inspire them. Notably, the RBAs have made limited progress in reducing overlap, competition and duplication of work.

The latent opportunities inherent in these findings are highlighted by the strongly collaborative spirit that has developed in those countries where there is a clear advantage in working together. Good technical collaboration on certain themes and tasks suggests that coordination is generally easier around thematic and advocacy work than in formal operational project settings. These efforts have resulted in enhanced sharing of knowledge, lessons and good practice at all levels. While the depth, quality and practical value of this sharing vary, the evaluation notes significant technical interdependence between the RBAs in many fields.

To exploit these latent opportunities, and truly bolster progress towards attainment of SDG 2, the evaluation report recommends further embracing the new joint programming mechanisms at the country level and ensuring constructive, collaborative RBA engagement with these mechanisms.

Furthermore, the report also recommends restructuring and reinforcing the coordination architecture for RBA collaboration within the framework of the UN Development System reform, whilst assessing the costs and benefits of proposed collaboration, and proceeding only if the benefits outweigh the costs.

Access database [here] @ ioe.ifad.org

FURTHER READING

Transformational Change for People and the Planet

T

he book Transformational Change for People and the Planet is now available for purchase and free download online. Published by Springer, the book intends to provide an authoritative, interdisciplinary perspective on innovative and emerging evaluation knowledge and practice related to environment, natural resources management, climate change, and development.

The book has its roots in the 3rd International Conference on Evaluating Environment and Development, held in Prague, in October 2019, organized by the Global Environment Facility Independent Evaluation Office, jointly with the International Development Evaluation Association and the Earth-Eval community of practice.

Download open source book, here.

Access chapter by Dr I. Naidoo, here Access chapter by Dr S. Nanthikesan et al., here Access chapter by P. Kotturi, here Purchase hard copy, here.

The list of authors includes Indran Naidoo, Nanthikesan and Prashanth Kotturi, Director, Lead Evaluation Officer and Evaluation Officer at IOE, respectively. In the book’s opening chapter, Indran draws on national and international case studies, examining the concept of transformation from a contextual perspective and noting the relativism in the concept. In the chapter he co-authored with Laura Silici, Jerry Knox and Andy Rowe, Nanthikesan outlines how lessons from CCA interventions can be transferred via three learning domains that are essential for transformational change: scaling-up (in its multiple forms), knowledge management, and the human-environment nexus. In his chapter, Prashanth reflects on how IOE has addressed the growing demands on the evaluation function in terms of incorporating concerns on environment and climate within existing methodological frameworks, and also adapting its methodology to meet internal and external evaluation demands.

13

FFEATURE E AT URE STORY 14

TRIAD CO

Spotlight

OMES FULL CIRCLE

t on IFAD’s member states 15

M

ember states comprising IFAD’s governance play a pivotal role in bolstering the utility of evaluations, which ensure that the organization performs optimally, and in safeguarding the full intellectual, methodological and communicative independence of IOE. In this regard, while the first edition of Independent Magazine presented IOE’s new strategic vision and the second captured the perspectives of senior management vis-à-vis the role of evaluation in embedding reflective milestones in the IFAD development journey, this third edition completes the triad by discussing the governance stream.

IFAD’s Executive Board and Evaluation Committee play a pivotal oversight role. Review of independent evaluation reports, follow-up with IFAD senior management vis-à-vis implementation of report recommendations, approval of IOE workplan and budget, and appointment of the IOE Director – these are but some of the direct responsibilities and sole prerogatives of IFAD’s member states, which help entrench the learning, transparency and accountability themes threaded in the first two editions of Independent Magazine.

Undoubtedly, engagement between IOE and IFAD member states is essential. The Director and staff of IOE have the authority to communicate and interact directly with members of the Executive Board, with in-country partners, and with others outside the Fund, as appropriate, for the undertaking of evaluations. Building on this, IOE’s new vision and strategic direction re-enhances the Office’s drive to engage with all member state groups in an ongoing and systematic fashion.

To ensure that we present an honest, balanced and inclusive picture on this important topic, Independent Magazine has drawn insights from esteemed representatives of all member state groups of the Executive Board.

We are most grateful to Dr Yaya Adisa Olaitan Olaniran, Permanent Representative of Nigeria to Rome-based UN agencies; Ms Michelle Winthrop, Development Policy Director, Department of Foreign Affairs and Trade, Ireland; Dr Bommakanti Rajender, Minsiter (Agriculture) and Alternate Permanent Representative of India to Rome-based UN agencies; and Ms Sandra Paola Ramirez Valenzuela, First Secretary, Mulitlateral Affairs, Rome-based UN agencies, Mexico, for generously taking the time to sit down with us, and for kindly sharing their experiences, insights and perspectives.

Excellencies, ladies and gentlemen, good morning.

Good morning, Alexander.

Could you briefly describe your institutional role as it relates to oversight and evaluation?

Ms Michelle Winthrop In terms of oversight and evaluation, it is my responsibility to ensure that IFAD delivers on Ireland’s stated objective of ‘reaching the furthest behind first’. This is the principle that guides all of Ireland’s oversees development assistance. IFAD has a good track record in this regard, and we see the evaluation function as being a really important element in ensuring that IFAD continues to do so. In this regard, IOE is uniquely positioned to ensure that IFAD is equipped with the necessary information and the evidence, whilst also to provide the first early warning flags that signal if IFAD institutionally starts to stray away. That is why Ireland put itself forward to be on the Evaluation Committee (EC).

16

Dr Yaya Adisa Olaitan Olaniran The oversight function that I am involved in is in the context of the Evaluation Committee. As members of the Committee, we review IOE’s proposed work plan, budget, various products, such as the IOE multi-year strategy, IFAD evaluation policy and evaluation manual. Upon review, we make suggestions, we discuss with IOE, and eventually recommend the documents to the Executive Board (EB) for approval. Furthermore, as part of the Evaluation Committee, it is part of our responsibility to ensure the independency of IOE. We ensure that IOE can look at what it wants to look at, without interference by management. This includes ensuring that management implements evaluation recommendations as best it can. Where we feel that management is not proactive enough, we urge them to act. In the event of complex situations, we invite IOE and management to sit down and iron out the problems.

Ms Sandra Paola Ramirez I am the delegate in charge of IFAD affairs at the Permanent Mission of Mexico. As such, I am in charge of preparing Mexico’s participation at IFAD’s Evaluation Committee’s and Executive Board’s meetings. On the one hand, I study and analyse the evaluations produced by the Independent Office of Evaluation (IOE). I assess their pertinence and the way its results could help IFAD to be more efficient and effective. On the other hand, I also have to monitor that new policies and decisions always respect the independency of the IOE and that IFAD management utilizes IOE deliverables to enhance its work.

Dr Bommakanti Rajender I have been in Rome for the last three years, including as a member of the IFAD Evaluation Committee. We have been members of the Executive Board since its inception, and India’s Governor is a member of the Governing Council. These are our institutional roles.

Could you kindly explain the processes and dynamics of the evaluation function within the structure of the IFAD governance architecture?

Ms Michelle Winthrop At its most basic level, the evaluation function provides the raw material for strategic decision-making by the IFAD Executive Board (EB). As EB members, we need to use this information to encourage management when it makes the right decisions, and equally to challenge management when we see decisions that are contrary to what the evidence is telling us. The role of the EC is to be an intermediary of that evidence, of that raw material.

The EC makes sure that Board members see what they need to, filtering the evidence and steering the work of IOE based on the EB’s needs. It’s a supply and demand relationship, with the commodity being evidence. EC members are the traders making transactions in both directions, helping to articulate both supply and demand.

I think, in a more general sense, we also set the tone for how IFAD management responds to IOE evaluations, and how IOE is empowered to challenge management. This is important. We need to help ensure that this relationship is a constructive one. It’s not always a friendly one – and that’s OK –, but it needs to be a constructive working relationship. We have a responsibility to make sure that this is the case. If we see management not responding appropriately, we need to call them out. Likewise, if we see IOE being a little unfair or subjective – which, by the way, never happens –, it’s our job to call that out. I am very struck by the fact that IOE has become a victim of its own success, during the last year. By generating high quality evaluations, and by ensuring good 17

processes, we are seeing a big surge in demand. Everyone wants a nice IOE evaluation done on their area of work, country or region. This means that at every turn, IOE management have a set of challenging choices to make. The EB and EC need to help equip IOE to make the right choices about which evaluations to prioritize, and how to maximise efficiencies across multiple evaluations.

Dr Yaya Adisa Olaitan Olaniran The IFAD governance architecture is set-up to ensure that the Fund is able to help the poorest of the poor. The programme management department designs programmes to this end. These programmes are discussed with national governments, including on the basis of an analysis of past projects. This is where IOE comes into play.

IOE’s analysis dovetails the design of country strategy programmes, including by capturing the perspectives of government officials on-the-ground vis-à-vis the issues that they see affecting the implementation of past programmes. For this reason, IOE’s evaluations are very, very important in order to get the right results. Equally important are IOE’s corporate level evaluations. For example, issues like organizational decentralization, and the effectiveness of this process vis-à-vis staffing levels, government dialogue and outputs are issues that IOE is and will need to look at closely.

Ms Sandra Paola Ramirez IOE is the one in charge of promoting accountability and learning through independent, credible and useful evaluations of IFAD’s work. In this regard, its evaluations should contribute to promote accountability and learning to produce more efficient operations to reduce rural poverty. The Evaluation Committee is a sub-committee of the Executive Board which performs reviews of selected evaluation issues, seeking to enhance the effectiveness of the Executive Board.

Dr Bommakanti Rajender As per IFAD’s mandate, the main function of evaluation is to look at whether the organization is meeting its goals and objectives. The EB and EC play a pivotal oversight role in this regard. The EB discusses proposed country strategies and programmes, and does so also in light of past analysis carried out by IOE. In terms of the dynamics, there is a distinction between programme and donor countries. Donor countries contribute their tax payers’ money to the Fund, and thus want to understand the results and impacts that their pledges have made with reference to the targets set by IFAD. Evaluation obviously plays a big role in this context. Regarding the schemes, there are institutional mechanisms to ensure that members of the EB have the opportunity to witness first hand select programmes on the ground. These missions also relate to the work carried out by IOE. Do you believe that the evaluation function helps to ensure that IFAD performs optimally as an organization? If so, why and to what extent?

18

Ms Michelle Winthrop Absolutely yes, it does. The management response is the measure of whether IOE evaluations have been applied. Often, the quality of the management response is determined by the relevance of the IOE evaluation.

Those two metrics, relevance and application, are really important. I think we have seen a real surge in the quality and relevance of IOE evaluations. There are two examples that I would point to, which are indicative of the importance of this relationship. Nutrition is one of the mainstreaming themes that Ireland believes in very strongly. At this stage, IOE has a very important role in helping to generate the evidence to help us see if IFAD management is being sincere in its commitment to embed nutrition across its work. The other piece is IFAD’s work in fragile and conflict-affected states. We are seeing some really interesting evidence emerging from IOE about how effective IFAD actually is in fragile states, and some of the real financial costs of working in these contexts. This is essential information for us. We would be lost without the evaluation function. In the absence of other forms of triangulation, which we often cannot find in these complex environments, we are completely reliant on the IOE evidence to guide our discussions about what IFAD should be doing, where and with what resources.

Dr Yaya Adisa Olaitan Olaniran Evaluation is indicative of IFAD’s ability to function. However, many things are necessary for IFAD to function optimally, things which are beyond the control of IOE. What IOE can do, is to point out what these things are, find reasons for shortcomings vis-à-vis envisaged results, and recommend actions for improvement. One aspect of IOE’s work that I personally appreciate is its attention to understanding why projects may be delayed. This is usually due to lengthy internal government processes. In more recent times, through dialogue with management, IOE has been able to capture issues also during the programme design stage. Results may not be perfect, but collaboration between and IOE and management has certainly improved programme design and results.

Moreover, it is the learning loop that marks the difference between a successful project and one that fails. I believe that the interaction among all IFAD departments and IOE are central to IFAD’s added value compared to other IFIs. Ms Sandra Paola Ramirez Yes, it does contribute to perform in an effective and efficient way. To what extend depends on how much recommendations are taken into account and implemented.

Dr Bommakanti Rajender Absolutely, there is no denying that evaluation is the sine qua non of any organization. Evaluation is like a doctor, who is there to diagnose whether your body is functioning properly or not, and to give remedial by prescribing medicines. We may not like what the doctor tells us, but we have to face the reality if we want to continue living. Likewise, an organization without evaluation will become ‘sick’. There will be a lack of direction; it will be a ship that is going without a radar. Evaluation thus plays a fundamental role in correcting problems, where necessary. For this reason, evaluation gives member states confidence in the work that IFAD is carrying out. Evaluation also shows us where the successes are, and help us identify those initiatives that can and should be scaled-up and replicated in other countries. IOE is doing a great job in this regard, I am convinced – this is my experience over the past three years.

19

Do you believe that member states comprising IFAD’s governance structure play a pivotal role in bolstering the utility of evaluations? If so, what are the key elements that help ensure that this is the case?

Ms Michelle Winthrop We, the member states, should always endeavor to have the views that we express at the EB supported by evidence. Using that evidence is probably the most useful thing that we can do. It is good governance, and it is also a means to reinforce the importance of the evaluation function institutionally, as both an accountability mechanism and as a learning tool. As EC members, we also have a role in championing evaluation, encouraging discussions of them and accessibility and communications. In this regard, I think it’s important to remember that EB members also have an onus to not politicize the results of evaluations, where evaluations can be slightly critical of government performance or commitment. I know that IOE works really hard on objectivity and independence. Personally, I have a lot of faith in the independence of the evaluation function. To be honest with ourselves, we should be able – as an Executive Board – to take bad news on the chin and be mature about it, and process those lessons for the broader benefit of IFAD. Dr Yaya Adisa Olaitan Olaniran I find the IFAD Board unique. All representatives come from different backgrounds, which is something that helps us to forge a consensus on the best way forward. However difficult a problem may appear, with consultation and discussion we always get things done and come to an amicable agreement. In this context, member states do their best to ensure that recommendations given by IOE are always followed. Moreover, we must keep in mind that one thing is getting results, something very different is getting the right impacts. We may get results, but if these are not making a positive impact on the lives of people, then these are not the results that we should be looking for. For this reason, it is so important to ensure that members of the EB join IFAD management and IOE in going to the field to see where money is being spent, and to interact with the people that are doing the work in the field, to understand what their thinking is.

Ms Sandra Paola Ramirez Yes, the IOE responds to member states requests. So, if member states identify the need of assessing a particular policy, aspect or performance, IOE would work on a deliverable that satisfies such a request. The key elements to consider in such a case are the identification of evaluation needs and the request of member states to the IOE. Dr Bommakanti Rajender Engagement between IOE and IFAD member states is essential. This has improved over the years, and there is scope for further growth. We must recognize that there can be significant political changes in countries, every two-to-four years. This may result in turnaround of government staff deployed internationally, which in turn has consequences for institutional memory.

Understanding the organizational dynamics of IFAD, and the role that evaluation plays within it, takes time. For this reason, there should be institutional mechanisms to help newly appointed staff to come to grips with the evaluation function. Visual materials and briefing packages are what is needed. Traditional approaches are not sufficient. In order to successfully compete for the attention of member states, the information provided will need to be useful, visually attractive and easy to digest.

Looking ahead, what do you see as being the main opportunities for evaluation to further permeate an evidence-based learning culture across IFAD, and for this culture to trigger life-changing impacts for the organization’s stakeholders in the field?

@p

20

pixabay/FelixMittermeier

Ms Michelle Winthrop The accessibility of findings is really important. Getting the balance right between presenting the rigour of the process and demonstrating all the amazing data gathered, on the one hand, and distilling this down to key lessons that can help guide decisions, on the other, is a really important part of the evaluation function. I know that Indran has invested a huge effort in comms, including the launch of the new website, to ensure that evaluations are as easily accessible as possible.

This is the most important investment, really. The decentralization push within IFAD is something that we are supportive of, and we need to find ways to make sure that lessons are embedded in operations in the field, and communicated effectively to IFAD staff in the field. Outreach and dissemination must be done in a way that reflects the needs of outposted staff, to foster a reflective culture on evaluation findings. In addition, IFAD has a very important role to play in helping to build a foundation of good data around rural and agricultural development, particularly in Africa. I would like to see – with the limited resources IOE has – more work to foster community practice, so that it’s not just the people sitting in IFAD offices who are reflecting on the lessons, but also the broader set of stakeholders, many of whom IFAD will be working with ten years down the line on different projects. Lastly, the recent evaluation of government performance was a really good example of how a large, highly complex evaluation can be distilled into some really compelling lessons. Despite our respective experience and intellectual capabilities, we are all being bombarded on a regular basis with lots of information. For this reason, I like to say that it is possible to be both rigorous and compelling – that is how you embed institutional lessons and change.

This work that IOE is doing around good communications, synthesising in different ways, coming up with snappy solutions to present findings, it is all extremely positive as far as I’m concerned.

Dr Yaya Adisa Olaitan Olaniran There is no crystal ball, but I believe a lot has been done in the last 5-6 years in IFAD that has given us a new understanding of the problems we are grappling with. The governance structure of countries affects what IFAD is able to do. It is easy for governments to say “let’s look at what results we can get over the next four-to-five years working with IFAD”. However, the truth is that the impact of IFAD interventions cannot be really felt in four years. So, governments must not see programmes through the lens of their office’s tenure, but rather give all the support necessary to achieve long-term impacts. Furthermore, with the ODA shrinking, it is important for IFAD to show countries the value-for-money of its intervention, and to do so by demonstrating the possibility of scaling-up the Fund’s interventions in a sustainable manner. These are the areas where IFAD needs to focus its efforts, and where IOE needs to increasingly focus its attention.

Ms Sandra Paola Ramirez The evaluation of the implementation and impact of recently adopted policies is a key opportunity. Also, the decentralization process in all its aspects would require constant evaluation in order to monitor its implementation and impact on the ground.

Dr Bommakanti Rajender In my experience, seeing is believing – especially when we are talking about the use of tax payers’ money, and the impact of these resources on people’s lives and livelihoods. This needs to be evaluated properly. Field visits are very important in this regard, not only for evaluation specialists, but also for member state representatives. This is a trend that should increase. The work carried out by IOE should continue to support these field visits. We need to see for ourselves where the bottlenecks are in order to fully appreciate how these relate to evaluation recommendations.

In turn, these recommendations must increasingly be backed by a sound, robust and scientific methodology, which must be grounded in the evidence-based and context-specific nature of evaluation work. One-size-fits-all solutions are not credible. While generic recommendations may be easier to ‘digest’, and people may not always appreciate very specific recommendations, it is this level specificity that will increasingly give value and credibility to the evaluation function.

21

PATHWAYS FOR TRANSFORMATION CHANGE INDISPENSABLE FOR CLIMATE ACTION TO TAKE PLACE mational change for climate action post COP26 – how?’.

The participant line-up also featured Rob D. van den Berg, member of IOE’s Evaluation Advisory Panel, and Jo Puri, Associate Vice President of IFAD’s Strategy and Knowledge Department.

Indran A. Naidoo, PhD [online profile]

M

ajor transformational changes will be necessary in our economies and societies to shift the world from the brink of a climate catastrophe. These changes will need to be systemic, comprehensive and implemented in record time. They will also have and include perspectives such as climate justice, just transition, and the full range of Sustainable Development Goals. High-level representatives from the worlds of policy action, research and evaluation underscored this call for action during the second climate change-focused dialogue in the three-part series organized by Wilton Park, which took place on 1 February 2022.

Indran A. Naidoo and Fabrizio Felloni, respectively Director and Deputy Director of the Independent Office of Evaluation (IOE) of IFAD, joined over fifty participants to discuss the theme ‘making transfor-

22

Convened two months after COP 26, the dialogue session looked at some of the evidence and research that is building-up about how transformational climate action can take place. By discussing the existing evidence, participants identified ways to break down the many practical, political and structural barriers that may lead to improved collaboration and aligned networks among policy, research and evaluation.

Issues discussed and questions addressed included what transformational climate action needs to look like after COP26, both in terms of the outcomes of COP26 and the wider ambition to keep global warming to 1.5 degrees; what progress is really being made to generate the transformational change needed; how to create change across the many often siloed sectoral approaches, pilots and initiatives; and how collective action can take place for adaptive, anticipatory and robust climate action.

Fabrizio Felloni [online profile] Wilton Park is an Executive Agency of the UK’s Foreign, Commonwealth and Development Office and is considered one of the world’s leading institutions for in-depth discussion of international policy issues and challenges. To hold this event, Wilton Park partnered with IFAD, The International Development Evaluation Association (IDEAS), Climate Investment Funds, Global Environment Facility, Green Climate Fund, University of Sussex, Institute of Development Studies, United Nations Development Programme Independent Evaluation Office, and the International Evaluation Academy.

The first discussion in the Wilton Park series took place in September 2021, under the theme ‘Pathways towards transformational climate action: connecting the worlds of research, evaluation and policy for action..’

ERA OF BUSINESS-AS-USUAL APPROACH TO CLIMATE ADAPTATION IS OVER

Nanthikesan, PhD [online profile]

T

ransformative climate change adaptation (CCA) solutions are needed to achieve related Sustainable Development Goal (SDG) targets by 2030, and to avoid catastrophic consequences.

Evaluations can play a pivotal role in this regard, as they are critical for building-up the evidence-based knowledge base of CCA solutions. Suppiramaniam Nanthikesan, Lead Evaluation Officer, Independent Office of Evaluation of IFAD (IOE), delivered this message during his presentation at the 3rd International Conference of Asia Pacific Evaluation Association (APEA) and EVALFEST. “We are 8 years from 2030.

Moreover, the IPCC 2021 report points to an urgent need to act. Therefore, ‘good is no longer good enough’. We need transformative solutions”, affirmed Dr Nanthikesan.

Organized New Delhi, India,

from 21 – 25 February 2022, this year’s APEA Conference & EvalFest captured broad spectrum stakeholder participation. The overarching theme underpinning the events was ‘Towards a Vibrant Evaluation Ecosystem: Creativity, Collaboration and Convergence’.

The topic reflected the need for the evaluation ecosystem to be energetic and dynamic in order to respond to the emerging and evolving evaluation arena, with pro-active thinking.

Dr S. Nanthikesan delivered his presentation on 23 February, during the session ‘Adaptation and Resilience to Climate Change: Lessons from Evaluations’. Moderated by Geeta Batra, Chief Evaluation Officer and Deputy Director, GEF Independent Evaluation Office (GEF IEO), the session also saw presentations and interventions by Debbie Menezes, Chair, Adaptation Fund Technical Evaluation Reference Group, and Anna Viggh and Gabriel Sidman, Senior Evaluation Officer and Evaluation Officer, respectively, at GEF IEO. Moving away from anthropocentric approaches to agriculture was at the centre of the arguments that Dr Nanthikesan advanced. Agriculture is critical to human life. It is severely impacted by climate change, while also contributing to cli-

mate change when climate adaptation solutions do not ensure environmental resilience. In this highly complex context, many governments face significant challenges to incentivize sustainable climate adaptation responses.

Ensuring an evidence base for for holistic CCA solutions that build climate, environmental and developmental resilience together is critical for ramping up climate finance and environmentally sustainable agricultural policies and practices. Evaluations could and should be looked upon as having the potential to play a critical role in this regard.

APEA is a regional evaluation network that aims to improve the theory, practice, and use of evaluation across the Asia-Pacific region. The Evaluation Community of India (ECOI) is a voluntary organization of professional evaluators, established in 2015 with objectives of promoting evaluation culture in the country and contributing to evaluation capacity building. APEA has organized two regional evaluation conferences in 2016 and 2019 and ECOI has held two knowledge-sharing events in 2018 and 2020. The two bodies have now come together to organize their first joint ‘evaluation knowledge-sharing’ event.

@pixabay/Annie Spratt

23

IFAD is first IFI to provide loan financing to horticulture and dairy in Uzbekistan

Simona Somma

lead author, Uzbekistan CSPE [online profile]

While IFIs support larger scale producers in Uzbekistan, IFAD has focused its efforts on promoting the dehkan farmers, who are a clear niche for the Fund. The Government and other financers have noted IFAD’s undertakings vis-à-vis horticulture and dairy production, which are sectors that are likely to be financially and economically sustainable, despite the negative impact of COVID-19 on markets.

I

FAD has been the first international financial institution (IFI) to provide direct support to the most vulnerable group in Uzbekistan, the dehkan farmers, by pioneering loan financing to horticulture and dairy. Simona Somma, Evaluation Officer, Independent Office of Evaluation of IFAD (IOE), highlighted this important finding of IOE’s Country Strategy and Programme Evaluation in Uzbekistan during an on-line learning workshop, on 24 February 2022.

“We highly recognize the results and findings of IOE’s review of IFAD’s strategy 2011-2020. It clearly generates new ideas and mechanisms for the future, and enhances partnership between Uzbekistan and IFAD”, affirmed his Excellency Otabek Khamidullaevich Akbarov, Ambassador and Permanent Representative of the Republic of Uzbekistan to IFAD, in his opening statement.

24

H.E. Otabek K. Akbarov Ambassador “IFAD is one of our most significant, flexible and responsive development partners. We are grateful to IFAD for supporting Uzbekistan’s agriculture development through the implementation of important projects aimed at facilitating agrofood production value chains”, stated the Honourable Vafaev Shukhrat Abdusharifovich, Deputy Minister of Investments and Foreign Trade.

Co-organized by the Government of Uzbekistan and IOE, in collaboration with IFAD’s Near East, North Africa and Europe

Hon. Vafaev S. Abdusharifovich

Deputy Minister of Investments and Foreign Trade

Division (NEN), the on-line virtual workshop brought together a wealth of high-level attendants, including the Hon. Vafaev Shukhrat Abdusharifovich, the Hon. Yuldashev Kakhramon Anvardjanovich, Deputy Minister of Agriculture, and his Excellency Otabek Khamidullaevich Akbarov. Over ninety participants joined the event, representing the government of Uzbekistan, development partners, civil society organizations, private sector partners, and IFAD senior management and staff.

The main objectives of the CSPE were to assess the results and performance of the IFAD strategy in the period 2011-2020, which focused support to the dehkan farmers. The findings, lessons and recommendations from this CSPE will inform the preparation of IFAD’s Country Strategic Opportunities Programme (COSOP) in 2022.

During the period under review, IFAD’s support in promoting rural development was aligned with the country needs and

Country Strategy and Programme Evaluations

priorities and, according to the government, will continue to be relevant for Uzbekistan given the persistent disparities in living standards between urban and rural areas and the effects of the global pandemic, which is reducing growth and creating additional financing needs.

In this regard, the Government of Uzbekistan is paying increasing attention to the poorest and to technical innovations, partnership building and policy dialogue – which opens important opportunities for IFAD’s next COSOP. “This evaluation gives us an opportunity to take stock of the IFAD COSOP, draw lessons from the experience, and adjust to the weaknesses and build on the successes in the rapidly evolving context of Uzbekistan”, noted Ms Dina Saleh, Regional Director, IFAD NEN. Participants acknowledged that it is not possible to quantify the rural poverty impact of IFAD’s strategy and programme in Uzbekistan, largely due to the fact that available data is not sufficiently robust. In this regard, it is difficult to draw clear links to policy development and attribute the growing interest of the Government on dekhan farmers to IFAD. At present, it is not possible to know whether poorer dehkans are accessing finance or participating in project activities, as poverty data on this group are not available. In practice, the large size of the loans and the collateral requirements would suggest that they are not.

“The CSPE found that IFAD’s targeting strategy was not sufficiently tailored to the needs of the different beneficiary groups,

including the poorest smallholders - the dehkan farmers”, underscored Indran Naidoo, Director, IOE. Future targeting strategies should be more effective in reaching genuinely poor dekhans, narrowing the gaps between men and women and between generations, in rural areas. Furthermore, IFAD could add value in policy and capacity building on issues such as pro-poor value chains, climate smart agriculture, public-private partnerships and private sector engagement. In this regard, future grants could be used to support piloting of innovations as they are developed.

Hon. Yuldashev K. Anvardjanovich Deputy Minister of Agriculture

IFAD and the Government must also work together to ensure data collection, analysis and use moving forward. Data should be collected according to a clear plan and analysed to ensure course correction as needed.

“Collecting data and statistics is one of our priories. We have to create a system of automatic data collection, and ensure transparency in collecting reliable data. For the future, we hope that in cooperation with IFAD we will be able to address these issues”, stated the Hon, Yuldashev Kakhramon Anvardjanovich, Deputy Minister of Agriculture.

Access full database [here] @ ioe.ifad.org

@ Dunyo news agency

Royal Asiatic Society of Great Britain ready to cooperate with Uzbekistan* Uzbek diplomats met with the Vice-President of the Royal Asiatic Society of Great Britain and Ireland (RAS), Barbara Brand. Discussion centered on cooperation between research institutes and relevant state organizations of Uzbekistan with RAS through the implementation of joint projects.

The meeting was also attended by the administrative director of the society, Alison Ohta, and the director of the RAS library, Edward Wich.

An exposition was organized for the diplomats, featuring individual manuscripts of the 11th-16th centuries relating to the history of Uzbekistan.

RAS representatives noted the provision of Uzbek experts with access to the society’s collections of manuscripts, the digitization of manuscripts of interest to research centers in Uzbekistan, online seminars and other joint work.

Founded in 1823, RAS is one of the most authoritative British cultural organizations that conducts research and international events in the field of science, literature and art dedicated to the countries of Asia. music was an honour. *source: https://kun.uz/

25

EVALUATION AS A DRIVER OF DEMOCRACY IN AFRICA

“I

n Africa , evaluation is seen as advancing demo cracy and improving rural liveliho o ds” , stated Indran A .

Naidoo , Director of the Indep endent O ff ic e of Evaluation of IFAD (IOE) in his keynote addre s s during the 10 th Af ric an Evaluation A s so ciation (Af rE A) International C onferenc e , on 16 March 202 2 .

Hos te d by the Zimbabwe Evaluation A s so ciation , the C onferenc e to ok plac e f rom 14 - 18 March 202 2 under the theme “ Evaluation that leave s no - one b ehind: emp owering pro gre s s towards the Af ric a we want amids t the COVID -19 pandemic and the other crise s and opp or tuni tie s f acing us ”. Par ticipation in the c onferenc e was drawn f rom a wide range of s takeholder s on the Af ric an c ontinent and globally, including National Evaluation A s so ciations , the Af rE A Emerging Evaluator Ne t work , world leader s in the moni toring and evaluation f ield and s tu dent s .

In his pre sentation , Dr Naidoo trac e d the grow th and journey of evaluation in the c ontinent , f rom the 3rd Af ric a Evaluation C onferenc e , held in C ap e Town in 20 0 4 , c o -ho s te d by the S ou th Af ric a Public S ervic e C ommis sion . T he keynote addre s s was de dic ate d to Dr Sulley Gariba and Profe s sor S tan S angweni , b oth late , who have b e en though leader s in developing the evaluation profe s sion on the c ontinent , providing supp or t to the prac tic e , and advancing evaluation f rom an Af ric an p er sp e c tive which is ver y roote d in c ommuni tie s . Dr Naidoo also paid tribu te to the C onferenc e organizer for Af rE A , Dr Zenda O f ir, for her thought leader ship.

26

@pixabay/Mitchell Luo

FURTHER READING

Evaluation capacities to advance sustainable development for all T he C onferenc e ac te d as the foundation for promoting and ad vo c ating Af rE A’s ‘ Made in Af ric a’ approach , and allowe d k nowle dge sharing , c ollab oration and ne t work ing wi th a wide range of international organiz ations and individuals . T hrough this interac tive approach , the event promote d learning and ac tion to supp or t evaluation prac tic e s that c ontribu te to re sp onsive , agile and sus taine d development in Af ric a . In addi tion , the c onferenc e also f acili tate d building the the or y and prac tic e of evaluation in Af ric a by providing a plat form for the development and sharing of a high - quali t y b o dy of c on tex tually-relevant k nowle dge on evaluation .

T he Af ric an Evaluation A s so ciation (Af rE A) was founde d in 19 9 9 , in re sp onse to Af ric a’s growing app eal for advo c acy, information sharing and advanc e d c apaci t y building in evaluation . T he organis ation’s chief fo cus was to c ounter limi te d evaluation op p or tuni tie s by building s trate gic bridge s for Af ric an evaluator s to c onne c t , ne t work and share exp erienc e s .

To ac c e s s the pre sentation by Dr Naido o, please click here To ac c e s s the event web -page , please click here

T

he 2 nd number o f the 2 1s t volume o f t h e Jap ane se Journal o f Evaluat i on Studi e s has b e e n publ i she d and i s now avai l abl e onl i ne . Th i s i s sue o f the Journal f e ature s an openi ng ar t i c l e b y Indran Nai doo, D i re c t or o f I O E , ent i tl e d ‘Evalu at i o n C apac i t i e s t o A dvanc e Sus tai nabl e D e ve l opm e n t f o r A l l ’. In the ar t i cl e , Indran sugge s t s there i s a n e e d f or i nve s tment i n e valuat i on f or gre at er de ve l o p m e n t e f f e c t i vene s s at the c ountr y l e ve l .

The paper i l lus trat e s chal l enge s i n l i nk i ng nat i o nal e valuat i on and A genda 2 030, and the i ni t i at i ve s o f e Independent Evaluat i on O f f i c e (I E O) o f U N D P t o suppor t c ountri e s i n di agno si ng thei r e valuat i o n c ap ac i t i e s . I t sugge s t s the e f f or t s o f the Uni t e d Nat i o ns shoul d f ocus on addre s si ng i ns t i tu t i onal and p ro c e s s is sue s t o ac c e l erat e e valuat i on c apac i t i e s . To t h i s end , the Uni t e d Nat i ons Evaluat i on Group (U N E G) , as an i nt eragenc y pro f e s si onal ne t wor k , w i th i t s t o o l s and e x per t i se , shoul d under tak e advoc ac y e f f o r t s t o advanc e the rol e o f e valuat i on i n the SD G f o l l o w u p and re v i ew proc e s s .

To ac c e s s the ar t i cl e , pl e ase cl i ck here .

27

Whole value chain approach found to be unique in Indonesia

T

he evolution from productivity-focused to whole value chain approach adopted by the IFAD portfolio in Indonesia has been noted as unique to the region. The Independent Office of Evaluation of IFAD (IOE) presented this finding during the Indonesia Country Strategy and Programme Evaluation (CSPE) national learning workshop, held on 18 March 2022.

Co-organized by the Government of Indonesia and IFAD’s IOE, in collaboration with IFAD’s Asia and the Pacific Division, the on-line learning workshop benefitted from the participation of representatives of the Government, development partners, civil society organizations, private sector partners, and IFAD senior management and staff. Over one hundred people joined the event, including from the Ministry of National Development Planning; Ministry of Foreign Affairs; Ministry of Finance; Ministry of Agriculture; Ministry of Villages, Disadvantaged Regions and Transmigration; Ministry of Public Works and Housing; Ministry of Environment and Forestry; Ministry of Home Affairs; and UN partner entities.

“I believe that our fruitful discussions will help to assess the results of IFAD’s strategy and programmes in Indonesia, as well as to generate findings and recommendations for improvement of

28

the partnership between IFAD and the government of Indonesia. This is crucial to effectively support our efforts towards inclusive and sustainable rural transformation”, affirmed Dian Lestari, Director of Centre for Climate Finance and Multilateral Policy, Fiscal Policy Agency, Ministry of Finance.

to reduce poverty and enable inclusive and sustainable rural transformation. The farmers themselves report that they have benefitted from farmer field schools and from working with village facilitators”, explained Indran A. Naidoo, Director of IOE. Another significant achievement was the Sustainable Management of Peatland Ecosystems in Indonesia (SMPEI) GEF5 project, which introduced an innovative approach for peatlands management as it engaged farmers in real-time fire monitoring and warning systems.

Dian Lestari

Director of Centre for Climate Finance and Multilateral Policy, Fiscal Policy Agency, Ministry of Finance

Meeting participants recognized the significant contribution that IFAD projects and GEF grants have made to national and regional policies. A notable contribution to policymaking includes improvements to village governance, particularly in participatory practices and the use of the Village Fund. In this context, services provided have been responsive to beneficiaries’ needs, as Village Facilitators provided effective support and motivation to beneficiaries, contributing to high beneficiary participation during project implementation.

“The partnership between IFAD and the government of Indonesia has been highly pertinent

Donal Brown

AVP, Programme Management, IFAD

“We are pleased to see that we have been successful in disseminating enhanced technologies which have translated into increased yields and productivity.

Of particular importance to us is the joint efforts of the government and IFAD on joint natural resource management, specifically on the peatland management which has received global recognition at COP 26, in Glasgow, as best practice”, noted Donal Brown, Associate

Country Strategy and Programme Evaluations

Vice-President, Management IFAD.

Programme Department,

Despite the priority given to innovation, sufficient time and resources have not been committed to develop useful monitoring and evaluation (M&E) and knowledge management (KM) systems, which are key instruments for documenting and sharing innovations and models.

“This has limited the possibility of demonstrating the achievement of projects, including innovations, scaling-up and the sharing of lessons with partners inside and outside of the country”, stated Dr Naidoo.

On the ground, evidence of rural impacts attributable to project interventions is scarce, while evidence is lacking of the projects’ impacts on human and social capital. Moreover, effectiveness in market access and value chain development has been limited. All projects faced challenges in establishing market linkages and value chain approaches adopted to-date have been largely promoting market orientation rather than being market-led.

Jeanette Cooke

Temporary Professional Officer

“Designs need to be less complex, and include components

on strengthening the capacities of implementing agencies and partners. Sufficient time and resources are needed to set-up the financial and management systems at start-up, and that needs to be factored into the design”, highlighted Jeanette Cooke, Evaluation Analyst at IOE. Going forward, priority must be given to developing simple, relevant, focused M&E tools for farmers to use themselves that can be aggregated for project purposes. KM should also be prioritized through a country programme wide strategy, which engages partners, promotes policy dialogue and stimulates regionally and internationally recognized technical capacity.

be designed to suit the capacity of implementing agencies, the needs of targeted districts, and project duration.

“The United Nations system, with the support from the Indonesian Government, remain committed in improving food systems by boosting rural development to achieve zero hunger and poverty in the critical times of a climate urgency”, underscored Valerie Julliand, UN Resident Coordinator in Indonesia.

Valerie Julliand

UN Resident Coordinator in Indonesia

Rd. Siliwanti

Director of Multilateral Funding Cooperation

“With the appropriate strategy and close monitoring, we except that the implementation of future programmes can be improved.

This country strategy and programme evaluation provides the right platform to highlight lessons, challenges and opportunities for our future development cooperation”, stated Ms. Rd. Siliwanti, Director of Multilateral Funding Cooperation, Ministry of National Development Planning of the Republic of Indonesia.

Since the beginning of its operations in Indonesia in 1980, IFAD has approved 21 projects, of which one was cancelled. Accounting for counterparts funding, the total cost of the remaining 20 projects is US$2,765 million, of which IFAD has financed US$670 million.

The nine investment projects covered in this evaluation received funding commitments of US$2.2 billion, with IFAD loans comprising US$449 million (21 per cent).

IOE’s latest CSPE in the country covers the years 2013 to 2021.

The two previous evaluations were completed in 2004 and 2014.

With regard to the development of future projects, these should

Access full database [here] @ ioe.ifad.org

29

STRATEGIC FACILITATION SERVICES NECESSARY FOR MARKET ORIENTATION OF LIVELIHOOD ACTIVITIES – ESPECIALLY IN CHALLENGING ENVIRONMENTS Village Development Programme in Indonesia Full report on-line here]

I

FAD-supported community-driven development programmes should take a phased, longer-term view of operations in order to support systemic and transformational changes in institutional capacities. In complex geographical settings, these changes should encompass establishment of well-resourced cadres of facilitators and facilitation services, which can maintain a presence in project areas after completion. This, according to the report published by the Independent Office of Evaluation of IFAD (IOE), on 1 February 2022.

The IOE report presents the findings of the project performance evaluation of the ‘Village Development Programme’ (VDP) in Indonesia. The VDP was the redesigned version of the ‘National Programme for Community Empowerment – Agriculture’, which was operational in Papua and West Papua from 2009 to 2014. The VDP was closed in June 2019, after two years of implementation. IFAD’s financing for the VDP was US$7.2 million.

The report found the impact of the VDP on human and social capital to be marginal. A contributing factor was the lack of priority that programme facilitators gave to technical capacity-building and training of groups. These were formed for the purpose of availing project benefits, without any particular affinity among group members. Moreover, facilitators could not link community groups to inputs, markets and information and encourage productive economic activities. Target groups also could not participate in local government planning and implementation. The presence of a robust and permanent facilitation structure, embedded in the local communities and able to respond to their needs, would have mitigated these operational shortcomings.

30

Project Performance Evaluation

Lack of facilitation and coordination further resulted in VDP’s activities failing to attract additional resources from ongoing public programmes. The VDP’s performance was also affected by changes in local policy environment and nascent capacities of institutional partners such as the Ministry of Villages.

In terms of achievements, the biggest success of the project was its scaling-up through the Integrated Village Transformation Programme (TEKAD).

Moreover, the VDP had a positive influence on the experiences of the Ministry of Villages and Development of Disadvantaged Regions in the form of feedback into the design of the TEKAD, which represents a much larger co-financing of over US$560 million by the national government, with a smaller investment of US$34.3 million by IFAD. The programme also featured several innovative features, including the extensive use of social media for disseminating information, knowledge management and monitoring and evaluation.

For the future, the IOE report recommends that IFAD invest in a long-term and well-resourced cadre of facilitators and facilitation services, which can maintain a presence in project areas after completion. In addition, future projects should include a detailed diagnostic analysis of the existing market demand, and map it with existing production patterns in the project area.

Papua and West Papua are the eastern-most provinces of Indonesia and are well endowed in terms of natural resources but lag behind the rest of Indonesia in terms of socio-economic development. The poverty rate is roughly three times higher than the average for Indonesia at large. Access to sanitation facilities and clean potable water also remains a challenge, particularly in Papua.

Access database [here] @ ioe.ifad.org

Breathing new life into Bali’s ghost buildings*

T

he Balinese urban landscape has dramatically changed during the past two decades. Modern development has generated a distinctly 21st-century phenomenon of Balinese, Indonesians and foreigners seeking sustainable investments and lifestyles. They have built properties and businesses to meet the growing demands of visitors and people relocating to the island. The onslaught, however, has led to an unsightly conundrum. Many buildings are constructed without contextual or sustainability consideration and are therefore abandoned, left incomplete or in disuse.

Israeli architect Ran Ben-Shaya, visiting Bali for an internship in 2018, was fascinated by the ghost buildings littering the island. His meeting with Indonesian designer Eva Natasa birthed a fusion of unique ideas. The outcome is Project Etere (PE) which introduces a fresh and potentially far-reaching experimental model. It seeks to counterbalance and offer an alternative to some of the development woes that describe modern Bali. Launched in 2019, PE integrates real-estate development, architecture, design, contemporary art, economics and social impact.

Realizing the potential for collaboration, Ubud-based Italian artist Marco Cassani responded to the concepts raised by Natasa and Ben-Shaya. His mission is to create social sculptures in the form of exhibitions and performances, adopting art as an instrument to help socialize and re-evaluate the territory, the Bali population and society in general.

“PE has transformed an abandoned second-story structure, neglected for five years, within a traditional compound into a self-contained one-bedroom apartment featuring modern architecture and design components. – Marco Cassani stated – “The expansion of PE involves the mapping of several abandoned buildings, intending to collaborate with the owners to transform and breathe new life into the structures.”

Sustainable solutions are vital during this unprecedented era of global disruption and socio-cultural change. Collaborative innovations from varying disciplines introduces an exciting alternative opportunity for the Bali urban landscape. *source: https://www.thejakartapost.com/

31

@pixabay/geralt

@pixabay/TheDigitalArtist

@pixabay/TheDigitalArtist 32

Evaluations face challenging times

T

he inadequate response of evaluators to COVID-19 and the importance of understanding the eco-system impacts of climate change adaptation interventions were the two salient issues that senior representatives of the Independent Office of Evaluation of IFAD (IOE) brought to the discussion table during the symposium organized by the Islamic Development Bank (IsDB), in close cooperation with the Global Evaluation Initiative (GEI), between 29-30 March 2022.

Titled ‘Reshaping Evaluation through the Lens of Recovery’, the symposium brought together evaluation practitioners from multilateral development banks, peer development institutions, academia, and the CLEAR Centers of Anglophone Africa and South Asia to strengthen and reshape evaluation work, and to forge a roadmap that facilitates progress towards the achievement of the SDGs and sustainable recovery from the global pandemic. Indran A. Naidoo and Suppiramaniam Nanthikesan, respectively Director and Lead Evaluator at IOE, provided important contributions to the discussion.

In a context of crisis, do evaluators stop rendering judgement and start supporting management in doing monitoring, or do they continue to evaluate and support the building of national evaluation capacity? Dr Naidoo placed this question at the centre of debate during the session titled ‘Providing Inputs During Times of Crisis’, noting the limitations of the response provided by evaluators to the COVID-19 pandemic.

“During a pandemic, reliance on national capacities becomes even greater because evaluators cannot fly-in. In the case of the COVID-19 crisis, this reliance was emphasized by the inadequate response rendered by evaluators. This was largely due to evaluators’ particular configuration, and does not mean that they are irrelevant”, stated Dr Naidoo.

While the entire world is tackling the challenge of recovering from the COVID-19 pandemic, examples exist of regional approaches where knowledge and capacities from either the COVID-19 experience or other disaster situations were successfully applied, as well as of evaluative national capacity building.

“The GEI is well-placed to address this situation, insofar as its mandate focuses on building national evaluation capacity.

This effort follows in the footsteps of UNDP’s National Evaluation Capacity series, which enabled conversations by national authorities to self-determine their futures. We need to move

away from a modus operandi that parachutes-in foreign consultants to render judgement. The real thrust is to facilitate what is already there”, remarked Dr Naidoo.

Another IOE contribution to the event was based on its recent thematic evaluation of IFAD Support to Smallholder Farmers’ Adaptation to Climate Change. In the session titled ‘Approaches and Methods to Climate Change and Environmental Evaluations’ Dr S. Nanthikesan pointed out that ‘Good is no longer good enough’ when it comes to climate adaptation. He underscored the need to assess the eco-system impacts (environmental sustainability) of climate change adaptation interventions. In addition, he noted that: “We need a conceptual framework of resilience and an approach to capture eco-system effects of agricultural solutions to be able to come up with winning adaptation solutions”.

Holistic win-win solutions are necessary to achieve climate, environment and development resilience together. These call for an adequate knowledge base of evidence-based holistic climate adaptation solutions and sufficient financial resources to enable required changes to existing practices. Evaluations can play a pivotal role in this regard, supporting the significant challenges that governments may face to incentivize sustainable climate adaptation responses.

“Climate adaptation must do no harm to the environment. To date, there is a clearer understanding of mitigation issues compared to adaptation, which is a relatively younger field. In this context, evaluations are critical for the development of an evidence-based knowledge base of sustainable climate change adaptation solutions, and for building awareness of the limitations of anthropocentric agricultural development”. Dr Nanthikesan reiterated the challenge of the context specific nature of adaptation interventions in building useable knowledge base of adaptation solutions.

The symposium helped to shed light on promoting evaluative thinking to enhance development interventions, to inform decision-making processes, and more notably to leverage partnerships in order to accelerate recovery in the context of the post global pandemic.

FURTHER READING

Evaluations under COVID-19 Fabrizio Felloni, IOE Deputy Director, has co-authored a paper alongside Roberto La Rovere and Sally Smith, published by the African Development Bank. The paper reflects on the experience of the Project Performance Evaluation of the Coastal Climate Resilient Infrastructure Project (CCRIP) in Bangladesh, conducted by IOE during the height of the COVID-19 pandemic in 2020. It focuses on how the project’s evaluation methodology and approach were adapted to the challenges and restrictions imposed by COVID-19.

These included the use of remote interviews and GIS data and imagery to compensate for the ban on international and domestic travel. Thanks to the local networks of national team members, the evaluation team was able to reach out to local technical staff and project beneficiaries in project sites that could not be visited. They participated in interviews, took photographic images, and recorded short videos documenting the road and market infrastructure built by the project, which the evaluation used as vicarious observations.

While ‘traditional’ approaches, entailing field visits and the collaboration of national and international specialists, remain the ‘first-best’ option, and while technology cannot substitute for field visits, this evaluation points to measures that can be taken to control for bias in data collection and analysis. However, an important caveat on the use of Geographic Information System (GIS) data and imagery is that CCRIP had a strong focus on infrastructure, making remote sensing suitable to assess the quality of construction and specific aspects of its use. The same may not apply to other types of projects or evaluative questions To access the paper on-line, please click here. @pixabay/geralt 33

Ad maiora: Chitra, Prashanth and Eoghan

T

hree pillars of IOE bid farewell to friends and colleagues at IFAD between March and April 2022, as they moved on to the proverbial ‘greener pastures’. Chitra Deshpande, Prashanth Kotturi and Eoghan Molloy have been with IFAD for a combined twenty-seven years, eighteen of which at IOE’s service – clearly, we said goodbye to a piece of our recent history.

Before all was said and done, and we wished our sought-after friends Godspeed, Independent Magazine found a quiet moment to sit down with Chitra, Prashanth and Eoghan for one final waltz down memory lane.

Good morning, colleagues.

Good morning, Alexander.

Prashanth Kotturi Evaluation Officer

Chitra Deshpande, PhD Senior Evaluation Officer

Eoghan Molloy Evaluation Officer

How many years have you served with IOE, and where and when are you going next?

I have been with IOE since 2012, when I started as an Evaluation Analyst. And now, after ten years of honoured service, I am on my way to the Evaluation Unit of the Green Climate Fund (GCF), where I will take-up my new role as Evaluation Specialist.

34

I’ve been in IOE for almost five years. This is significant, because I’ve been at IFAD for almost fourteen years and this has been my longest tenure in a position. I find this to be a testament to how interesting IOE’s work is. To keep the learning going, I will be moving to the Fund’s Operational Policy and Results Division (OPR), where I will be the Lead Advisor Policy and Results. The nice thing is that I will be in the Division that is the counterpart to IOE, and thus look forward to new challenges and continued relationships.

I joined IOE in 2019, so I’ve served the Office for about three years. I’m now taking up a senior position in the Independent Evaluation Department of the Asian Development Bank (ADB), in Manila, Philippines. With the ADB, I’ll be working on thematic and country-level evaluations, which is somewhat similar to the evaluations I’ve worked on here in IOE.

What do you feel most proud of during your time with IOE? I have seen this office evolve and grow in many ways, including the way it carries out evaluation, and approaches issues and the independence construct. It is a very mature office, and I have been fortunate enough to have been witness to it its evolution. The chance to contribute to this growth over ten years, this is what means the most to me.

Even just being a small cog in the machine, makes me feel very proud.

Working on the ARRI is what gave me the greatest professional satisfaction. I worked on it for three years, during which time I introduced new statistical methods to ensure validity and robustness of our data, and push new qualitative research methods, such as Nvivo. All of this arguably helped make the ARRI a better product. Within this context, I am probably most proud of the 2018 report, in which we recommended that the targeting policy be revised. The report provided a basis for us to organize an international conference on rural inequalities, attended by over two-hundred participants, which really raised the importance of this issue that is at the heart of IFAD. I am pleased to report that management responded to our recommendations by adjusting the targeting procedures, first, and by updating the policy, this year.

I’m proud of the number of evaluations I’ve completed with IOE, and also of the variety of assignments. I’ve led project performance evaluations in very different contexts – such as India, Senegal and Uzbekistan –, but I’ve also worked on a country strategy and programme evaluation in Pakistan – which was a massive undertaking given the size of IFAD’s portfolio in the country, and the ‘remote’ methodology that we used.

This being said, my proudest moment was when I went to a village in China and I chose to have lunch with the rural beneficiaries, rather than with the local authorities. At the end of the visit, I was speaking to an elderly man about his life and he turned to me and said “you really care about us”. That was the proudest moment of my career at IFAD. A couple of weeks later, a staff from OPR visited that same village to design a project and she met that same beneficiary, and he told her that I care about them. That was the highest accolade I could receive.

I was also proud to serve as IOE focal point for gender and disability inclusion. This involved overseeing our annual reporting on UN system wide action plans to improve gender mainstreaming, and disability inclusion. I spent a lot of time liaising with colleagues from the evaluation offices of other UN agencies, particularly on the development of guidelines for addressing the inclusion of persons with disabilities in UN evaluations.

This is an area of work that is still ongoing, but I was proud to have been involved during these formative discussions.

I have also been the coordinator of IOE’s validations of IFAD project completion reports – we do about 25-30 of these validations every years, and each one is like a mini-evaluation. As coordinator, I streamlined the format, reducing the length, focusing more attention on the ratings and the areas of disconnect between IOE’s assessment and that of IFAD management.

35

What has been the biggest challenge that you have overcome whilst working for IOE? This was early on in my career. I was one of the only two long-term junior consultants in my cohort that came to IOE. I was expected to support high-level evaluations, but had almost no other junior staff to fall-back on, or young people whom I could ask the nitty gritty of where to find information, etc. Understanding how the organization works, who does what, whom to approach for information (being dismissed many times!)

– it was a very steep learning curve. It was trial and error for the first year here. After all these years, that remains the biggest challenge in my professional career. I was thrown into stormy high seas and expected to swim. This was not done on purpose, it was simply the result of the scale and magnitude of the evaluations that IOE typically undertakes.

IOE managers supported me to the extent they could, but, there was only so much they could do.

I think that one challenge, which is particular to IOE, is the evaluator’s role of ‘speaking truth to power’. Being able to present our findings in a very open manner is a privilege that we have as independent evaluators. Unfortunately, this approach is not always well received, and can thus be very challenging at times. For example, it was especially difficult to present results from the ARRI that were not positive in circumstances such as replenishment consultations. Nevertheless, we have to believe that disclosing these findings in a transparent manner is part of the Fund’s strength. Maintaining this integrity requires our data and methodology to be air tight, and also requires excellent communication skills – because while the saying goes “don’t kill the messenger”, there is an instinct to do so!

The obvious challenge was adapting to the COVID-19 pandemic and the new ways of working and conducting evaluations. On this front, I’m very proud of how we at IOE ensured business continuity.

Even in the darkest days of lockdown, in 2020, IOE quickly adapted evaluation methodologies and approaches, and we delivered our full work programme for 2020 and 2021, despite the difficulties.

Of course, it has been a challenge managing and leading evaluations remotely – not being able to physically travel to project sites, to meet with stakeholders in person, and see the results first hand. We have had to place a lot more trust and responsibility on national-level evaluators. This has been a learning process on both sides – for us here at HQ, and for our national evaluators, who have taken on greater responsibility, leading in-country field visits, for example.

COVID-19 has shown that a lot of our work can be done remotely. However, certain activities do require in-person interactions, and ‘boots on the ground’. I think, going forward, all of us in the evaluation community need to take stock of the experiences during the pandemic to decide what can continue as ‘remote’ and what specifically requires in-country travel, especially if we are to aim for carbon neutrality and reduced emissions from excessive travel.

36

What will you miss the most about IOE? The thing I will miss the most is the camaraderie. I made friends for a lifetime. For me, this is much more than an office. I gave my ‘twenties’ to this office, and this office gave my twenties to me. People have come and gone, but we have still keep in touch even if they are in other corners of the planet (IOE has emissaries in evaluation offices all across the world!). This is not something that you can take for granted, I assure you. I will most sincerely cherish this, from the bottom of my heart.

Besides the daily interaction with our colleagues, I will miss the independence. As I said, independence brings challenges. However, independence also allows us to have open and candid conversations and discussions with IFAD staff and partners, including governments, because they know that conversations are confidential and they can speak freely. Going to management, there will be different power relations that might interfere with these dynamics.

I will miss my colleagues and the collegiate atmosphere. I will miss the dutiful attention to quality and rigour in all our evaluation products, and the healthy atmosphere of peer support.

The peer review processes at IOE are excellent. There is a very strong sense of peer support. Our colleagues and supervisors always push us to grow and perform better.

There is a real sense of intellectual rigour in IOE.

Looking forward, what is the most significant lesson that you have learned or skill that you have acquired with IOE that you think will help you to make a difference in your future endeavours?

The one thing that I really will take with me, which is a trait, is the value of perseverance and optimism for the future. There will be times when things don’t go your way, when you don’t live-up to the expectations of others. Yet, there is always a new day ahead. A time will come when you will get accolades for an evaluation. The important thing is to persevere. I say this not because I was subject to any maltreatment, but because this is something that my supervisors and colleagues have taught me through their actions. The optimism that IOE colleagues have taught me is something I will really take with me.

The thing that I’ll bring is the IOE experience. I’ve been with IFAD for fourteen years, and what is unique to IOE are its independent in-depth field missions that bring you so deeply into the country context. The experience of travelling to the very remote villages where IFAD operates, sometimes where beneficiaries have not met IFAD staff before, to interact with the people on the ground, to witness their lives and have long conversations with them, this is what will bring huge value to my new role. Having ‘touched the data’, I know what the real issues are, and this ground-trothing will allow me strengthen IFAD’s corporate M&E systems, as well as our messages and results reporting.

I will definitely carry forward this attention to detail, rigour and quality in my evaluation work. At IOE, I felt there was genuine interest from my supervisors and senior management in the quality of my work, and in every word that I wrote in my evaluation reports. This attention really motivated me in my work, propelling me to do better, to write better, to pay closer attention to details.

37

Any final words?

Something that I feel very strongly about, and which comes from my personal experience is: “giving people a chance”. I come from a noname college. I did not have the opportunities of being a UN Junior Professional Officer (JPO). I didn’t have an influential mentor that would give me that first ‘foot in the door’.

The Director who first hired me, Luciano Lavizzari, and Fabrizio, who mentored me in this office for a very long time, both gave me the chance to prove myself. Subsequent directors such as Oscar and Indran, and supervisors such as Nanthi and Johanna gave me a chance to take on increasing responsibilities. A chance is what anyone asks for. Many deserve it but only a few get it.

This office gave me a chance when there were easier paths that it could have followed. It could have given that chance to someone else, but it chose to give it to me instead. I’ll never forget this, never. I hope to emulate this in the future, by giving those same chances to young people to grow. I will always value and cherish this the most, it will always be with me, no matter where I go.

When I look back, it’s really very difficult not to be emotional.

38

It’s a very privileged position that we have as evaluators at IFAD. The ability to think deeply and independently about the work that the organization does is really unique to this office. On a personal note, I will miss the staff and all that we built together. In my new role, I am actually hoping to help build bridges between IOE and management, and to play a strong role in implementing the new evaluation policy that envisages this stronger learning loop. I am really looking forward to bringing this added value, and helping IOE’s products to gain even further use and exposure – which they thoroughly deserve.

Professionally speaking, going forward, I hope to emulate the style of support that I received in IOE. This means showing genuine interest in the work produced by people whom I may supervise in the future, mentoring junior colleagues to grow professionally, and helping them to develop the skills to produce better quality evaluations.

On a personal level, Manila will be a big change from living in Rome – it will be an adventure, that’s for sure!

IFAD striving to increasingly define beneficiary target group

“W

e found, in 2018, that there was a lack of agreement within IFAD on the Fund’s target group and the strategies needed to reach them,” reported Chitra Deshpande, Senior Evaluation Officer of the Independent Office of Evaluation of IFAD (IOE), during an on-line seminar on 4 February 2022.

Organized and hosted by IOE, the seminar brought together over ninety IFAD staff, including from the Gender & Social Inclusion team from the Environment, Climate, Gender and Social Inclusion division (ECG), as well as from different divisions at country, regional and headquarters levels.

Enriching discussions took place on targeting performance, methods and issues, during the event.

Participants also had the opportunity to reflect upon and discuss the findings and conclusions of the literature review on ‘Targeting of the Poor and Ultra-Poor’ carried out by Professor Tauhidur Rahman, associate professor in the Department of Agricultural and Resource Economics and Department of Economics at the University of Arizona.

“The traditional approach used to be conducting extensive household surveys, and based on that you determined who was poor. However, this approach in addition to being expensive has a lot of measurement errors. In response to these limitations, there are five alternative methods that can be found in the literature. These are proxy means test-

ing, geographic targeting, demographic targeting, self-targeting, and community-based targeting”, Professor Rahman noted during his presentation.

Advantages and disadvantages of each of these approaches were discussed.

Dr Deshpande moderated the seminar, which also featured opening remarks by Dr Indran A. Naidoo, Director of IOE, an overview of the review of the IFAD targeting policy by Ndaya Beltchika, Lead Technical Specialist on Gender and Social Inclusion in ECG, and closing remarks by Fabrizio Felloni, Deputy Director of IOE. During her intervention, Ms. Beltchika explained that within IFAD there is a strong commitment to focus the Fund’s target group on the poorest of the poor, including indigenous peoples and people with disabilities.

Feedback gathered during the discussions will provide substantive inputs into the Evaluation Synthesis Note (ESN) on targeting, which IOE is conducting in 2022 to contribute to the updated IFAD targeting policy that will be presented to the Executive Board in December 2022. The ESN is a new IOE product. Dr Naidoo explained that the goal of the ESN is to learn from experience and practice based on evidence from independent evaluation, through a rapid assessment, with a view to providing timely and useful input for decision-making in IFAD.

@pexels/Quang Nguyen Vinh

39

A LIFETIME OF SERVICE TO THE RURAL POOR

T

hirty-five years as a UN development practitioner, thirty-two of which with IFAD. After an illustrious career filled with accolades and notable appointments, Shantanu Mathur embraced a much-deserved retirement in March 2022. Trying to recount the highlights of this exemplary journey is an industrial task in its own right.

Shantanu served as Lead Adviser of IFAD’s Global Engagement and Multilateral Relations, in his last post, prior to which he was the founding Head of the Quality Assurance Group in the Office of the President and Vice President, the Head of the Corporate Grant Secretariat, and Lead Adviser to the Shantanu Mathur Associate Vice President for Programme Management. Not to mention that, Lead Adviser of IFAD’s Global Engagement and Multilateral Relations externally, he was co-chair of the Global Donor Platform for Rural Development, the first CGIAR Fund Council Chair of the Evaluation and Impact Assessment Committee, and also served as Vice-Chair of the CGIAR Finance Committee. The five books he edited and contributed to in the fields of rural development, research and impact assessment help to round-off Shantanu’s remarkable profile.

On the eve of his retirement, Independent Magazine had the privilege to catch-up with Shantanu for what proved to be an extremely pleasant, insightful and candid chat.

Good morning, Shantanu Good morning, Alexander

Could you tell us about your professional experiences in dealing with evaluations?

When I joined IFAD in 1988, the organization was rather nascent. A lot of the approaches to poverty reduction and eradication had not been fully developed and practised across the international development system, at that time. We were rather naïve in how we developed our investment portfolio, piggy-backing significantly on World Bank project design and appraisals and focusing our rural poverty reduction approaches very sharply on those engaged in agricultural production. We have since very much moved on, focusing more on the broader rural space, embracing off-farm activities and looking at value chains (with employment generation opportunities for the youth) and now, increasingly the rural-urban nexus, straddling food systems all the way ‘from the farm to the fork’. This coherent systems approach was developed only because evaluations threw up those hard lessons and informed the way that we think about the inclusive rural transformational agenda and space.

Evaluations of IFAD’s portfolio over the decades have been key in developing a much more sophisticated understanding of how we should design programmes – the process and content – in order to have a better impact on rural poverty reduction.

In your opinion, is it important to have independent evaluation for IFAD to course correct?

Course correction is extremely important. That is what we have been doing almost on a continuous basis for the past three or four decades. You used a very important term: ‘independence’. To better understand the impor40

tance of this term, we need to look at the evaluation architecture at the Fund and explain that different typology of “evaluation” co-exists at various levels and across different departments, within IFAD. All of these are an important part of the evaluation culture at IFAD. And all the evaluation workstreams are complementary to each other. On the one hand, you have IOE looking at the relevance, performance, effectiveness and efficiency of our portfolio. On the other, you have ex ante assessments (appraisals – which fall within the broad rubric of application of evaluation methodology); you have the Operations Policy and Results (OPR) division, which plays a key role in reporting on self-evaluations, bringing into the picture many key stakeholders and also serving as an important monitoring function. You also have the Results & Impact Assessment (RIA) division, which deploys impact analysis tools and application of sophisticated quantitative methodology in its work. This is an intrinsic part of any evaluation process, going well beyond results, looking at development outcomes and impact. However, these aspects of evaluation are also intrinsic to IOE. For this reason, I feel that there is a need to build greater complementarity towards a better interface between these divisions and functions. There needs to be a better rationalization of what these functions mean – do we need boundaries and maintain the integrity of the independence you spoke about? Or perhaps we need to build a more coherent approach, so that these functions are not counter to each other but rather mutually reinforcing and complementary – with sufficient firewalls built into processes so that the learning objective of evaluations go hand in hand with the accountability required.

So, you are saying that in order for IOE to focus more on the impact, there needs to be greater coherence with the functions that other outfits are performing?

Yes, indeed - but always without undermining or impinging on the independence of IOE. A carefully- managed consultative approach is what is called for, to draw on insights and conclusions, and develop relevant and meaningful recommendations to inform the next generation of the portfolio. I would say, building links and complementarities, reinforcing and mutually informing each other. In the nineties, we used to have a dedicated ‘core learning partnership’ attached to every project evaluation. This was a great learning platform complemented by a tool called the Evaluation Knowledge System (EKSyst) which reinforced the feedback loop important for successive generations of project design and implementation. The problem was that over time in this very participatory evaluation approach perhaps the “accountability” dimension may have gotten undermined – or at least perceived to have been lost somewhat. The learning objective was great and is extremely important, but the evaluators were seen by some to have ‘gotten into bed’ with the management and sometimes co-opted into the whole process in a manner that may have weakened their independence.

This being the case, what do you see as being the main opportunities for evaluation to further permeate an evidence-based learning culture across IFAD? IFAD’s evaluation function has come a long way. As IFAD’s own portfolio design has become more and more sophisticated, so has our understanding of the triggers and influencing factors of the attractors, where systems tend to gravitate towards. This has happened because we have moved from single-sector approaches, towards a great appreciation of intersectoral linkages, interdisciplinary and multi-stakeholder approaches to conducting evaluations. We now have a deeper understanding of the entire spectrum of the policy change and of the institutional architecture in which our projects are embedded. However, we need to move towards an even more systemic approach to how we develop better informed theories of change to better capture the triggers and influencing factors, including the socio-economic, cultural, bio-physical, policy and institutional underpinning.

I am talking about a more dynamic understanding of how a project was designed, in order to build a clear line of logic. This approach needs to be informed by both qualitative and quantitative evaluation methods. Evaluations must remain objective but, while assessment of performance is important a rigorous assessment of the development impact of the “intervention” needs to be better captured and articulated. Going forward, we need to build more sophisticated tools for use by M&E units – tools which will help us to be ready for surprises – for instance, counter-intuitive results, which in turn will require us to adjust our theories of change as we go along.

41

Previously, you touched upon the importance and at times the difficulty of striking the right balance between learning and accountability. In light of what you just said now, do you think that having a more dynamic outlook and more flexible theories of change will allow us to find the right balance between learning and accountability?

Very well said indeed, that’s exactly how I would put it. I feel that the learning objective has improved IFAD’s portfolio a lot and IFAD is seen as the leading IFI, globally, in the creation of rural vibrancy in poorly served and marginalised geographies of the developing world. As I mentioned earlier, robust evaluations at IFAD have also helped us to move away from looking only at project performance and better appreciating development outcomes of rural poverty reduction. However, this must never come at the cost of undermining accountability, which is crucial. It might be seen as unnecessary finger-pointing by defensive managers – but sometimes this is a necessary pain to endure, if we are to improve.

You mentioned the importance for evaluations to capture and decipher counter-intuitive results.

Do you have any examples that might help illustrate this point?

There were times in which we went with the single objective of income generation in poor rural communities, but subsequently found that the project achieved this goal at the expense other important development outcomes. No one thought of the possible trade-offs. For example, in a project in Sri Lanka we improved the incomes of communities in a big way. Overtly, it was an outstanding success story. However, the more income communities disposed of, their basket of consumption goods changed. With increased income they started to purchase non-traditional food items such as white bread or more butter. This led them to move away from a well-balanced diet. This was captured by an evaluation, which had a nutritionist/socio-economist on-board. and which took an interdisciplinary approach looking at consumption patterns. This resulted in some sobering findings. In another instance, in West Africa, a project managed to build wells in the middle of a village with the overt objective of reducing the drudgery of rural women. Prior to the wells being built, the women had to walk for miles to fetch water from the nearest body of water. However, when the evaluators elicited feedback from the women participants in the project – the so called primary beneficiaries – they were surprised to find the women extremely unhappy with the “wells” component of the project. Going to fetch the water, away from the hustle and bustle of their busy village chores, away from the men-folk, gave them the time to chat and interact among themselves and build their social capital. In the end, the labour time saved by the availability of water in the village wells - resulted in more time available and devoted to activities such as weeding – a double-whammy – but who should’ve known? These are counter-intuitive results and lessons that get captured only if you equip the evaluation team with sophisticated development professionals. Such seemingly well-intentioned project activities would, in future, be designed with more careful consideration of the whole spectrum of possible outcomes – in consultation with the primary “target-groups”.

In closing, is there anything that you would wish to add?

I would just like to congratulate IOE for having invested heavily in the development of new outreach products, for having brought onboard such talented communication professionals, and for striving to deploy the latest and most cutting-edge evaluation tools available. These aspects, married with the very strong and insightful new leadership of the Office, will help ensure that IOE is able to maintain its full independence while further building the learning culture into its work, that we spoke about earlier, so that people won’t look at evaluations as painful exercises but as painstaking activities to help us improve performance and impact - something that can truly help them along their journey as development practitioners. Many congratulations and all the best in this endeavour to further improve the evaluation culture at IFAD. Thank you very much for your time, Shantanu.

You’re most welcome, Alexander.

42

Evaluation capacity building at country level is a collaborative effort

M

onitoring and Evaluation (M&E) is a powerful public management tool that can be used to improve the way governments and organisations achieve results. Thus, there is an attendant need for enhanced results-based M&E of policies, programme outcomes and project results that aim to forward sustainable socioeconomic development in Africa, as well as across the globe. Indran A. Naidoo, Director of the Independent Office of Evaluation of IFAD (IOE), made this point in his presentation during the inaugural lunch of the Regent Business School Journal Club, on 15 March 2022.

In a globalised world, there are growing pressures on governments and organisations around the world to be more responsive to the demands of internal and external stakeholders for good governance, accountability, transparency, greater development effectiveness, and delivery of tangible results. In response, most countries in Africa have put forward strategies to reform their governance frameworks as part of national development plans, and to a lesser extent as part of anti-corruption strategies or efforts to improve governance, accountability and transparency. However, progress has been uneven between countries and challenges remain in several areas, such as public service delivery, insufficient checks and balances, high levels of corruption and inequality.

Reflecting on the National Evaluation Capacity (NEC) series – which grew to become the largest UN evaluation capacity building event by country participation, under his leadership – Dr Naidoo underscored that evaluation capacity building at the country level is a collaborative effort. This effort should encourage the public sector to pursue and sustain excellence in basic services provision with specific reference to service standards, accountability and transparency. As such, M&E systems should act as critical development tools that enhance the legitimacy of governments by holding them accountable, and ensure a more transparent public service.

The Regent Business School Journal Club is an educational platform in which a group of individuals (academics, researchers and students) discuss published articles, to keep themselves abreast of new knowledge, promoting in them the awareness of current research findings, teaching them to critique and appraise research, and encourage them to utilise critical outcomes of the discussions for their own research, and teaching and learning applications. The event has been established as an extramural activity for Staff, Students, Alumni and other invited guests.

@pexels/Magda Ehlers

43

Communicating, independently

Alexander Voccia, PhD Coordinator, Evaluation Communication Unit [online profile]

F

or many of us, working in the field of evaluation, the construct of ‘independence’ lies at the heart of what we do. Independence permeates our work, subsuming freedom and rigour, infusing impartiality, neutrality and unbiasedness into investigative methods, paving the way to useful, constructive and learning-oriented reports and recommendations. I ask myself, though, how does all this translate into communication products and outreach materials, in practical terms?

44

We need to be clear. There is an unsubtle difference between communicating independence and independently communicating. We, the niche of evaluation communication specialists, may be tweeting about the latest report that our evaluation office has published, sharing excerpts on social media, and cross-posting soundbites across blogs, podcasts and the like. Does this really mean that we are exercising the same independence that our colleagues exercised when they drafted the report? No way.

What we communicate and the way in which we do so are two completely different things.

Let’s cut to the chase. Posting information about a report that was written in an independent fashion adds up to nothing more than communicating independence, i.e. we are telling

people that a report has been published in observance with independence criteria. As for us, we are likely channelling this information through corporately owned outreach channels, using cookie-cutter media solutions, adopting predefined templates, abiding to standardized communication guidelines – and are maybe even subject to external vetting. I don’t know about you, but that doesn’t sound much like independence to me.

Make no mistake, this is no trivial matter. The credibility of our offices’ work is at stake. Are independence and transparency perhaps not synergistic?

Indeed, they are. “Independent evaluation supports learning to improve transparency and accountability”, affirmed Dr Indran A. Naidoo, Director of the Independent Office of Evaluation of

IFAD (IOE), in the preface to the second edition of Independent Magazine [here]. However, our ability to leverage independence to improve transparency is debatable – at best – if we are constrained in the shape and form that our outreach products take.

If we are to leverage independence to improve transparency, then we need to ask ourselves if we are in a position to independently communicate. It’s quite simple. We need to look at the extent to which we are able to decide how our content is presented, packaged, posted, shared and disseminated. Last year, we at IOE did just that.

Lo and behold, our strategic communications assessment found that a lack of autonomous ownership was affecting the brand awareness that need45

ed to be associated with the independent nature of our office. This, in turn, hindered our levels of stakeholder engagement by hurting our transparency. In simple terms, from a communications standpoint, the claim that IOE maintained full intellectual independence was partially skewed by the fact that its major outputs were almost exclusively accessible only through IFAD’s corporate gateways. Clearly, something needed to change – and it did, big time.

Over the past twelve months, IOE has re-positioned its brand identity. A cohort of new communication resources now define the office’s visual persona, embodying its independent stature tout 46

court. Two products best reflect this newfound ability to independently communicate: IOE’s website and its Independent Magazine.

Written, graphically designed, managed and published entirely in-house, Independent Magazine has ratcheted-up an audience of over 8,500 people across 83 countries – this, with its first two editions alone. Arguably, merit of an unconventional approach to independently communicating evaluations.

Independent Magazine brings to the forefront of the global development dialogue the major efforts undertaken by IOE, while seeking to advance IFAD’s vision of vibrant, in-

clusive and sustainable rural economies, where people live free from poverty and hunger.

As IOE’s flagship communication product, Independent Magazine embodies the Office’s re-envisioned focus on multilateral strategic engagement, permeating international fora and global evaluation networks such as the United Nations Evaluation Group (UNEG), the Evaluation Cooperation Group (ECG) and the Global Evaluation Initiative (GEI). To present the richness of rural life, and detail facets of local community lifestyle, Independent Magazine also zooms in on cultural activities and landmark occurrences in countries featured by IOE’s evaluations.

The Magazine’s impact has been bolstered by the launch of IOE’s first fully independent website. The website, for which the Office maintains full intellectual ownership, is structured to best meet the specific needs of IOE, with the adoption of dynamic functionalities that maximize opportunities for user engagement and ensure an intuitive, easy navigation experience.

The platform adopts a strong visual identity that ensures continuity with IFAD’s image whilst providing scope for a clear, coherent and visually independent brand image.

Most importantly, the website speaks to the core of what our users look for. This means that messages are tailored and products customized. For us,

cookie cutter solutions are a thing of the past.

Through the website, the magazine and the assortment of new products, we are building safe spaces for user interaction, which invite our stakeholders to continuously reach out to us in a dynamic, spontaneous and organic fashion. Exciting opportunities for a more personal connection between our Office and the people we serve are burgeoning.

Admittedly, since “beauty is in the eye of the beholder”, the fact that we have drawn inspiration from some of the most innovative and graphically appealing visual solutions adopted by award-winning designers is helping to drawn

people in. ‘Clean, crisp, fresh and modern’ is what people are looking for – and that’s exactly what we are striving to give them. However, fancy layouts fall short of capturing the results that we are beginning to enjoy.

Credibility dictates the impact of all our communication efforts. If we are credible, stakeholders are inclined to listen-up and give credit to what we have to say. It is then – and only then – that we achieve genuine engagement and, consequently, buy-in. It’s all about transparency. The ability to independently communicate lies at the heart of this construct. That’s what’s driving our credibility. That’s the game changer. @Unsplash/alexander cifuentes

47

Independent Office of Evaluation International Fund for Agricultural Development Via Paolo di Dono, 44 - 00142 Rome, Italy Tel: +39 06 54591 - Fax: +39 06 5043463 E-mail: evaluation@ifad.org www.ifad.org/evaluation www.twitter.com/IFADeval www.youtube.com/IFADevaluation

48