Independent Magazine - Issue 1, 2021

Independent

Accountability. Learning. Partnership.

MAGAZINE

T H E N E W D I R E C TO R Dr Indran A. Naidoo: Captain on board

BREAKING THE MOULD Discussing pandemic-shaped evaluations

Independent Office of Evaluation

Issue n.1

2021

I N F R A S T RU C T U R E

Most successful IFAD investments

1

Independent Magazine brings to the forefront of the global development dialogue the major efforts undertaken by the Independent Office of Evaluation of IFAD, while seeking to advance the organization’s vision of vibrant, inclusive and sustainable rural economies, where people live free from poverty and hunger. To present the richness of rural life, and detail facets of local community lifestyle, Independent Magazine also zooms in on cultural activities and landmark occurrences in countries featured by IOE’s evaluations.

Editor-in-Chief: Indran A. Naidoo, PhD Editor, writer and graphic design: Alexander Voccia, PhD @Unsplash/Frame Harirak 2

CO N T E N T S 04

Breaking the mould: discussing pandemic-shaped evaluations

25

Successful community-based infrastructure investments

07

Evidence Summary on COVID-19 and Food Security

26

IFAD projects contribute to household incomes in Uganda

08

IFAD-supported innovations in Morocco

27

Smartphones and agriculture: are the times ripe in Uganda?

09

The 2020 Argana International Poetry Prize

28

Territorial approach bolsters living conditions in Niger

10

Dr Indran A. Naidoo: captain on board

29

At a glance: Ecuador country strategy programme evaluation

18

Gathering community feedback in times of remote evaluation

30

Behind the glossy cover: communicating evaluations

20

IFAD project in Bangladesh is a climate resilience pioneer

32

More time needed for communityapproach in Tajikistan

21

Bangladeshi teenager awarded Children’s Peace Prize

33

Cultural contest to celebrate Tajikistan independence

22

Environmental consequences of development interventions

34

IOE Family: Farewell, Hans!

24

Evaluating Environment in International Development 3

BREAKING THE MOULD: Discussing pandemic-shaped evaluations

A

irplanes grounded. Cruise ships at bay. Passenger trains silenced. Post-apocalyptic science fiction?

No, COVID-19. The year 2020 caught the world by storm, but there were those who found a way to weather it. The ability to react to the pandemic proved the prerogative of those able to adapt to the unforeseeable, to react to the unimaginable. While perfect coping strategies may have been difficult to come by, creative planning, imaginative approaches and blue-sky thinking paved the way for some exciting results.

The pandemic called for a resourceful mind-set; and a resourceful mind-set was what IOE put on display.

As the first evaluations carried out by IOE during the pandemic are becoming publicly available, Independent Magazine caught-up with Fabrizio Felloni, IOE Deputy Director, for an insightful one-on-one chat.

Good morning, Fabrizio.

Good morning, Alexander.

What were the main challenges IOE was confronted with in the face of the pandemic?

Arguably, the main challenge was the inability to conduct international missions, as well as extensive field data collection, which had a number of implications for IOE’s work. For instance, we had to consider how to maintain methodological rigour and credibility vis-à-vis corporate governing bodies, how to command sufficient attention from IFAD management and government counterparts, how to maintain IOE office functionality and working relationships, and how to ensure the well-being of all staff members and consultants, to name but a few.

To address these challenging circumstances, what measures did IOE take to ensure robustness of the analysis, while maintaining the functional and behavioural independence of the evaluation team?

Evaluations adopted an innovative combination of approaches. These included remote interviews and mini-surveys, where feasible, by phone, Skype and Zoom, in addition to questionnaires sent via email to stakeholders. We also tested alternative methods for primary and secondary evidence collection and validation. For instance, in some cases we used geospatial data and analysis, especially for projects investing in physical infrastructure (for example on market infrastructure in Bangladesh). A colleague also tried Rapid Evidence Assessment for his work on climate change adaptation for smallholder farmers. This technique synthesizes secondary evidence related to an evaluation from published literature and websites. We also used external reviewers to crosscheck contextual information and enhance the quality of the analysis.

Obviously, given that IOE staff and international consultants could not travel, more responsibilities were devolved to national consultants.

4

Did this increased reliance on national consultants pose challenges in terms of impartiality and behavioural independence? If so, how did IOE go about it?

Inevitably, there can always be risks. National consultants may receive pressure from national stakeholders, or might not be entirely impartial if they have spent a significant part of their professional careers working with the Government or the local IFAD Country Office. To minimize these risks, we spent additional time briefing them, preparing questionnaires and checklists, and agreeing on the scope of their work, so that, for example, they would focus on collecting factual data. In addition, we ensured that the whole team, including IOE staff and international consultants was fully engaged in discussing and triangulating the data and information collected.

What about your office, how was it affected by this rapid turnover of consultants?

First, I must say that, in addition to consultants, interns also proved critical for delivery. As for team building, team meetings and staff meetings were held during the lockdown and partial lockdown period. However, engaging the whole team remained a challenge in the absence of more informal and unplanned opportunities for interactions and gatherings.

These challenges notwithstanding, did the cancellation of international travels offer a silver lining in terms of cost savings?

In part this was true, but not for all evaluations. The reality is that cost savings were often offset by the need to increase the workload for international consultants – for example to conduct more thorough desk review or studies, and to collect alternatives to primary evidence, such as geospatial data and analysis, and secondary evidence – or by the increased number of national consultants.

In light of all this, would you say that the alternative approaches and methods used

by IOE provided a valid substitute for traditional evaluation approaches?

Valid, yes. Perfect, no. To be honest, I would say that there is no perfect substitute for face-to-face meetings, household interviews and primary data collection through field visits. No available technology can replace in-person interactions with stakeholders, visits to project sites and interaction with the beneficiaries.

5

Are you suggesting that, ultimately, the results achieved by the evaluations were below expectations?

I would not say that, no. What I would say, instead, is that limitations of the evaluations during COVID-19 have to be clearly stated up front, where possible indicating the level of confidence of findings, as well as those where more field visits would have helped. It is important to highlight the substitute methods undertaken to warrant the level of confidence claimed. The level of confidence brought about by substitute methods can be adequate when evaluating completed operations, especially for those with good, monitored evidence. This may not be the case for new and ongoing projects.

That being said, do you think that the experience in dealing with COVID-19 offers opportunities to revisit some of the evaluation practices and processes, which could be considered not only for future crises but also for ‘normal’ times, to add evidence and simplify processes?

Absolutely, yes. There are a number of very valuable lessons learned and best practices that we could look at mainstreaming. For a start, rationalizing travels and reducing the ‘carbon footprint’ of evaluations is important, regardless of crises or pandemics. The COVID-19 experiences has in part proven that this is possible. We can enhance efficiency of country missions through a combination of remote and face-toface, in-country interviews and greater emphasis on project site visits. We could spend less time driving in the traffic of capital cities while moving from an interview place to the next one. Some of the interviews in the capital could in fact be replaced by remote interviews. We can develop mechanisms and practices to ensure more in-depth document review and preparation for missions and interviews. We can make greater use of information technology to augment evidence and analyse secondary evidence.

Any final thoughts?

I would simply add that in case of (partial) lockdown, it is important to ensure that teamwork is not interrupted, while recognizing the social-psychological burden this entails. In particular, we must always pay special attention to colleagues who are away from their family and network of friends. This may even call for advocating for institutional support for families to balance professional and family needs. At the same time, it is also important to avoid overwhelming staff and teams with formal gatherings and very lengthy video interactions.

Thank you very much.

You are welcome.

6

FURTHER READING WW

Evidence Summary on COVID-19 and Food Security The UNEG report entitled Evidence Summary on COVID-19 and Food Security is now available here.

Published under the auspices of UNEG, the report is a joint effort of IFAD, FAO, WFP and UNIDO evaluation offices, funded by United Kingdom of Great Britain and Northern Ireland. The study aims to contribute to a greater effectiveness of the UN system response to COVID-19 in the area of food security. To do so, the report draws on findings, good practices and lessons learned from 65 evaluation reports prepared by 15 multilateral and bilateral organizations that focused on interventions to protect food security in times of crisis.

The report advances three overarching conclusions.

1 2 3

Interventions that support social protection were more effective when designed as a bundle of complementary modalities tailored to the local context and specific needs of target beneficiaries, and involving adequate partnerships.

Programmes supporting economic response and recovery of food insecure populations in times of crises were more effective when designed and implemented taking into account the realities of the context, and when they included clear scaling-up strategies with involvement of partners, private sector and political actors.

Interventions addressing food insecurity can better support social cohesion by broadening their scope to include not only core target beneficiaries but also the most vulnerable.

Patrick Breard was the lead author of the report, and Carlos Tarazona (FAO), Johannes Dobinger (UNIDO), Deborah McWhinney (WFP) and IOE’s Deputy Director, Fabrizio Felloni, formed the management group.

7

@Pexels/Ian Turn

IFAD-supported innovations enhance agricultural productivity in Morocco

I

nterventions on local production systems in Morocco, realized through IFAD co-financed projects, have contributed to increased agricultural productivity, sustainable management of natural resources and greater resilience to climate change. These and other issues were presented on 3 February 2021, as Moroccan senior policy makers, representatives of IFAD’s Near East, North Africa and Europe Division (NEN) Division and the Independent Office of Evaluation (IOE) met to discuss the findings of the country strategy and programme evaluation (CSPE), carried out in 2020.

resources, and to improving resilience to climate change.

“Many innovations have been promoted, often already tested in other contexts and introduced in areas intervention. They have helped to remove constraints in terms of productivity, resilience of ecosystems, empowerment of women and employment of youth”, stated H.E. Mohamed Sadiki, Secretary-General of the Ministry of Agriculture, Fisheries, Rural Development and Water and Forests of the government of Morocco (MAPMDREF), in his opening remarks.

Co-organized by the Ministry of Agriculture, Fisheries, Rural Development and Water and Forests of the government of Morocco (MAPMDREF) and IFAD’s IOE, in collaboration with IFAD’s NEN Division, the on-line virtual workshop brought together representatives of public sector institutions and national stakeholders, multilateral and bilateral partners, and selected private sector organizations. Over 100 people attended the virtual event.

Key to the achievements of IFAD’s co-financed projects in Morocco has been the improvement of rangelands management combined with efforts aimed at strengthening crop and livestock production systems. These interventions have contributed to improving agricultural productivity, to promoting sustainable management of natural

8

“IFAD’s investments have contributed to modernised production systems. They have promoted soil and water conservation; increased the access of small-scale producers to irrigation; and helped farmers to increase their productivity – of olives, apples honey and meat – by as much as 50% in some cases” stated Mr. Donald Brown, Associate Vice-President, Programme Management Department, IFAD.

During the event, meeting participants addressed the main findings, recommendations and issues emerging from the CSPE. Discussions centred on how to promote local products, including by supporting their continued access to markets in order to ensure sustainability of agricultural production systems. In this context, participants also focused on the numerous technical and institutional inno-

vations developed within the framework of each project thanks to partnerships with public and non-public actors.

“Remarkable results have been achieved in terms of human and social capital development, targeting and empowering of rural women and youth. Important results relate also to the infrastructure development through the implementation of more than 530 Km of irrigation networks and management of 14500 ha of plantations and crops and to the improvement of production techniques and yields”, underscored Ms. Dina Saleh, Regional Director, NEN, IFAD. During the 12 years (2009-2020) considered by the evaluation, the socioeconomic situation of Morocco has improved and the incidence of poverty has decreased significantly. Nevertheless, the grip of poverty remains high in rural areas, particularly in IFAD’s geographic areas of focus, where it stands at 9.5%, compared to 1.6% in urban areas. This, notwithstanding that the agricultural sector employed 38.7% of the labour force in 2017. Challenges thus remain, and call for increasing actions towards women and youths, strengthening rural organisations sustainably, as well as a more proactive involvement of IFAD in the formulation of agricultural policies and strategies.

Looking forward, workshop participants focused on avenues through which rural organizations of women and youth may evolve into modern agricultural entrepreneurs, and discussed opportunities for mobilizing external partners in order to further support and upgrade agriculture systems in disadvantaged and less attractive geographical areas.

“The recommendations emphasize the sustainable and sustained development of agricultural production systems, in line with the promotion of local agricultural produce, by women and youth, and sustainable access to interesting markets, in partnership with the private sector”, noted Fabrizio Felloni, IOE Deputy Director.

Mohammed Achaari and Rajaa Alem, joint winners of the 2011 Arabic Booker Prize @wikimedia_commons

Moroccan Poet wins 2020 Argana International Poetry Prize* Moroccan poet, Mohamed Achaari, has won the 15th annual Argana International Poetry Prize for the year 2020. Achaari’s works “have contributed for more than four decades to the consecration of writing as resistance by widening the spaces of freedom in language and writing through a poetic practice placing freedom at the center of its interests,” Morocco’s House of Poetry said in a statement.

The poetry commission said Achaari’s poems have helped to free up spaces and language in Morocco in favor of “values and in the interest of life.”

The poet was born in 1951. He studied law and administration and held several positions in political and cultural journalism. His work includes several collections, including “Journal du feu et du voyage” (Journal of fire and travel), “Biographie de la pluie” (Biography of the rain), and “Le livre des fragments” (The book of fragments).

Morocco’s House of Poetry launched the award in partnership with the Capital Private Equity (CDG), and the Ministry of Culture. The award seeks to pay tribute to artists by recognizing their contributions to society. *source: moroccoworldnews.com 9

2019 National Evaluation

FEATURE ART I CLE 10

NEC Conference 2017 @UNDP_IEO

n Capacity (NEC) conference @UNDP_IEO

NEC conference 2015 @UNDP_IEO

NEC Conference 2013 @UNDP_IEO

NEC Conference 2011 @UNDP_IEO

DR INDRAN A. NAIDOO: Captain on board I

OE has a strong crew, a crew that is now even stronger with the arrival of its new captain. In March 2021, Dr Indran A. Naidoo officially joined IOE to serve as Office Director. Indran is a globally recognized thought leader. Manager, author, instructor, over the past 25 years Indran has spearheaded international reform initiatives, transforming the oversight function across the United Nations system and the public sector. Multiple international journals document his work and accolades.

11

Today, unchartered waters lie ahead for Indran and his IOE team. These are challenging times, uncertain times. The COVID-19 pandemic has recoiled evaluators far from the field, placing projects and primary stakeholders seemingly out of reach. In the midst of the pandemic, IFAD is changing, evolving at mesmerizing pace.

Forty million people are set to benefit from IFAD-funded interventions by the year 2030 – that is double the number of those reached in 2020. Forty-five percent of IFAD staff are gearing-up to be field-based, as proximity to governments, partners and primary stakeholders becomes ever stronger. Ambitious targets, no doubt, supported by an unprecedented increase of 40% in member contributions and a new, resource-efficient financial architecture.

@pexels/artem @pexels/Thor Garlan beliaikin

US$1.55 billion in new contributions and US$1.2 billion in borrowed resources to finance a Programme of Loans and Grants of US$3.8 billion. A target for co-financing of 1.5 dollars for every dollar of IFAD financing. US$200 million for the Private Sector Financing Programme (PSFP), and US$500 million for the Enhanced Adaptation for Smallholder Agriculture Programme (ASAP+). Altogether, this will enable IFAD to expand its Programme of Work to US$11.1 billion in IFAD12, compared to a target of US$8.4 billion in IFAD11.

Will this affect IOE and influence its work? Most certainly, yes. The real question, therefore, is “how?” What opportunities might this scenario unlock for IOE: is IOE ready for ‘the new world’?

Independent Magazine sat down for a ‘virtual coffee’ with Indran to chat about these issues. This is what the new Director had to say.

Good afternoon, Indran.

Year of Evaluation, 2015 Glass Flame

Good afternoon, Alexander.

The world is changing. Or, maybe, we should say that the world has changed. We have a new, global focus on agricultural development, on the one hand, and an era-defining pandemic on the other. How do you see IOE adapting to the challenges and opportunities that this new international environment brings?

Yes, the world has changed fundamentally, from both a development and an evaluation perspective. I see IOE as well poised to engage with the new challenges. It will be quite easy for the office to reconfigure itself to support government response efforts to the pandemic and, more specifically, to help make IFAD a more effective organization moving ahead. This becuase IOE is a mature office, an office with a very good international track record and an excellent cohort of professionals. In this regard, I thank 12

former IOE Director Oscar Garcia, now UNDP IEO Director, for his leadership and contribution to IFAD evaluation. I look forward to ongoing collaboration as a member of the United Nations Evaluation Group.

Specifically in terms of this pandemic, two elements need to be taken into account. First, the pandemic has affected attainment of the SDGs, many of which are pertinent to the mandate of IFAD. Second, it has triggered a process of reverse urbanization. To survive, people are moving back to the field, closer to the land, to offset the cost of expensive in-city accommodation. In this context, food security has become even more important. We work with the rural poor; the most marginalized, the most vulnerable. Therefore, whilst this is a crisis of unbelievable proportions, it opens great opportunities for IFAD to increase the impact of its actions people’s lives and livelihoods. The question, and the challenge, becomes how to increase and optimize IFAD’s offerings to the governments around the world.

This global focus on agricultural development has also resulted in the increased funding you mentioned.

Today, IFAD is a busy organization, with high demands and a growing mandate. In this context, the assumption that evaluation takes up time is going to be a challenge. This is a misconception. Evaluation needs to be looked as part of the overall operational effectiveness of an organization, even though it stands aside for obvious reasons that are strategic. Evaluation is not an additionally; it is a part of good management to have review and feedback, so that one can adjust.

IFAD will need to adjust, and demonstrate that it is doing so. This because increased resources come with questions about the value for money that recipient countries get from IFAD. During my tenure, I will improve methodologies, so that we can capture the value-for-money proposition. I will be travelling, moving, working closely with the office of audit and with all partners in the professional networks to build-up a methodology that will demonstrate the value proposition of IFAD. It is extremely interesting that you identify opportunities inherent in this period, nested in the midst of the COVID-19 crisis, including avenues for organizational transformation.

This being the case, do you think that flexibility provides an opportunity for evaluators to remain relevant and to learn from practice around transformation? If so, how?

Yes, flexibility is pivotal in this new era. Among evaluators, there is a tendency to operate in a predefined mode, to make assumptions about predictability, and to use methodologies in a very standardized fashion. This is and will remain important for the rigour of our work. However, we are moving into an environment where the entire evaluation oversight architecture is changing. The information needs of funders, of partners, of governments has changed.

Information turn-around is key.

This means that, as evaluators, we need to be flexible, agile; we need to be able to work within

Dr Naidoo receives evaluation torch from US State Department and US Foreign Assistance @UNDP_IEO

13

a shorter timeframe, whilst continuing to use rigorous methodologies. We know that the pandemic has affected our ability to travel. However, with big data, with the use of information communication technologies – and IOE has made great progress in this – it is possible to extract and customize information.

It is a fine balancing act. Flexibility is important, as I said, but, at the same time, we must not lose sight of the principles that govern us. We have a set of norms, standards and values for evaluation – that is what ensures our credibility. Our independence is what gives us access to governments and donors around the world. The IOE Director is accountable and reports directly to the Executive Board. This means that the information we deliver is free from any external influence. Independence does not mean isolation, though. We engage with all stakeholders, and seek to do so increasingly. Through the evaluation process, we produced points of contact, intersection and reflection. It is only through this rubric that you can trigger opportunities for organizational learning.

While IOE gears-up to seize the opportunities that you mentioned, IFAD is responding by pushing for greater proximity to the field. How do you think decentralization will affect IOE’s strategic direction?

Decentralization is critical. IFAD claims to be a field-based organization, with presence across the globe.

The fact that IFAD is moving its services closer to the field, closer to the governments, closer to the countries themselves, means that it understands very clearly that it cannot be effective by being based at HQ, in Rome. This is very good for IOE. IFAD’s greater localized capacity will enable IOE to understand better its operational environment, and to develop findings, conclusions and recommendations that are more context specific.

As we know, IEO evaluates both government and IFAD performance. Decentralization gives us far greater latitude to deepen our analysis, because it will help us better understand the exact context where the organization works. We privilege fieldwork, ground-truthing and validating. We do not like doing

Indran A. Naidoo Promotes Evaluation Capacity building, through networks and teaching.

Believes strongly in advancing evaluation quality and oversight collaboration. ▶ Led the National Evaluation Capacity of the IEO of UNDP from 2013-2019 to become the largest global event by government participation. ▶ Taught and presented keynotes since 2009 at the International Program for Development Evaluation Training, Canada and Switzerland. ▶ Founding Board member of the South African Monitoring and Evaluation Association (SAMEA). ▶ Advanced evaluation in Africa through co-hosting with the Public Service Commission of South Africa the African Evaluation Association Conference 2004, focusing on the link between democracy, development and evaluation. ▶ Spearheaded the first institutional effectiveness assessment between evaluation and audit at UNDP. 14

evaluations from afar. The COVID-induced remote working modalities are temporary remedies. We are using them to cross pandemic-infested waters. I am sure that we will retain some of the methods and innovative approaches that we have developed, as we move into the future. However, nothing replaces an evaluator’s ability to be personally on the ground, to talk to beneficiaries, to see projects and understand the value additionally of IFAD in situ.

I commend IFAD for moving ahead with its decentralization strategy. It will help IFAD 12. It will clearly demonstrate to funders across the globe, who contribute to the organization’s multi-billion dollar budget, that IFAD takes service delivery seriously. IFAD is all about results delivery.

Moving from the strategic to the more operation level, you mentioned that IFAD’s enhanced field presence would help pave the way for IOE to engage increasingly with rural communities and government counterparts. Will this also lay the groundwork for greater uptake and use of its evaluations?

I do believe it will. Remote evaluations have limitations. Their main limitation is that they tends to rely on secondary and tertiary information. Working in an organizational environment that is more field based, closer to the ground, will give us the platform to engage with IFAD infrastructure at the local and community level. It will allow us to sense the happenings, developments and constraints.

We need to understand that any development intervention is complex and difficult. Very seldom do development interventions work to plan. When you move into an operating environment, you meet resistance, you meet complexity and you find that many of the design features that you put into place do not work in a particular context due to political, social, cultural, environmental and other factors. For this reason, as an evaluator, unless you have the proper linkages between HQ, regional and country level, you will not gain access to the right information. In all honesty, my experience has taught me that access to information is the biggest challenge in any evaluation portfolio.

World Bank partnership @UNDP_IEO

Evaluation Advisory Panel, Annual Meeting 2019 @UNDP_IEO

15

Access to information is also constrained by the fact that, at the country level, record keeping tends to be sparse. In many cases, when we conduct evaluations, we are actually constructing the information.

For this reason, I think that decentralization provides an opportunity to improve information management, not only within country offices, but also within agriculture ministries and governmental agencies.

Moreover, there is an opportunity for IFAD to design monitoring and evaluation curricula to build local capacity for its programmes and projects. I am exploring this possibility with IFAD senior management, given that similar initiatives require close collaboration with management and very strong country-level partnerships. In the long run, if the governments have stronger monitoring and evaluation systems for their rural and agricultural programmes, it will make our life easier because we will be able to draw data from those systems. I am quite confident that, during my tenure, we will build on the existing foundations, and expand evaluative capacity not just across IFAD as an organisation, but also within relevant government institutions in the field.

Partially in response to the strategic and operation opportunities that you have discussed so far, IOE is developing a multi-year evaluation strategy and evaluation manual. How will these new resources and IOE’s upcoming products, including the thematic evaluations and cluster evaluations, fit into the grand scheme of things?

African Evluation Conference 2004 @PSC of South A

I fully support the strategy and multi-year programme; it allows looking at evaluation products over a longer period. I am very excited about this approach. Working based on the calendar year is inadequate; time goes fast, and one is unable to sequence multiple activities. If you look at the replenishment cycles, they are not one year bound. The more IOE can align its work programme to IFAD’s cycles, the greater the synergies.

We want to improve the evaluation culture across the organization. It is UNDP Administrator Achim Steiner not just about IOE, it is about the entire organization. IOE has a role to visits IEO @UNDP_IEO play but it is one role, it is not the entire role. We need to take a modest view, and recognize that IOE does not have all the answers and expertise – we too can learn. Learning happens when we take an interactive approach, when we are reflective, when we listen.

“We want to improve the evaluation culture across the organization. It is not just about IOE, it is about the entire organization.”

Each of the new products you just mentioned has opportunities for learning, which will enable us to look at evaluation services. Evaluation, in fact, is not just about the production of reports; it is also about the facilitative role that evaluation plays. When we conduct evaluations, we are stimulating conversations across agencies and government entities, creating opportunities to reflect on what has been done and why. This is part of the learning process, which happens through the rubric of accountability.

We have started this process by inviting senior management to not just to comment on approach papers, but also to engage in a dialogue, so as we move through the evaluation process we have a common vocabulary, a common set of concepts and understanding. This way, the final report does not become the ‘be all and end all’. The journey to get to the report is as important as the report itself.

16

Africa

We also are engaging the regional offices, and we will finalize our plans based on what we hear from them. This means that some evaluations might be dropped, some might change, some might be brought forward and some might happen later on. Buy-in is essential.

Ray Rist and Dr Naidoo, 2015 IPDET guest speakers @IPDET

NEC 2019 UNDP IEO Team

“I am quite confident that, during my tenure, we will build on the existing foundations, and expand evaluative capacity not just across IFAD as an organisation, but also within relevant government institutions in the field.”

If I wanted to summarize, would I be correct in saying that you see IOE’s new resources and upcoming products as timely com modities that will help seize the opportunities of this ‘new global environment’ and – in doing so – to further mainstream and internalize evaluation approaches in a way that fosters enhanced reflection and improved performance, results and impact?

Thank you, Alexander. You have summarized it perfectly. Yes, these are the concepts that I am going to be pushing and advancing. The focus is on the products, but also on the intersection nodes between IOE, governments, management and the Executive Board. Each of these becomes a point of reflection on how to improve the quality of IFAD’s work. The focus has to be on embracing new ways of thinking, on piloting new methodologies that get products out quicker whilst increasing opportunities for conversation. Let me explain.

What I would like to advance in IFAD, as part of an overall mandate, is an improved evaluative thinking, which is much more reflective, much more focused on testing claims. To be quite honest, there is a tendency in all international organizations to make big claims: ‘reduce poverty’, ‘improve quality of lives’, ‘reduce discrimination’, etc. That is fine; it is important, it is normative. However, since these organizations receive funding, evaluators must test these claims. The outcome of IOE’s testing depends on how careful and circumspect IFAD is when it makes its claims. For this reason, we would like the organization to be modest in setting its goals, and then achieve more.

In this context, the link between policy and implementation within IFAD is critical. I am having conversations with members of the Executive Board and all senior management, because I wish to understand the thinking that drives the policy agenda, which informs programme design and project implementation. Ultimately, this will allow me to better align and adjust IOE’s strategy in order to implement a theory of change that truly improves IFAD’s the operational effectiveness.

Thank you very much, Indran.

You are welcome.

17

MAKING VOICES COUNT: Gathering community feedback in times of remote evaluation Johanna Pennarz, PhD, Lead Evaluation Officer, Independent Office of Evaluation of IFAD Prashanth Kotturi. Evaluation Officer, Independent Office of Evaluation of IFAD [First published on EvalForward.org]

I

n 2020, IOE undertook a project evaluation of the National Programme Community Empowerment in Indonesia. This was a Community Driven Development (CDD) project in Papua and West Papua provinces in the eastern-most part of Indonesia. These provinces are characterized by geographic remoteness, lack of infrastructure, scant presence of government structures, civil unrest in some parts and large indigenous populations.

Due to the Covid-19 outbreak, international travel was not possible. As a result, the evaluation had to adopt a methodology for remote data collection. Whilst the methodology provided some useful findings, it affected the scope and inclusiveness of the approach.

The approach: a vertical sampling and data collection The evaluation adopted a bottom-up approach. It did this by examining the experiences and perceived benefits of community groups, first through interviews and then triangulating these with the perspectives and views of project staff such as village facilitators, district facilitators and regency facilitators. The interviewing and data collection strategy followed the vertical facilitation structure, composed by community groups, village, district, regency and provincial facilitators.

18

Vertical sampling approach following the project’s facilitation structure

@Unsplash/Tyler Morgan

The villages were first selected using a simple random sampling procedure within each province using the village database available; then the evaluation selected a random sample of groups in each selected village. Once a community group was identified for interviewing, the respective village, district and regency facilitators supporting that group, directly and indirectly, were also identified and interviewed.

The questions for each level of the facilitation structure were formulated only after interviews for the level below were finished. The vertical interviewing strategy enabled the evaluation to review the issues emerging from the interviews at a lower level in the facilitation structure and validate them during the interviews at the higher level of the facilitation structure and vice versa.

Issues encountered in the remote evaluation process Inability to reach selected community groups due to lack of cell phone coverage. Linguistic diversity in Papua and West Papua and lack of fluency in lingua franca – Given the lack of reliable mobile coverage, the local consultants made an introductory call to set up an appointment with the groups so that they could be present in an area with better mobile coverage. Given the linguistic diversity of the indigenous populations, the evaluation team also sent the interview questions in Bahasa Indonesia before the appointment. This ensured that the evaluation team made the most of the limited time and network coverage. In cases where calls were not sufficiently audible due to network coverage, video recordings of answers on selected questions were provided through WhatsApp. Saturation of development interventions and inability to distinguish between programmes – Papua and West Papua have large public programmes and numerous donor-funded projects. The evaluation team found that the target groups were unable to distinguish between various programmes. To address this, the evaluation team used the introductory call to introduce the evaluation and to clarify the project it would focus on.

Speaking to female community members – The evaluation tried to reach female community members by phone. In some cases, however, the evaluation found that men took over the phone while women were speaking to the national consultants, and insisted that the evaluation team should speak to them.

The evaluation team had to be careful to avoid any serious consequences for the women given the high rate of domestic violence prevalent in Papua and West Papua. In those instances, the evaluation team would continue the interview with the man by asking a question or two before closing the call. The community member was then replaced randomly with another female community member to ensure integrity of the sample.

Final reflections on the evaluation methodology CDD gives control of decisions and resources to communities. They are expected to make informed decisions about how they want to use local resources, who will benefit and how they will benefit. Therefore, they should participate in the evaluation from the outset. Participatory M&E would have been a key element of a mixed methods suite of evaluation tools. Ideally, people’s indicators should become the most important indicators of change.

19

@Unsplash/Hasib Matiur

IFAD co-financed project in Bangladesh is a climate resilience pioneer

T

he Coastal Climate-Resilient Infrastructure Project (CCRIP) co-funded by IFAD, the Asian Development Bank and KfW Development Bank, and implemented in Bangladesh by the Local Government Engineering Department (LGED) has proven a pioneer in climate resilience efforts. This, according to the latest report of the Independent Office of Evaluation of IFAD (IOE), released on 17 March 2021.

The CCRIP was one of the first LGED projects to integrate climate resilience features in infrastructure and to develop a network of small, medium and large roads and markets as the basis for rural economic development in Bangladesh. Other innovations included research on environmentally friendly technologies and the testing of new approaches to women’s empowerment.

The newly released IOE report notes the value of this pioneering approach, and discusses its impacts on the livelihoods of the 3.7 million people who reside in the CCRIP market catchment areas. Benefits include an 11% increase in incomes in market catchment areas, a reduction in food insecurity, better management of local markets, and more connected and vibrant rural communities and markets.

These results were driven by increased sales of agricultural outputs, and were made possible by virtue of improved road and market infrastructure

20

– able to withstand monsoon flooding and extreme weather events. More resilient roads and markets led to a significant increase in the activity of traders, producers and transport providers, and enabled year-round access to services for households in remote rural communities.

The IOE project performance evaluation began in February 2020, just before the COVID-19 pandemic took hold. In light of the travel bans and social distancing requirements introduced to fight the pandemic, IOE decided to change its approach in conducting the PPE to perform its first fully remote evaluation. Various adaptations were made to address the associated methodological, practical and ethical concerns. These included the leveraging of national consultants and of modern technologies, especially Geographic Information Systems and satellite imagery.

Looking ahead, the IOE report highlights the importance of accompanying investments in infrastructure with broader support for climate-resilient livelihoods, tailored to project area contexts. Other recommendations include the need to undertake activities to enable value chain development, enhance women’s participation in labour markets, and engage with central and local governments to enable the development of policy responses and strategies to deal with systemic issues related to market leasing and maintenance.

Bangladeshi teenager awarded Children’s Peace Prize for fighting cybercrime*

A

1 7-year-old Bangladeshi teenager, Sadat Rahman, has been awarded the prestigious 2020 KidsRights International Children’s Peace Prize for taking action against cyberbullying and an online crime against children. Rahman developed a mobile app to help teenagers report cyberbullying and cybercrime in the western district of Narail in Bangladesh.

“Serious action needs to be taken right now. Teenagers continue to remain vulnerable to online crime and cyberbullying, particularly in the times we live in,” Rahman said in a remote interview.

The ceremony, hosted by the Netherlands-based KidsRights foundation, was held online this year due to the ongoing coronavirus pandemic.

Rahman’s mobile app is called Cyber Teens and puts children in contact with a team of youngsters, called Narail Volunteers. The team then gets in contact with the local police officers and social workers and helps the victims out of situations of cyberbullying.

The app also has a section of helpful tips and hints about different kinds of online behaviour, and tips about how to identify and avoid getting in contact with sexual predators of the internet.

As of now, the app has been downloaded around 1,800 times and the volume been successful in resolving nearly 60 complaints of youngsters so far.

On the basis of these complaints, eight criminals have also been arrested by the local police offi@Unsplash/Neil Soni cials. The complaints received had adults sending inappropriate messages and pornographic content to children “The idea started after a 15-year-old girl committed suicide because of online bullying,” Rahman said. “I decided that teenagers needed help and that we should take action to try to avoid other children facing the same tragedy.”

Rahman has received prize money of 100,000 euro ($118,000) which he plans to use for investment for projects, and further develop the app across Bangladesh “and hopefully to serve as a model for the rest of the world.” *source: www.wionews.com

21

AVERTING A ‘TRAIN WRECK’: Taking stock of environmental consequences of development interventions Nanthikesan, PhD, Lead Evaluation Officer, Independent Office of Evaluation of IFAD [First published on EarthEval..org]

I

t is widely recognised that urgent action is needed to minimise and reverse the damage that human actions, including development activities, have caused to the environment. Across the UN system, efforts are under way to develop common programming safeguards to prevent environmental harm. Parallel efforts among evaluation units to assess the environmental consequences of development actions are very much needed.

IFAD is one of the few exceptions. Over the past 10 years, IOE has gradually institutionalised environmental and social considerations in all its evaluations. Today, its reports are required to assess the performance of projects along a prescribed set of evaluation criteria. These include how well the interventions promoted environmental and natural resources management and strengthened adaptation to climate change.

How did IFAD get there?

Motivation: On the one hand, environmental agencies, such as the Global Environment Facility, and donors, such as the United Kingdom, provided grants for IFAD to test how best to integrate environmental considerations in its projects and, eventually, how to mainstream the practice. On the other, the leadership in IOE shared this recognition, and began to mainstream environmental consequences as an evaluand in all its evaluations.

The system: While the specific pathways may vary from one organization to another, three key guiding principles become evident. First, the mainstreaming the evaluation of environmental consequences should be systemic, not an ad hoc choice of individual evaluation managers. Second, a system does not magically appear in one swoop, but evolves over time to best address organizational needs. Third, this system is likely to be robust and sustained if accounting for environmental consequences is a shared concern among management, programming units, evaluation units and governing bodies.

In IOE, this system that ensures integration of environmental consequences has five interlocking elements: 1. Mainstreaming environmental (and social) considerations in programmes was declared a corporate priority.

2. This corporate priority is mirrored in the evaluation policy, which stipulates that IOE would prepare an evaluation manual. The manual integrates environmental consequences into its evaluation criteria.

3. Capacity and resources are allocated to assess environmental effects.

4. Internal peer reviews are carried out to assess the quality of all evaluations, including by focusing on the coverage of environmental consequences.

5. The responses of IFAD senior management to evaluation recommendations are tracked, including those related to environmental consequences, and progress in implementation is reported to IFAD’s Executive Board.

22

[KEY: EC = Environmental Consequences (intended and unintended)]

Challenges and Way Forward IFAD and IOE have institutionalized all these elements, which are expected to reinforce each other, and thereby offer resilience to organizational changes. Clearly, no system is perfect. IOE continues to face a number of challenges in its path-breaking effort to mainstream evaluation of environmental consequences of IFAD interventions. For instance, the depth of coverage of environmental effects varies across evaluations. As pointed out earlier, not all evaluations can afford the capacities and time required to accomplish the task. Another challenge is the proliferation of mandatory evaluation criteria, which considerably adds to the workload of the small office.

The elements of this ‘system’ continue to evolve in IFAD and IOE. For instance, IFAD’s commitments continue to ratchet up the importance assigned to mainstreaming environmental considerations in its interventions; and the evaluation policy and manual are in the process of being updated with the understanding that the criteria will remain the same and guidance on assessing environmental consequences be strengthened. These changes are expected to strengthen the existing efforts to mainstream evaluating environmental consequences of IFAD interventions.

The elements of IOE’s system may not always be applicable to other organizations. The specifics associated with each element may vary across organizations – for instance, mechanisms for quality assurance, or platforms for learning and accountability.

In short, there is no blueprint, no one-size-fits-all path to follow.

The hope is that IFAD’s experience can serve as a compass to point to possible pathways for others who are embarking on this important endeavour.

23

@Unsplash/Thijs Stoop

FURTHER READING

Evaluating Environment in International Development

The second Edition of the book Evaluating Environment in International Development is now available here in open access format.

The book provides an in-depth perspective on evaluating environmental and sustainability issues in developing countries. The main argument is that a sustainable environment is at the centre of humanity’s future. Evaluating environmental sustainability is thus presented as the thread that intersects and connects global challenges and guides humanity’s response. The book has a foreword by Michael Quinn Patton (former President of the American Evaluation Association), and messages from Oscar Garcia (Director, IEO, UNDP) and Jyotsna Puri (Director, Environment, Climate, Gender and Social Inclusion Division, IFAD).

Within this rubric, the volume addresses the challenge of identifying the intended and unintended consequences of development interventions on the environment, and calls for a systemic view of the interaction between natural and human systems. The strength of the book lies in its ability to bring together state-of-the-art thinking in evaluation related to environment, while combining it with real-world experiences and lessons on the ground.

This second edition follows the first edition, available here. This had forewords by Yolanda Kakabadse (President, World Wildlife Fund International) and Dr Indran Naidoo (Director IOE).

Related readings include the research paper entitled Local, National, Global: Evaluative Evidence of Scaling up the SDGs, available here, presented by Dr Ulitto and Dr Naidoo during the Annual Meeting of the American Association of Geographers, on 3 April 2019.

24

Community-based infrastructure among most successful IFAD investments

T

he construction of irrigation and drinking water infrastructure, roads, bridges and other forms of transport to and from markets are key ingredients of agricultural development programmes – and among IFAD’s most appreciated investments. These and other issues were presented on 26 March 2021, as IFAD country directors, technical experts and senior representatives of IOE met to discuss the findings of the 2020 evaluation synthesis report entitled ‘Infrastructure at IFAD (2001-2019)’.

With 30% of all approved IFAD funding having gone towards infrastructure projects during the past twenty years, these investments constitute a very significant share of IFAD’s portfolio.

“Infrastructure plays a pivotal role in IFAD’s 2016-2025 Strategic Framework, which sees enormous potential in infrastructure investments as they support agricultural commercialization and market access of the rural poor”, stated Dr Indran Naidoo, IOE Director.

The on-line webinar, which was open to the public, offered an important opportunity to share lessons and experiences on infrastructure based on the evidence collected from 35 IFAD-funded projects. During the event, IFAD country directors and technical advisors provided first-hand insights into issues related to the design, procurement and implementation of infrastructure investments on the ground.

These perspectives were complemented by those shared by a panel of distinguished experts, who paved the way for a thought-provoking discussion on IFAD’s future projects by presenting options for green and pro-poor infrastructure investments.

Discussions centred on the positive track record and added value of IFAD-financed infrastructure projects. Specific issues included

the importance of co-financing partnerships to provide infrastructure at scale, IFAD’s comparative advantage in the provision of small-scale, climate-smart and pro-poor infrastructure, and the increasing demand for infrastructure investments in partner countries, particularly in middle-income countries where the decreasing availability of concessional loans and grants drives the demand for productive investments.

Addressing the workshop, Ms. Meike van Ginneken, Associate Vice President of the Strategy and Knowledge Department, and Representative of IFAD senior Management, highlighted the importance of backstopping and post-construction support facilities for institutional sustainability, and underscored the associated challenges and opportunities.

In light of the above, experts recognized the urgency to reconcile IFAD’s strategic infrastructure approach with its infrastructure support capacity. Other areas that would require future attention include the decrease of investments in drinking water, the need for sustainable arrangements for infrastructure ownership and maintenance, and the limited availability of specialized technical staff and capacity to track the performance of infrastructure investments from design through to implementation and completion.

Looking ahead, the meeting concluded that increased future borrowing for infrastructure would need to remain closely linked to IFAD’s mandate to facilitate better access and sustainability for IFAD’s target groups, to enhance livelihoods resilience, to minimize elite capture and to safeguard the interests of poor and vulnerable groups. In this regard, the ongoing transition from government-owned and maintained infrastructure to more inclusive and stakeholder-owned models remains a top priority.

25

IFAD-funded projects contribute to higher household incomes in Uganda

R

ural development projects co-financed by IFAD have helped to increase household incomes in Uganda over the past seven years. This and other issues were presented on 5 February 2021, as Ugandan senior policy makers and representatives of IOE met to discuss the findings of a country strategy and programme evaluation (CSPE), carried out in Uganda in 2020.

of Uganda and IOE, in collaboration with IFAD’s East and Southern Africa Division, the on-line workshop brought together policy makers from the Ministries of Agriculture, Animal Industry and Fisheries, and Local Government, and development counterparts, civil society organizations and private partners. Over 70 participants attended the virtual event.

Key to the achievements of IFAD’s co-financed projects in Uganda has been the building of infrastructure and market support services, which have reduced transport costs and increased market prices due to improved road access. Equally important have been IFAD’s investments in rural finance, which have supported regulatory reforms and linkages between local savings and credit groups and service providers.

Hon. Dr Ajedra Gabriel Aridru, Minister of State for Finance, Planning and Economic Development, delivered welcome remarks and an opening statement for the event. Mr Donal Brown, Associate Vice President, Programme Management Department of IFAD, gave a statement, followed by Mr Fabrizio Felloni, who pronounced an introductory statement. Ms Chitra Deshpande, Senior Evaluation Officer, IOE, then presented the main findings and conclusions emerging from the evaluation, as well as recommendations for future work in Uganda. After a question-and-answer session, Ms Lakshmi Moola, Country Director, and Ms Sara Mbago-Bhunu, Regional Director, East and Southern Africa Division, IFAD, gave statements on the future directions for the IFAD programme in Uganda. Closing remarks were delivered by Mr Felloni and the Hon. Bagiire Aggrey, Minister of State for Agriculture, Animal Industries and Fisheries. Ms Maris Wanyera, of the Ministry of Finance, Planning and Economic Development, acted as Master of Ceremony.

“Our evaluation shows that the IFAD-supported projects have been overall successful at reaching poor rural people in Uganda. They have resulted in increased productivity and better access to markets and rural financial services. The projects have also contributed to growing productivity and incomes, and strengthening individual and group capacities”, stated Mr Fabrizio Felloni, IOE Deputy Director.

Organized by the Ministry of Finance, Planning and Economic Development of the government

26

Meeting participants discussed the main findings and issues emerging from the CSPE, reflected on opportunities and challenges in the IFAD-Government partnership and addressed strategic priorities for IFAD’s continued financing in Uganda.

In particular, during her presentation, Ms Deshpande noted that while projects have contributed to growing productivity and incomes, particularly through the value chain approach, climate variability is increasing and needs to be addressed more extensively to avoid negating the portfolio’s positive achievements on rural livelihoods.

Uganda’s economy has grown in the past 20 years, during which time agriculture has provided a quarter of the country’s GDP whilst employing 72% of its labour force.

However, multiple structural challenges constrain agricultural growth, including climate change and unsustainable natural resource consumption patterns.

To offset these challenges, government policy frameworks have sought to transform agriculture into a commercially viable sector around a set of key value chains. However, notable challenges remain. Moreover, while overall funding for agriculture has fallen below the government’s international commitment, the rising impacts of climate change could setback IFAD’s achievements if not addressed promptly.

Looking forward, the meeting called for exploring ways to expand IFAD’s effective value chain approach to other commodities with greater beneficiary outreach potential, for mainstreaming climate change more extensively in IFAD’s future in-country investments, and for delivering more transformative approaches and interventions tailored to the specific needs of women and youths.

“Our evaluation recommends developing a strategy for knowledge management, partnerships and country policy engagement that is backed by sufficient resources”, underscored Mr Felloni.

@Pexels/Ketut Subiyanto

Smartphones and agriculture: are the times ripe in Uganda?* According to Dr Charles Malingu, Director of Innovations and ICT at the ExcelHort Consult and agribusiness Incubation Centre in Mbarara, Uganda, if the youth are to be enticed into embracing agriculture, then there is one tool that they can be hooked with: the smartphone! Malingu explains that the phone must be turned into a major farming tool. “We are using the phone to predict weather for example” he says.

Additionally, a smartphone can also be used to keep records of the farm. “Unfortunately, very few Ugandans use smartphones to their maximum technical abilities,” Malingu says.

At the Centre, in Mbarara, they are using the smartphone to carry out extension services with farmers.

“We give them weather predictions using the phone,” he says. Farmers are also able to monitor market rates, get planting tips and even identify pests and diseases using their phones.

According to the Uganda Communications Commission (UCC), there are over 20million active phones in Uganda, with over 5milion of these being smartphones. However, the number of active farmers with smartphones is not known. *source: www.newvision.co.ug 27

Innovative territorial approach bolsters living conditions in Niger

I

FAD co-financed projects have contributed to increasing rural incomes, and improving household assets and living conditions in Niger through an innovative territorial approach.

These and other issues were presented on 15 April 2021, as senior policy makers from Niger, representatives of IFAD’s West and Central Africa Division (WCA), and the Independent Office of Evaluation (IOE) met to discuss the findings of the country strategy and programme evaluation (CSPE), carried out in 2020.

“Our evaluation found that the country programme funded by IFAD has helped to improve the resilience of rural households and communities”, stated Dr Indran A. Naidoo, IOE Director, in his opening remarks.

Key to IFAD’s achievements in Niger has been the economic development poles approach. First adopted in 2012, this methodological innovation combines civil engineering with social mobilisation in all investments in production, road building and construction of commercial infrastructure.

Results achieved include the setting-up of five new agricultural markets – most of which are being well used –, and the construction and rehabilitation of feeder roads that facilitate better access of smallholder producers.

Co-organized by the Ministry of Agriculture and Livestock, the Ministry of Planning and the Ministry of Infrastructure of the government of Niger, and IFAD’s IOE, in collaboration with IFAD’s WCA Division, the on-line virtual workshop brought together a wealth of participants. These included representatives of public sector institutions, national stakeholders, research institutes, selected private sector organizations, and multilateral and bilateral partners, including international cooperation agencies, development banks, and UN programmes and funds.

During the event, participants focused on the main findings, recommendations and issues emerging

28

AT A GLANCE from the CSPE. In addition to discussing the economic development poles approach, stakeholders also commended the development of irrigated areas and the provision of seeds and fertilizers, in addition to water and soil conservation activities, including degraded land reclamation. These interventions have made it possible to recover nearly 30,000 hectares of previously uncultivated land, and have contributed to improving agricultural production and productivity, with the yields of different vegetable crops recoding increases of up to 36 per cent. In this context, the combination of environmental restoration activities with ‘food-forwork’ initiatives has created incentives for communities to engage in labour intensive activities and provided collective benefits in the long term.

“A key area of success has been the regeneration of local vegetation, and the rehabilitation of land through simple techniques that local communities are now able to master and maintain”, Dr Naidoo underscored in this regard.

Since 1980, IFAD has financed 14 projects in Niger, totalling US$ 739 million in loans and grants.

To date, eighty-three per cent of the population in Niger lives in rural areas. In 2018, agriculture represented 39 per cent of GNP, and was the backbone of the economy. Niger is exposed to several sources of fragility, such climate-related risks and drought, and security threats, including along its borders with Mali and Nigeria. The country also suffers from a high prevalence of child malnutrition, with a stunting rate of 47.8 per cent.

Looking forward, workshop participants noted the importance of striking a balance between the emphasis on large infrastructure investments for economic development poles, and the provision of basic services and activities that cater to the immediate needs of rural communities, such as access to drinking water, sanitization, and functional literacy. These interventions would also need to take carefully into account latent tensions between different natural resource users.

Ecuador: Country Strategy Programme Evaluation

I

OE recently published its second country strategy and programme evaluation (CSPE) in Ecuador. It covers the period 2009–2019, reviewing four loan projects and nine grants.

The evaluation found that IFAD’s performance in Ecuador has been moderately satisfactory.

The strategic design of IFAD’s programme was sound and enabled its activities to remain relevant despite the significant changes occurring in the country. There was an improvement in the access to assets and resources to support the diversification of Ecuador’s rural economy, through the promotion and strengthening of production and associative enterprises.

However, the lack of attention to marketing and market access limited the expected outcomes.

For the future, IOE’s recommendations include reinforcing the differentiated territorial approach with local stakeholders’ participation in project implementation, considered key in enabling access to differentiated markets and valuing Ecuador’s biocultural heritage; and promoting the sustainability of enterprises, one of the cornerstones of IFAD’s portfolio in the country.

29

@Unsplash/Andres Medina

BEHIND THE GLOSSY COVER: Six reflections on communicating evaluations Alexander Voccia, PhD, Coordinator, Evaluation Communication Unit, Independent Office of Evaluation of IFAD [First published on EvalForward...org]

H

ave you ever noticed that beneficiaries always tend to be smiling on the front cover of development reports? I sometimes wonder how realistic this is. From a communications perspective, it certainly paints a nice picture. Is it the right one, though? I guess the answer depends largely on the report’s target audience – or, otherwise stated, on its intended purpose. The intended purpose of an evaluation report is usually very clear, insofar as it stems from the principles that guide the organization’s evaluation processes. In the case of IFAD, for instance, the guiding principles of the Independent Office of Evaluation (IOE) are “accountability, learning, independence and partnership”. Yes, that’s right, “accountability and independence”: two words that raise alarm bells and normally drive people into hiding. So where exactly does this leave communications? In a bit of a conundrum, to be quite honest.

When compared to the commonplace tools used to promote most development reports, the communication approaches necessary to enhance visibility, reach and consumption of evaluation findings present a somewhat distinctive set of opportunities and challenges. It’s a whole different ball game. The unfortunate misconception of evaluators being a quasi-police service, who descend upon recalcitrant project staff from the safe haven of their ivory towers, where they spend most of their time blissfully unaware of the hardship of field operations, does not bode terribly well for the acceptance of their work – let alone for its visibility. And it gets worse. The need for evaluation offices to hold their organizations accountable, risks placing corporate and evaluation communication teams at loggerheads.

I must admit that this is a somewhat gloomy scenario. Is it exaggerated? Maybe. Is it inevitable? Certainly not. I recently carried out an in-depth assessment of IOE’s communication endeavours. The exercise draws inspiration from the best practices of the world’s leading multilateral development financial institutions, and builds on the generous insights of communications and knowledge management specialists from evaluation offices of international development banks, United Nations programmes and funds, and academia. After spending a decent amount of time dissecting findings, triangulating data and analysing trends, I decided to take a step back, and reflect on the underlying essence of what I had captured. What emerged were the following six reflections on how to address the recurring challenges inherent in communicating evaluations.

1.

Foster engagement – Maximize opportunities for stakeholders to interact with evaluation offices in a proactive way: stakeholder dialogue does not end with an evaluation report.

The first thing I noticed is that communication products should show how offices reach out to their stakeholders in an open and welcoming fashion, and make them feel empowered by seeking their inputs even after an evaluation is completed. This means that communication products need to open the door for an on-going, open and engaged discourse with stakeholders to ensure that they continue to feel connected 30

with the office beyond the end of an evaluation process.

2.

Build empathy – Create a personal connection, a sense of empathy between evaluation offices and stakeholders: less corporate image, more personal brand. I also found that evaluation offices need to show that there are real people behind the logo, people that care, people that have at heart the organization, its staff and the beneficiaries that it serves. This requires evaluation offices to be ‘liked’ by their stakeholders, thus overcoming their perceived investigative role, role that triggers in users a (tacit) desire to distance themselves from their work.

3.

Cultivate audience – Map stakeholders, maximize their breadth, depth and variety, and identify priority clients. My findings also suggest that evaluation offices should ensure that the reach of their products moves well beyond the closed circle of evaluation specialists and technical experts, to embrace much broader horizons. This means that evaluation offices should not only broaden their communication stakeholder base but also – prior to doing so – understand whom their stakeholders are, and determine how to group and classify them.

4. Feed the need – The right message for the right audience through the right product. What also emerged was the importance of connecting stakeholders with messages and communication products. Evaluation offices should first understand the type of information that their different stakeholders need. Second, they should find the right way to tailor content to match those needs; and third, they should identify the most appropriate vehicle to convey tailored content to selected target audiences.

This also means ensuring a ‘reactive-proactive communication balance’. This balance should respond to user interests and needs (reactive), while gradually modifying user consumption trends (proactive), with a view to securing closer synergies between consumption preferences and communication products.

5. Capture presence – Be seen as credible, reputable, authoritative and respectable. Be seen as present. Evaluation offices should show their stakeholders that the lessons, recommendations and messages from its evaluation reports generate interest and engagement among a variety of different audiences, thus fostering an appreciation of the scope of the evaluation function, and underscoring their crucial role in implementing said function.

6.

Bring value – Safeguard credibility; create opportunities for change. My final reflection is that focusing only on the good stuff is just not good enough. Offices should foster opportunities for critical reflection and dialogue among a variety of stakeholders and stakeholder groups through their communication products, in order to play a prominent role in the global development discourse. This means giving due visibility to both challenges and opportunities for improvement by leveraging intellectual independence – which is wherein the value added lies.

Far from presuming to be universally valid guiding principles, these are mere points for reflection, food-for-thought that underscores the need for a strategic approach to communicating evaluations.

For if it is true, as it is, that evaluation offices are uniquely positioned to foster critical reflection and dialogue among a variety of stakeholders, it is by embracing a strategic approach to communications that they are able to broker honest conversations for change – conversations that tackle opportunities for growth, and thus give strength and credibility to future organizational successes. @Pixabay/Sasint

31

More time needed to sustain innovative community-driven approach in Tajikistan

T

he Khatlon Livelihoods Support Project (KLSP) in Tajikistan featured an innovative community-driven approach, and promoted the use of new agricultural technologies for the local context.

However, the project was unable to deliver the holistic and sustainable approach to poverty reduction expected at the time of design. This, according to the latest report of the Independent Office of Evaluation of IFAD (IOE).

The KLSP was the first IFAD-financed project in Tajikistan. All participating communities, including 82 village organizations, benefited from some investments. Training and capacity building afforded the biggest contribution to rural poverty reduction. The provision of agricultural machinery also brought immediate benefits in terms of cost savings and income for the majority of villages targeted.

Provision of basic infrastructure

@IFAD/Shukhrat Igamberdyev

32

plement an exit strategy in partnership with local stakeholders.

Looking ahead, the IOE report notes the need to take a longer-term perspective to community-based approaches in the country, and to align said approaches with ongoing initiatives supported by development partners and the government agendas. Other recommendations include allocating sufficient technical and human resources to ensure a pro-poor and gender-focused approach that addresses local needs, including the need for water. @Unsplash/Frans Hulet

further contributed to improved living conditions, albeit in a smaller number of communities.

These results notwithstanding, the strong motivation of the participating communities and the effective partnerships established were not sufficient to meet the high expectations of beneficiaries, partners and stakeholders. The initial scope of the project proved overly ambitious, and the demand for community infrastructure overwhelming. At the same time, the assumption that village organizations would have sustained themselves was over-optimistic. These and other shortcomings might have been adverted if the project had been given additional time to fully utilize community resources and im-

Between 2000 and 2017, Tajikistan has made steady progress in reducing poverty and growing its economy. The poverty rate fell from 83 to 29.5 per cent of the population between 2000 and 2017, while the gross domestic product reached US$ 870.8 per capita in 2019 – which, however, remains the lowest among the Central Asian countries.

@IFAD/Shukhrat Igamberdyev

Tajikistan gears-up to celebrate 30-year independence with new cultural contest*

O

n 9 September 2021, Tajikistan will celebrate its indepdence. To commemorate this important milestone in the country’s history, President Emomali Rahmon recently announced the launch of three cultural contests.

To foster the broadest possible public participation in the event, relevant government bodies will hold at a high organizational level in all regions of the country. The Ministry of Finance has allocated the equivalent of almost USD 700 thousand (7.72 million Tajikistani Somonis) to support this important effort.

The three contests will be “Furughi Subhi Donoi Kitob Ast” (Knowledge Is a Shine of Dawn); “Tojikiston -- Vatani Azizi Man” (Tajikistan Is My Native Land); and “Ilm – Furughi Marifat” (Science Is the Beacon of Knowledge).

The goal of the first contest, “Furughi Subhi Donoi Kitob Ast”, is to boost public interest towards literature and thus promote the reading of books. To this end, the contest foresees the following three categories: connoisseur of Tajik classical literature; connoisseur of Tajik modern literature; connoisseur of children’s literature and folklore; and connoisseur of world literature. The winner of 33 each category will receive 25,000 Tajikistani Somonis. @Unsplash/Simon Sun *source: https://asiaplustj.info

IOE FAMILY

Farewell, Hans!

O

n 26 March 2021, IOE bid farewell to one of its most esteemed family members: Hansdeep Khaira. Well-respected by everyone, Hans became part of the family back in 2016, after first joining IFAD in 2013. His professionalism, commitment to results, welcoming personality and good spirit have permeated all his assignments, vibrantly shining through and lighting-up his work.

Having led numerous evaluations, ranging from impact evaluations in countries such as Georgia, Kenya and – more recently – Ethiopia, and project evaluations, including those in Guyana, Sri Lanka and – more recently – Uganda, Hans has had a long-standing impact also on the Annual Report on Results and Impacts (ARRI), of which he has been the lead author multiple times.

Always attentive to the context-specific circumstances of rural communities in different parts of the world, Hans’ empathy toward those most in need has been a defining element of his fieldwork and on-the-ground missions.

This work ethos is well-complemented by a professional background that speaks volumes. Hans has been working in the development field since 2002. His work experience covers over 35 developing countries in sub-Saharan Africa, Near East and North AWfrica, Asia, Latin America and the Caribbean.

The IOE family wishes Hans every success in his future endeavours with WFP. His marked sense of humour, innate calmness, team spirit, culinary advice and… cricket skills will be greatly missed!

All the very best, Hans!

34

35

Independent Office of Evaluation International Fund for Agricultural Development Via Paolo di Dono, 44 - 00142 Rome, Italy Tel: +39 06 54591 - Fax: +39 06 5043463 E-mail: evaluation@ifad.org www.ifad.org/evaluation www.twitter.com/IFADeval www.youtube.com/IFADevaluation

36