Independent Magazine - Issue n.4, 2022

LET’S TALK QUALITY ASSURANCE

Building utility through enhanced engagement

Quality endures, in all aspects of life. Evaluation is no exception. IOE takes quality assurance seriously – very, seriously. Only the very best will do. Rigor, precision and attention to detail are non-negotiable, and permeate the procedural, methodological and substantive robustness of the IOE approach.

In its pursuit of qualitative excellence, the IOE team draws from the contributions of a cadre of highly qualified professionals, with academic distinctions from some of the world’s most reputable institutions. We come from thirteen different countries, master seven different languages, and have direct field experience in every continent. Our areas of expertise cut across the full spectrum of the evaluation rubric encompassing, inter alia, international development, research, teaching and capacity development, strategic planning and communications, and programme and knowledge management and administration.

Team members make substantive contributions not only to their own areas of work, but also provide valuable feedback to multiple evaluations through the newly revised peer review processes. These processes ensure that the highest qualitative standards are met, as every product draws from the widest possible array of insights.

Building on these solid foundations, in 2021 our Director, Indran A. Naidoo, established the IOE Evaluation Advisory Panel (EAP) to further our Office’s contribution to IFAD’s organizational and development effectiveness. During their first year, the members of the EAP provided comments on the IOE Multi-Year Evaluation Strategy, the 3rd edition of the IOE Evaluation Manual, and the Thematic Evaluation on Climate Change Adaptation. In addition, between 2021 and 2022, they delivered seminars on their respective areas of expertise. Henceforth, the EAP members will meet annually with the IOE team, as well as with IFAD Management and Governing Bodies representatives, as required, to provide insights and recommendations on the work undertaken and future strategic directions for IOE.

In the margins of the EAP’s first annual meeting, in the afternoon of Thursday, 14 July 2022, Independent Magazine sat down with the five members of the Panel, for a thought-provoking conversation on the present, past and future of IOE.

Good afternoon, esteemed colleagues.

Good afternoon, Alexander

Could you briefly describe the role and functions of the Evaluation Ad visory Panel and how these fit into IOE’s quality assurance process?

Donna

Our role is really to provide diverse perspective. We’ve engaged in some substantive reviews related to strategic planning. Being here has really given us the opportunity to understand the structure of the organization and how IOE supports and interacts with the other parts of the organization.

Gonzalo

Evaluation is a never-ending story. You will never get the perfect evaluation system. That will never happen, ever. What happens is that you need to adapt, change and find new ways to use and produce evaluation because it has to do with exchanging with people and convincing people. Therefore, it is important to have various voices that can help you at various levels and in various circumstances to do the job in a better way. The good thing about the EAP is that we are a diverse group, and thus we can bring a variety of insights into the processes and work of IOE, including on how to improve current practices.

Rob

Our value added is at the strategic level, mostly. The overarching documents such as the evaluation manual, the multi-year strategy, the evaluation policy, those are the ones that we have all provided inputs to. Timewise, we cannot comment on everything that is happening in the office. It’s more on an invitation basis. The structural support is, therefore, really at the strategic level. For this reason, it has been very important for us to understand what the other parts of IFAD are doing, and how evaluation relates to and interacts with them.

How do you see your individual areas of expertise contributing to strengthening the quality assurance process of IOE’s evaluation products?

Rob

I’m part of the second wave of transformational change. Donna established this for social justice, and from 2015 onwards the nexus between development and environment became extremely important because of climate change, biodiversity loss and a broad variety of eco-system related problems. This is now termed as the ‘sustainability crises’ of the world, because we cannot sustain the way that we are currently living. I bring this additional perspective to the group and to the evaluation office.

Bagele

I have written a lot on indigenous methodologies. This is about how we can make sure that those voices that have been left out, in terms of informing our evaluation processes from design to dissemination of findings, can be brought to the table. Currently, there is a movement on decolonizing evaluation, and I happen to be one of the activists in this regard. I bring this to IFAD, giving my own experience in terms of how the organization can engage with decolonizing evaluation so that evaluation is relevant, the methodologies are legitimate, and what we are told is working actually is working. Unfortunately, most of the time we are told that projects are doing well, the outcomes are good, while on the ground very little has changed. The international evaluation community is beginning to realize that the world is very complex. We need to integrate knowledge systems, and now we are at a point where we are asking “how”. My role is just to share my own experiences in this regard, and advise on what can be done.

Donna

My work has been in the development of a transformative framework for evaluation. That work preceded the sustainable development goals (SDGs), which in turn preceded the use of transformation that we are hearing a lot about now with regard to the SDGs. I am delighted to see that this is a concept that has now come into prominence because when I do evaluations I want to design them in ways that are inclusive of a full range of stakeholders. Yes, we need to consider the funders and the programme people, but we also need to consider the people who have traditionally not had their voices heard, and not just in the provision of data, but also in the discussion around the important questions and the ways that we can collect the data so that it has validity within the context that we are working in. We need to be consciously aware of the power inequalities that are inherent in evaluation situations. How do we address those as evaluators and bring in the voices of people with disabilities, women and all those that have been pushed to the margins? This is what most of our programmes are designed for, those are the intended beneficiaries. What motivates me is to work with people with the goal of affecting this kind of change.

Gonzalo

What I bring is the knowledge and the experience that I have had in terms of helping to put together monitoring and evaluation systems. This is not only adding up evaluations, but it means looking at how you can produce and build a robust system. I have delivered this work at country and agency level. I think this is important because you may have a system, but within it there may be several areas for improvement. For instance, on the communications side, sometimes evaluators are not very clear in presenting results. That’s not very helpful for countries and policymakers. The way you communicate results, the way you address evaluation findings, is an area where evaluation offices, traditionally, can improve.

What do you see as being the most important points of reflection and feedback within IOE’s current quality assurance process?

Bagele

The biggest mistake that I really zoom in is that of excluding other knowledge systems. I see myself as contributing to the quality of evaluation from that perspective. Right now, the mistake that evaluation is doing is to show that millions of dollars have been spent on development programmes, and while very little has changed, evaluations have been saying that projects have worked very well. This means that the discipline has been focusing more on compliance and accountability. It is time that all UN organizations begin to reflect on this big mistake that is recurring. If the SDGs are going to really bring about change, then we need to look at this mistake and try to address it, otherwise the outcomes of the SDGs are going to be the same as their predecessors, the MDGs.

Donna

I would say that I’m a methodologist, that’s who I am, that’s what I do. Can I provide inputs from the beginning to the end of a process? Of course, I can. However, my strength and experience lie in the methodology. It is not just methodology, its methodology associated with social, economic and environmental justice. If we are going to talk about those as our goals, then I look at how do we integrate them into the decisions that we take about methodology.

Hans

Upward accountability is very important, and we need to look increasingly more into that, while not forgetting that we live in a system where many countries are funding UN organizations out of their tax payers’ money, and thus have an obligation to account for what is happening with this money. Therefore, we need both types of accountability to be able to balance interests in an equitable way and make institutions work.

Rob

There is also a micro/macro paradox in place, which is that at times there are very successful micro interventions that really succeed in bringing change, while the overall change in the country is not there, and even what is achieved is only temporary and does not continue beyond the lifecycle of the project. We see this especially in the environment-development nexus, where you try to address both and while some real transformations are possible, the overall trend is that the environment gets further degraded, we are going to have massive biodiversity loss, and we are running towards climate change. This, despite interventions that show how it could potentially be tackled. There is role for evaluation to point this out and not just be happy to report that a certain project had a temporary and limited results, but to highlight what is going in the wrong direction in countries and internationally.

What do you think are the strengths of the Office and going forward, which are the areas that have opportunities for growth vis-à-vis emerging priorities for evaluation?

Hans

It’s really important to build on IOE’s innovative practices. We are in a situation where we have had a pandemic that has pushed millions of people into poverty. This is an enormous challenge for IFAD. Getting innovative solutions and scaling them up is critically important in these times.

Gonzalo

We see strengths in the office, starting with the leadership from Indran that brings methods, technical knowledge and the focus on independence. There are areas for improvement whilst maintaining the independence. For instance, in the way we communicate and exchange ideas with others.

Donna

There’s a definitive strength in that evaluation is very integrally connected throughout IFAD. Every part of the organization that we talk to spoke about how they use and can use evaluation products, and sometimes that they wanted even more. For me, that speaks to the perceived value of evaluation, which is not always present in every organization. In terms of future opportunities, moving into a position where more data can be collected about the cultural barriers that are preventing the advancement of the people targeted. We need to foster the advancement of everyone, especially the most marginalized and the poorest of the poor, who have not had access to resources.

Bagele

For me the strength is in the existing structures, such as the evaluation policy – which is very progressive and has transformational and decolonizational intentions –, the multi-year strategic plan, which is also very progressive, and the evaluation manual, which spells out the methodologies. This is very transparent, and points to direction and intention. Above all, the strength is in the open mindedness of the leadership. In this regard, I can see how the EAP’s contributions are starting to influence and permeate the outputs of the office. Now, the challenge is in documenting innovative practices that come from what is being done. IOE has the strength and opportunity to come up with outputs such as an ‘indigenous evaluation framework’ that allows the marginalized to voice their concerns and be truly heard.

Thank you, colleagues.

Thank you, Alexander